Enriching the Data vs Filtering in Apache Spark
Explore data enrichment vs filtering in Spark for improved performance and scalability in cloud-native applications, with practical examples and insights from Capital One's loyalty use case.
Designing Streaming Data Pipelines: Best Practices - Cloud Native 2024
Explore best practices for designing streaming data pipelines, covering key concepts, challenges, and considerations for scalability in cloud-native environments.
A Data-Driven Approach to Product Discovery
Explore data-driven product discovery techniques, with insights from Netflix and Spotify, to effectively implement this approach in your own projects.
Empowering Fearless Data Engineering with Rust
Explore Rust's impact on data engineering, focusing on performance, safety, and real-time analytics applications. Learn advantages and future prospects of Rust in the field.
The Streaming Plane - Bridging Operational and Analytical Data
Explore the streaming plane concept, data gravity, and efficient data movement strategies for platform engineering. Learn about synchronous vs asynchronous serving and optimizing global data replication.
Building Modern Data Streaming Apps with Python
Explore modern data streaming app development using Python, focusing on key benefits and unique aspects of building efficient, scalable applications.
Building an IoT Monitoring App with InfluxDB
Aprenda a construir um aplicativo de monitoramento IoT usando InfluxDB e Flask, explorando replicação edge-to-cloud e técnicas avançadas de gerenciamento de dados em tempo real.
Nuclear Rust - Introducing Rust in Nuclear Software Industry
Explore Rust's application in nuclear software, focusing on safety, digital twins, and lessons from implementing Rust in reactor simulations.
Metrics that Matter - Moving from Easy to Impactful
Explore impactful metrics in Kubernetes environments, from simple to economic. Learn to separate signal from noise, understand predictability vs variability, and reduce cognitive load for better decision-making.
FLiPN Stack for Edge AI - Flink, NiFi, Pulsar
Explore FLiPN Stack for Edge AI: Learn about Flink, NiFi, and Pulsar integration for efficient data processing and streaming in IoT environments.
Simplifying Kafka Governance for Developers
Explore Kafka governance simplification using Klaw. Learn about architecture, approval processes, and practical demonstrations for efficient Kafka management in DevSecOps environments.
4 Tricks to Leverage Your Airflow Pipelines
Discover four powerful techniques to enhance Airflow pipelines: CSV-based DAG creation, sharable config utilization, entity-based autogeneration, and event-driven implementation.
Pushing Your Streaming Platform to the Limit - Conf42 Chaos Engineering 2024
Explore streaming platform benchmarking techniques, including key criteria, USE method, and OpenMessaging initiative. Learn to assess limits and gain valuable insights for optimizing performance.
Apache NiFi, Kafka, RisingWave, and Iceberg for Stock Data Processing with LLM - Conf42 Python 2024
Explore stream processing with Apache NiFi, Kafka, RisingWave, and Iceberg. Learn real-time analytics, event-driven applications, and integrating stock data with LLMs in this comprehensive overview.
Sketching Algorithms: Making Sense of Big Data in a Single Stroke
Explore sketching algorithms for efficient big data processing, covering concepts, design principles, and practical applications using open-source tools like Apache DataSketches.
Leveraging Apache Flight Python Client and InfluxDB for Time Series Data Management
Explore Apache Flight Python Client and InfluxDB integration for efficient time series data management. Learn about columnar storage, benchmarks, and real-world applications in data architecture.
Building Apache NiFi 2.0 Python Processors - Conf42 Python 2024
Explore Apache NiFi 2.0 Python processors for data engineering, covering features, use cases, and real-time generative AI integration with a live demo.
Streaming DataFrames: A New Way to Process Streaming Data - Conf42 Python 2024
Explore streaming DataFrames as a novel approach to process real-time data, enhancing efficiency and simplifying stream processing workflows for Python developers.
Best Practices to Manage AWS Streaming Data Pipeline at Scale
Learn best practices for managing AWS streaming data pipelines at scale, covering data distribution, large record processing, schema registry, and validations.
Building a Modern Enterprise Data Ecosystem on Cloud - Conf42 Cloud Native 2024
Explore modern enterprise data ecosystems on cloud platforms, covering storage, organization, consumption, pipelines, and machine learning. Learn key components and AWS challenges.
How Well Do LLMs Detect Anomalies in Your Data? - Conf42 LLMs 2024
Explore LLMs' effectiveness in detecting data anomalies, understand data quality importance, and learn practical techniques for anomaly detection using OpenAI and BigQuery.