Partnering with Product for Effective Data Ingestion and Training Data
Discover strategies for effective collaboration between data science and product teams in this 19-minute talk by Daniela Santisteban from MLOps.community. Learn how to leverage product management partnerships to ensure quality data ingestion and training…
Dynatrace Business Events: Expanding Observability into Your Business Domain
Expand observability into business domain with Dynatrace Business Events. Learn to capture, ingest, explore, and analyze critical business data using OneAgent, API, and DQL for enhanced business analytics.
Building Data Pipelines with Apache Kafka and Confluent
Learn to build streaming data pipelines using Apache Kafka® and Confluent. Covers data ingestion, real-time transformation with ksqlDB, and data egress. Hands-on tutorials included for practical experience.
Building an Agentic Framework for Generative AI Applications
Discover how to build an agentic framework for GenAI applications, focusing on RAG-centric approaches, multilingual queries, and real-time data processing for enhanced customer assistance.
Why Lakehouse Architecture Now - Exploring Enterprise Data Warehouse Failures and the Need for Lakehouse Paradigm
Explore the shift from enterprise data warehouses to lakehouse architecture, focusing on benefits, challenges, and Starburst's role in powering SQL-based analytics on Delta Lake.
Designing and Implementing a Real-time Data Lake with Dynamically Changing Schema
Learn to build a robust streaming ETL pipeline that handles dynamically changing schemas and new event types without downtime, using Delta Lake and Databricks for real-time data lake implementation.
How Adobe Processes 2 Million Records Per Second Using Apache Spark
Explore Adobe's high-performance data processing using Apache Spark, covering optimization techniques, join strategies, structured streaming, skew handling, and integration with Redis for enhanced efficiency.
Automating FAA's SWIM Data Ingestion and Analysis with Databricks
Automating FAA's SWIM data ingestion and analysis using cloud-based services, showcasing infrastructure setup, configuration, and insights derived from flight data for improved air transportation management.
Predictive Maintenance on IoT Data for Early Fault Detection with Delta Lake
Explore predictive maintenance using IoT data and Delta Lake for early fault detection. Learn about Industry 4.0 applications, data analytics, and machine learning techniques for improved efficiency and cost savings.
AI-Powered Streaming Analytics for Real-Time Customer Experience
Explore AI-powered streaming analytics for real-time customer engagement. Learn to capture, process, and respond to data from various sources using Spark Structured Streaming for improved customer experiences.
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Learn best practices for building and deploying efficient, reproducible data pipelines in Apache Spark. Discover a toolkit that simplifies pipeline creation for non-engineers and explore deployment strategies.
Simplify and Scale Data Engineering Pipelines with Delta Lake
Learn to build scalable data engineering pipelines using Delta Lake's multi-hop architecture. Discover how to progress from raw data ingestion to machine learning-ready datasets efficiently and effectively.
Data Warehousing on the Lakehouse - Unifying Data, AI, and Analytics
Unify data, AI, and analytics with the lakehouse architecture. Learn success stories, use cases, and best practices for ingesting, storing, and governing data at scale for SQL and BI workloads using Databricks SQL.
Scaling Data and ML with Apache Spark and Feast - Feature Engineering for Production
Explore scaling data and ML with Apache Spark and Feast, focusing on feature engineering challenges and solutions for big data-driven machine learning in production environments.
Geospatial Data Analysis in Apache Spark - Challenges and Solutions
Explore geospatial data challenges in Apache Spark, comparing libraries, and learning best practices for data ingestion, storage, and retrieval using spatial indexing techniques.
Demystifying Anomaly Detection in OpenSearch - Setup and Implementation
Explore OpenSearch's anomaly detection capabilities through a hands-on walkthrough of remote logging setup, using fluent-bit and creative techniques to spark ideas for larger-scale solutions.
Building a Business-Critical Data Platform to Process Over £34 Billion in Card Transactions
Explore the development of a PCI DSS compliant data platform on Kubernetes, addressing challenges in the UK payments infrastructure and discussing key considerations for fintech scalability.
End to End MLOPS Data Science Project Implementation With Deployment
Comprehensive MLOps project implementation covering data ingestion, validation, transformation, model training, and deployment on AWS EC2 using MLflow and GitHub Actions.
Attempting to Tame the Data Beast: A BI Transition Tale
Explore the journey of modernizing a BI platform, from legacy systems to cutting-edge tools. Learn about data ingestion, dbt transformations, and Airflow orchestration, along with insights on overcoming challenges.
New Features of Apache NiFi - Tips and Techniques
Explore Apache NiFi's latest features, processors, and best practices. Build efficient data flows using cutting-edge techniques, with tips and guides for optimal implementation.
Data Engineering: A Deep Dive into Databricks
Master Databricks for efficient data processing, focusing on Apache Spark transformations, ingestion methods, storage, orchestration, and best practices to optimize data engineering workflows.