Automating Cloud-native Spark Jobs with Argo Workflows
Learn to orchestrate Spark jobs on Kubernetes using Argo Workflows, exploring architecture, resource management, and scaling advantages for efficient large-scale data processing in cloud-native environments.
Build a Reproducible ML Workflow with Kubeflow Pipelines
Learn to create reproducible machine learning workflows using Kubeflow Pipelines, enhancing efficiency and scalability in ML projects with Google's open-source platform.
Flyte - Cloud Native Machine Learning and Data Processing Platform
Explore Flyte, a cloud-native platform for machine learning and data processing, presented by Lyft engineers. Learn its features, benefits, and applications in modern data workflows.
Bringing Transactionality to the Streaming Ecosystem - Webinar
Explore modern streaming data platforms for real-time transaction processing, focusing on durability, scalability, and performance in critical data paths. Learn from Alpaca's experience re-engineering their order management system.
Hail Hydrate - From Stream to Lake Using Open Source
Explore open-source tools for efficient data streaming and lake management, enhancing your skills in modern data architecture and processing techniques.
Supercharging Machine Learning Development with Azure and Containers in VS Code
Explore how to enhance machine learning development using Azure and containers in Visual Studio Code, boosting efficiency and streamlining workflows.
Modernizing Event Emission Infrastructure with Fluent Stack - Asana Case Study
Discover how Asana modernized its event emission infrastructure using Fluent Bit, scaling to 3.5 billion events daily while improving flexibility, performance, and durability.
Building a Data Science Platform with Argo Workflows
Discover how Bloomberg and Pipekit leverage Argo Workflows and Kubernetes to create scalable data science platforms, enabling data teams to automate pipelines and self-serve their data engineering needs across multiple clusters.
Evolution and Scaling of Feature Store at Uber
Explore Uber's Palette Feature Store evolution, focusing on feature sharing, cost reduction, and scalability. Learn about serving 21000+ features at 25M queries/sec with 99.999% uptime.
How to Survive High Demanding Services - Managing AI with Kubernetes and Istio
Explore strategies for managing high-demand AI services, including design for seamless changes, real-time monitoring, and auto-recovery using Kubernetes and Istio service mesh.
Beyond Experimental - Spark on Kubernetes
Insights on leveraging open-source tech for Apache Spark on Kubernetes, addressing challenges in large-scale production environments and evolving ML data infrastructure in the cloud.
Production Ready RAG - DevWorld 2024
Explore production-ready Retrieval-Augmented Generation (RAG) techniques for enhancing AI applications with Michel de Ru's expert insights from DevWorld 2024.
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo Workflows and Argo Events
Discover best practices for building scalable Spark clusters on Kubernetes using Argo Workflows and Argo Events, enabling efficient resource utilization and seamless scalability for powerful data processing.
Serverless Data Pipelines in Rust - Rust Vienna Meetup
Explore building serverless data pipelines in Rust using DataFusion and object store, with deployment on AWS Lambda. Learn scalable techniques for complex pipelines in production environments.
Navigating the Data Engineering Landscape in the Era of AI
Explore data engineering's role in AI, addressing scalability and real-time processing challenges. Learn from case studies and gain insights into future trends shaping the field's evolution.
Faster, Safer, Serverless - Empowering Apache Spark Standalone Cluster on Kubernetes
Explore a K8s-native Serverless Spark Service that enhances speed and simplicity, featuring a new K8s operator for cluster creation and job submission, with rapid responsiveness and ML training integration.
Empowering Developers with Easy, Scalable Stream Processing on Kubernetes
Explore scalable stream processing on Kubernetes with Numaflow, an open-source framework enabling easy, cost-efficient data analytics for various engineering roles, powering large-scale operations at Intuit.
Highly Available Search Platform for Data Resilience and Compliance at Shopify
Explore Shopify's highly available search platform, leveraging Kafka for real-time indexing and Elasticsearch for efficient querying across global regions, ensuring data resilience and compliance.
Connecting Thanos to the Outer Rim via Query API
Explore Thanos' new Query API feature for connecting to managed Prometheus solutions, enabling global querying across diverse data sources and unlocking innovative use cases in cloud-native monitoring.
Transforming Data Science at PepsiCo - The Kubeflow Revolution
Explore PepsiCo's data science transformation using Kubeflow, addressing challenges in model reproducibility and enhancing efficiency, collaboration, and scalability in machine learning workflows.
Open Source BPM Comparison
Explore open-source Business Process Management solutions with insights from Capital One's experience, comparing features and benefits for effective workflow automation.