Apache Pulsar - Next Generation Messaging and Queuing System
Overview of Apache Pulsar's architecture, features, and benefits as a scalable messaging system with unified storage, processing, and serverless capabilities for building end-to-end streaming applications.
Zeus - Uber's Highly Scalable Distributed Shuffle as a Service
Explore Uber's Zeus: a highly scalable, distributed shuffle service powering data processing. Learn its architecture, integration with Spark, and performance advantages over traditional external shuffle methods.
Efficient Distributed Hyperparameter Tuning with Apache Spark
Learn to efficiently distribute hyperparameter tuning workloads using Apache Spark, avoiding common pitfalls and optimizing performance. Explore best practices with Hyperopt and joblib-spark for scaling ML model training.
Architect's Open-Source Guide for Data Mesh Architecture
Explore Data Mesh architecture, its principles, benefits, and implementation challenges. Learn about open-source tools like Apache Spark for building effective Data Mesh systems in real-world scenarios.
Using Docker Images in Production - Databricks Case Study
Learn how Condé Nast evolved Spire, their user segmentation service, from notebooks to Docker images on Databricks clusters. Explore the streamlined deployment workflow, challenges faced, and solutions implemented.
Databricks Lakehouse Platform - Governance and Security Fundamentals
Learn fundamentals of governance and security for cloud data platforms, covering identity management, private links, access monitoring, cost control, and policy enforcement for workspaces.
Efficient Query Processing for Unstructured Data Using Machine Learning
Explore efficient query processing techniques for unstructured data using machine learning, including proxy scores and clustering methods to accelerate analysis and reduce costs.
MLflow Model Serving - Methods and Best Practices
Explore MLflow model serving techniques, including batch and real-time scoring, open-source and Databricks-managed options, with emphasis on new production-ready model serving features.
Monitoring ML Models, IoT Data, and Quality Checks on Delta Lake - Quby's Approach
Discover how Quby monitors IoT data, ML models, and data quality using Databricks. Learn about Delta Lake, MLflow, and Slack integration for efficient data processing and model performance tracking.
Machine Learning and Earth System Modeling - From Parameter Calibration to Feature Detection
Explore machine learning applications in Earth system modeling, from parameter calibration to feature detection, enhancing climate predictions and understanding of complex environmental processes.
Interpretable and Generalizable Machine Learning for Fluid Dynamics - Steven Brunton
Explore interpretable and generalizable machine learning techniques for fluid dynamics, focusing on sparse identification of nonlinear dynamics and incorporating physical constraints for improved accuracy.
Physics-Guided Deep Learning for Fluid Dynamics - Rose Yu
Explores physics-guided deep learning for fluid dynamics, focusing on accelerating turbulence simulations and incorporating symmetry for generalization in ocean current forecasting.
Building Recommendation Systems Using Graph Neural Networks
Explore graph neural networks for recommendation systems, focusing on RECKON's encoder-decoder architecture to improve entity intelligence and personalize user experiences.
Quick to Production: Integrating Spark and TensorFlow for Efficient MLOps
Learn to quickly deploy deep learning models using Spark and TensorFlow, simplifying MLOps for data scientists. Explore distributed computing, feature engineering, and leveraging TensorFlow ecosystem libraries for efficient model production.
Identifying Sensitive Data and Mitigating Risk in Apache Spark and Databricks
Discover sensitive data in Apache Spark and Databricks using BigID's platform. Learn to scale discovery, label data, and apply necessary guardrails for compliance and risk mitigation in data analytics.
Unsupervised Machine Learning for Scaling Data Quality Monitoring in Databricks
Explore unsupervised ML algorithms for scalable data quality monitoring in Databricks, covering implementation, strengths, weaknesses, and practical applications for effective data management.
Observability in ArgoCD and Rollouts Using Streaming ML for Reducing MTTR
Explore real-time anomaly detection in Kubernetes using streaming ML and DAG-based processing. Learn how Intuit reduces outage response time with AIOps and integrates observability into Argo CD.
Connect All the Things - Using NATS to Simplify Communication Everywhere
Explore NATS, an open-source messaging system for simplified, globally scalable communication. Learn about its features, patterns, and applications in building adaptive, multi-cloud systems.
Colocating Hadoop YARN with Kubernetes to Save Costs on Big Data
Learn how to optimize resource utilization by colocating Hadoop YARN with Kubernetes, addressing challenges and implementing solutions to save costs on big data infrastructure while maintaining service quality.
Intuit's Customer-Centric Observability Journey Using AIOps
Explore Intuit's journey in improving operational excellence using AIOps, focusing on real-time data analysis, anomaly detection, and incident response for enhanced customer-centric observability.
Volcano for FinTech Big Data on Kubernetes
Explore how Volcano enhances FinTech BigData operations on Kubernetes, optimizing resource management and workflow efficiency for financial technology applications.