Practice of Building Real-Time Data Lake Based on Flink
Build a real-time data lake using Flink and Iceberg to address timeliness, diversity, consistency, and accuracy challenges in modern data architectures.
Introducing Apache Pinot: Real-time Analytics for Large-Scale Data
Explore Apache Pinot's real-time analytics capabilities, architecture, and data model. Learn about its query language, indexing, and use cases in high-performance applications.
Building a Real-Time Analytics Database
Odkryj tajniki tworzenia bazy danych do analityki w czasie rzeczywistym. Poznaj kluczowe koncepcje i najlepsze praktyki w tej dziedzinie.
Crisis Text Line's Data Platform for Mental Health Support
Leveraging Spark and Delta Lake on Databricks to scale mental health support, automate resource provisioning, and enable flexible data pipelines for Crisis Text Line's global operations.
Simplifying Data Ingestion for LLMs with Unstructured and Databricks
Learn to build a scalable solution for unstructured data ingestion in LLM applications using Unstructured and Databricks, addressing challenges and exploring efficient methods.
Data Engineering on the Data Intelligence Platform - A Comprehensive Guide
Explore data engineering on Databricks' platform: ingest, transform, and orchestrate data at scale. Learn about streaming pipelines, data quality, and AI-powered tools for real-time insights.
Efficient, Low Latency Ingestion to Large Tables via Apache Flink and Apache Iceberg
Learn efficient, low-latency data ingestion techniques using Apache Flink and Apache Iceberg for large tables. Explore integration challenges, performance optimizations, and comparisons with alternative solutions.
QuestDB - Building Blocks of a Fast Open-Source Time-Series Database
Dive into the architecture of QuestDB and discover how this open-source time-series database achieves high-speed performance through specialized timestamp handling, efficient data ingestion, and SQL extensions.
Unlocking Advanced Log Analytics With ClickHouse and Kafka
Discover how to build a scalable, cost-effective log analytics solution using ClickHouse and Kafka. Learn JSON handling, data compression, and materialized views for processing billions of log events efficiently.
Managing a Multi-Cloud Data and Analytics Platform - Challenges and Solutions
Learn to manage a multi-cloud data and analytics platform, addressing challenges and leveraging benefits across major cloud providers for enterprise-level data, analytics, and AI solutions.
Azure Synapse Analytics: Building End-to-End Analytics Solutions
Unified analytics service integrating data warehousing, big data, and BI. Query data flexibly using serverless or dedicated resources at scale. Ingest, explore, prepare, manage, and serve data for immediate insights.
Log Processing with Apache NiFi and Apache Kafka
Learn real-time log processing using Apache NiFi and Kafka. Set up the environment, transform data, create flows, and load data into OpenSearch for effective log analytics.
High-Performance Market Data in a Cloud-Native World
Explore challenges and solutions for integrating high-performance electronic trading data with cloud computing, focusing on encoding, protocols, and efficient data analysis techniques.
A New Data Model for Time Series Data Processing
Innovative time series data model enhancing ingestion, query performance, and storage efficiency. Introduces "one table per collection point" and "supertable" concepts, with benchmark results demonstrating effectiveness.
Fast, Cheap and Easy Data Ingestion with AWS Lambda and Delta Lake
Explore event-driven data ingestion using AWS Lambda with Delta Lake. Learn to process, update, and query Delta tables efficiently in Python and Rust for a more responsive data platform.
Data Engineering with Rust and Delta Lake
Explore data engineering with Rust, leveraging Delta Lake, data fusion, and arrow for high-performance pipelines. Learn to ingest, enrich, manage, and export data using Rust's powerful capabilities.
Quarkus Meets AI - Build Your Own LLM-Powered Application
Explore building AI applications with LLMs using Quarkus and Langchain4j. Learn data ingestion strategies and integration with Apache Camel for comprehensive resource utilization.
Quarkus Meets AI - Build Your Own LLM-Powered Application
Explore building AI applications with LLMs using Quarkus and Langchain4j. Learn data ingestion strategies and integration with Apache Camel for comprehensive resource utilization in Java-based AI development.
Leveraging Argo Workflows and Argo Events for Cloud Agnostic Enterprise Machine Learning Deployments
Explore implementing Argo Workflows and Events for efficient MLOps, including continuous deployment, model retraining, and best practices for cloud-agnostic machine learning pipelines.
How to Build a Real-Time ML Pipeline for Fraud Prediction
Build a real-time ML pipeline for fraud prediction, addressing challenges in online feature engineering and real-time data ingestion. Learn to generate ML features, use a feature store, and monitor for drift.
Global School Connectivity DataOps Platform
Explore the development of an open-source DataOps stack managing connectivity data for millions of schools, covering ingestion, pipelines, quality, and analytics integration.