Streamlining Big Data Workflows with Memoization and Work Avoidance
Optimize big data and ML workflows using memoization and work avoidance techniques in Argo Workflows. Learn best practices to reduce runtime and compute costs for data processing and machine learning jobs.
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo Workflows and Argo Events
Discover best practices for building scalable Spark clusters on Kubernetes using Argo Workflows and Argo Events, enabling efficient resource utilization and seamless scalability for powerful data processing.
Big Data and Algorithms for Ecology and Conservation Seminar - Daniel Sheldon
Explore big data and algorithms for understanding ecological processes like bird migration, using novel data sources and algorithmic frameworks to revolutionize conservation efforts.
Visualizing the Spread of Hoaxes - Innovative Technology in Misinformation Analysis
Innovative technology visualizes hoax propagation, revealing patterns and impact. Big data analyst Ismail Fahmi explores the pervasive nature of misinformation and its spread in the modern digital landscape.
Big Data's Fourth V: Why We'll Never Find the Loch Ness Monster
Explore the concept of Big Data's fourth V and its implications for data analysis, using the Loch Ness Monster as an intriguing analogy to illustrate key principles.
Mo Data, Mo Problems - Data Management Challenges
Explore data management challenges and solutions in the digital age, focusing on strategies to handle increasing volumes of information effectively.
Wallet Services in Cyber Security
Explore the world of wallet services and their impact on digital transactions with insights from industry expert Stewart Pavitt.
Productionizing Big Data - Stories from the Trenches
Uncover mysteries of data pipeline anomalies, learn approaches to fixing and preventing issues, and gain insights into handling real-world production challenges in big data and AdTech.
Make Your Data AI-Ready with Microsoft Fabric and Azure Databricks
Discover how Microsoft Fabric and Azure Databricks collaborate to create a comprehensive, AI-powered analytics solution for enterprises, enabling teams to transform data into a competitive advantage.
Let's Play with Charmed Spark - Apache Spark on Kubernetes
Explore Apache Spark on Kubernetes using Charmed Spark. Learn to deploy, configure, and run Spark workloads, integrate with Kafka, and monitor jobs through hands-on demonstrations.
YTsaurus - The Next Generation of Open Source Big Data Platforms
Explore YTsaurus, an open-source big data platform for distributed storage and processing. Learn about its components, features, and capabilities for creating large-scale clusters up to 20,000 nodes.
Faster, Safer, Serverless - Empowering Apache Spark Standalone Cluster on Kubernetes
Explore a K8s-native Serverless Spark Service that enhances speed and simplicity, featuring a new K8s operator for cluster creation and job submission, with rapid responsiveness and ML training integration.
Super Reliable Cloud Native Data Processing Using Apache Spark and Cloud Shuffle Manager
Enhance Apache Spark's reliability on Kubernetes with Cloud Shuffle Manager, enabling fault tolerance and cost savings through Spot Instances while maintaining performance for large-scale data processing.
Empowering Developers with Easy, Scalable Stream Processing on Kubernetes
Explore scalable stream processing on Kubernetes with Numaflow, an open-source framework enabling easy, cost-efficient data analytics for various engineering roles, powering large-scale operations at Intuit.
Highly Available Search Platform for Data Resilience and Compliance at Shopify
Explore Shopify's highly available search platform, leveraging Kafka for real-time indexing and Elasticsearch for efficient querying across global regions, ensuring data resilience and compliance.
Data Science: Complete Roadmap to Become a Data Scientist
Comprehensive roadmap to becoming a data scientist, covering essential skills, tools, and resources. Includes job prospects, learning strategies, and recommended books for career advancement.
How Your Attention is Monetized - The New Currency in Digital Advertising
Explore the monetization of attention in digital advertising, its impact on privacy, and the potential shift towards contextual targeting for a more ethical attention economy.
Database Native Support for CDC Streaming Pipeline
Explore a database-native solution for Cassandra's CDC logs, addressing common challenges and proposing an improved architecture based on industry insights and managed CDC solutions.
What Data Tells Us About Man-Made Disasters
Explore how Big Data unveils hidden patterns in climate-related disasters, offering hope for a better future through innovative analytical tools and resilient community strategies.
Cloud Native Development with Apache Camel - From Low Code to Pro Code
Explore serverless and cloud-native app development using Apache Camel, transitioning from low-code to pro-code approaches for enhanced efficiency and scalability.
Containerizing and Orchestrating Hadoop Infrastructure at Scale
Explore Uber's journey in containerizing Hadoop infrastructure, addressing technical challenges, designing zero-downtime migration, and transforming team culture for efficient large-scale operations.