Data Microservices in Apache Spark Using Apache Arrow Flight
Explore efficient machine learning pipelines using Apache Arrow Flight in Spark, enabling faster data transport and simplified microservices for improved ML model production.
AutoML Toolkit Deep Dive - Automating Feature Engineering and Model Optimization
Explore Databricks Labs' AutoML Toolkit for automating feature engineering, model selection, tuning, and deployment. Learn to accelerate machine learning workflows and enhance productivity.
Building a Data Platform for Mission-Critical Analytics on AWS with Databricks
Explore building enterprise data lakes on AWS with Databricks, complementing AWS strategy. Learn from HP's success in transforming business through unified data analytics and overcoming data lake challenges.
Delta from a Data Engineer's Perspective - Creating Robust Big Data Solutions
Explore a data engineer's challenges in creating robust Big Data solutions, covering Delta, Spark streaming, schema enforcement, and time travel functionality in data processing.
Running Apache Spark on Kubernetes - Best Practices and Pitfalls
Learn best practices and pitfalls for running Apache Spark on Kubernetes, covering setup, performance optimization, resource management, security, and future developments in this comprehensive overview.
Portable and Scalable Data Visualization Techniques for Apache Spark and Python Notebooks
Explore portable and scalable data visualization techniques for Apache Spark and Python notebooks, covering strategies, packages, headless rendering, animations, and big data integration.
Preventing Common Pitfalls in Production Streaming Jobs
Explore key strategies for optimizing streaming jobs in production, covering input parameters, stateful streaming, output configurations, and job modifications to ensure performance and fault tolerance.
Composable Data Processing with Apache Spark - Scaling Development and Error Handling
Learn about SIP, an extensible plugin framework for Apache Spark that enhances data processing efficiency, error handling, and scalability in Adobe's Experience Platform.
Re-Imagining Apache Spark Development - Tools for Productivity and Standardization
Exploring next-gen tools for Apache Spark development, focusing on productivity, code standardization, metadata, lineage, and CI/CD agility to overcome limitations of traditional ETL tools.
Introducing Apache Spark 3.0 - A Decade of Progress and Future Outlook
Apache Spark 3.0: Reflecting on a decade of progress and unveiling new features to enhance data processing accessibility, speed, and compatibility across various sources and environments.
Managing the Complete Machine Learning Lifecycle with MLflow - Introduction to MLflow Tracking
Introduction to MLflow: tracking experiments, packaging projects, deploying models, and managing lifecycle. Learn to record, reproduce, and compare ML runs using MLflow's components and UI.
State of the Industry - Machine Learning Platforms and MLOps
Explore the state of MLOps and ML platforms with industry leaders, focusing on challenges, common components, and Databricks' approach to unified data analytics and machine learning.
Predictive Maintenance on IoT Data for Early Fault Detection with Delta Lake
Explore predictive maintenance using IoT data and Delta Lake for early fault detection. Learn about Industry 4.0 applications, data analytics, and machine learning techniques for improved efficiency and cost savings.
How to Build a Cloud Data Platform - Machine Learning and Business Intelligence - Part 4
Learn to leverage Delta Lake for dashboards and advanced analytics, enabling downstream teams with machine learning and business intelligence capabilities on a unified data platform.
PLSE Seminar Series - Alessandra Gorla - Mining Android Apps for Anomalous Behavior
Techniques for detecting suspicious behavior in Android apps by clustering based on advertised functionality and identifying outliers in API usage, with applications in malware detection.
Cryptocurrency - Nick Gonella
Explore Bitcoin's inner workings, from blockchain and mining to wallet automation and real-world applications, in this comprehensive overview of cryptocurrency fundamentals.
Introduction to IOTA - A Feeless Cryptocurrency, Serguei Popov, University of Campinas
Learn about IOTA, a unique cryptocurrency using a tangled ledger system, exploring its consensus rules, advantages, challenges, and advanced properties like quantum cryptography.
Big Data Analytics - Ben Fountain, NCC Group
Explore big data analytics, its importance, key controls, and potential pitfalls. Learn how to leverage data for insights and decision-making in various industries.
Big Data and Cyber Security - David Stubley
Explore the intersection of big data and cybersecurity, covering data management, visualization, and key drivers in this informative talk by David Stubley.
How Slot Machines Are Advancing the State of the Art in Computer Go AI
Explore Monte Carlo Tree Search in computer Go AI, covering its advantages, UCB1 strategy, and recent refinements like AMAF/RAVE, with insights into ongoing research topics.
Cryptocurrencies - Trust, Verification, and Privacy - Max Zinkus
Explore trust, verification, and privacy in cryptocurrencies, including blockchain technology, proof of work, and cryptographic primitives for secure online transactions.