Scale Data Science Using Pandas API on Spark

Scale Data Science Using Pandas API on Spark

Learn how to scale data science workloads using Pandas API on Spark in this 27-minute conference talk from PyBay 2023. Discover how to overcome pandas' single-machine processing limitations while maintaining familiar pandas APIs for handling large-scale datasets. Explore the architecture behind Pandas API on Spark's optimized performance and gain practical knowledge for seamlessly transitioning existing data science workflows to handle vast amounts of data. Delivered at the regional Python conference for the San Francisco Bay Area, which brings together Python enthusiasts for deep-dive technical discussions and networking opportunities.