Cloud Fetch: High-Bandwidth Connectivity for BI Tools - Databricks
Explore high-bandwidth connectivity with BI tools through Cloud Fetch in this 20-minute Databricks video. Learn how to overcome the data transfer bottleneck in traditional data warehouses when extracting large query results using Business Intelligence tools like Tableau and Microsoft Power BI. Discover the new parallel data fetching mechanism via cloud storage, such as AWS S3 and Azure Data Lake Storage, which can result in a 10x speed-up in extract performance. Dive into the challenges of data growth, the intricacies of result pagination, and the improvements made with Apache Arrow. Understand the new data extract architecture, including hybrid results, data layout considerations, and parallel file downloads. Gain insights into Cloud Fetch's real-world performance and its ability to scale up extract workloads using cloud storage, ultimately enabling faster data ingestion for BI tools.