Processing Trillions of Records with Mini Serverless Databases at Okta

Processing Trillions of Records with Mini Serverless Databases at Okta

Explore serverless DuckDBs and their role in processing trillions of records at Okta in this 27-minute Data Council video. Delve into data preprocessing intricacies, operational metadata harvesting, and the optimization of batch ETL, streaming, SQL flexibility, S3 durability, and serverless scalability for cost-efficient environments. Learn how Okta's next-gen security platform leverages these technologies and gain insights into pressing questions about Kafka's necessity, batch ETL latencies, and ELT complexity. Discover the capabilities and potential applications of serverless DuckDB as Jake Thomas, Manager of Data Foundations at Okta, shares his expertise on this cutting-edge approach to handling massive data volumes.