Growing Pain: Scaling to 10K Workflows per Week

Growing Pain: Scaling to 10K Workflows per Week

Explore the challenges and solutions of scaling a centralized multi-tenant workflow orchestration platform to handle 10,000 workflows per week in this conference talk by Yao Lin and Harris Khan from Bloomberg. Discover how they tackled issues such as spiky submission rates, unpredictable resource requests, API server and workflow-controller demand, and the complexities of throttling and garbage collection. Learn about the unique challenges of serving diverse user profiles with varied requirements under a single cluster with a global Argo installation. Gain insights into the guardrails and mitigation strategies developed by the Bloomberg team, which can be applied to other batch workload scenarios in cloud native environments.