Explore the challenges and solutions of operating enterprise-grade Kubernetes clusters on bare metal infrastructure in this 32-minute conference talk by Anubhav Dhoot and Mayank Kumar from Salesforce. Gain insights into managing highly available Kubernetes clusters without public cloud managed offerings or VMs, focusing on specific challenges in setup, security, upgrades, and 24/7 monitoring using Puppet. Learn from the experiences of managing a fleet of 2600+ hosts across 20+ data centers over four years, including battle stories on etcd configuration issues, hairpin networking problems, and service account key rotation. Discover the intricacies of Salesforce's on-premise Kubernetes platform, delve into Puppet module features, and understand the operational aspects of monitoring bare metal Kubernetes. Explore potential pitfalls such as mounting hostPath, bridge networking failures, and inconsistent API server flags. This talk serves as a valuable resource for teams managing their own Kubernetes clusters, offering lessons learned and cautionary tales from Salesforce's journey towards managed Kubernetes offerings in the public cloud. Read more