Securing LLMs in Kubernetes - Best Practices

Securing LLMs in Kubernetes - Best Practices

Explore best practices for securing Large Language Models (LLMs) in Kubernetes environments through this informative conference talk. Learn how to deploy open-source LLMs like Mistral on Kubernetes clusters, implement Nginx ingress controllers for front-ending, and secure both LLM pods and API prompt communication. Gain valuable insights into preventing sensitive data leakage and ensuring robust security measures for LLMs in production environments. Presented by Meenakshi Kaushik and Jayanth Srinivasa from Cisco, this 30-minute session offers a comprehensive overview of essential techniques for operating LLMs securely within Kubernetes infrastructures.