Bare Metal Kubernetes with KOps - Gathering Community Wisdom
Explore architectural trade-offs and challenges of deploying Kubernetes on bare metal infrastructure, focusing on node discovery, hardware management, and community-driven solutions using KOps.
Navigating Failures in Pods with Devices: Challenges and Solutions
Explore common challenges and solutions for managing device-enabled pods in Kubernetes, focusing on GPU allocation, network cards, and memory placement for AI/ML workloads.
Solving the Kubernetes Networking API Rubik's Cube - Understanding DRA and KNI
Uncover how AI/ML drives Kubernetes networking evolution, exploring DRA and KNI implementations, hardware allocation for networking, and standardized API developments in this comprehensive technical overview.
All Your GPUs Are Belong to Us: An Inside Look at NVIDIA's Self-Healing GeForce NOW Infrastructure
Discover how NVIDIA manages GeForce NOW's massive GPU infrastructure using Kubernetes, exploring automated maintenance systems and self-healing operators that keep the game streaming platform running for millions of users.
Distributed Cache Empowers AI/ML Workloads on Kubernetes Cluster
Discover how to optimize AI/ML workloads on Kubernetes using distributed cache systems, achieving high-performance data access with 50+ GB/s throughput and sub-2ms latency for large-scale GPU clusters.
Bloomberg's Journey to Manage Multi-Cluster Training Applications with Karmada
Discover how Bloomberg leverages Karmada to manage multi-cluster GPU workloads, optimize resource utilization, and streamline AI model training across data centers with intelligent scheduling and high availability.
Best Practices for Deploying LLM Inference, RAG and Fine-Tuning Pipelines on Kubernetes
Master best practices for deploying and managing LLM inference pipelines on Kubernetes, covering optimization techniques, security measures, and efficient pipeline management using tools like KServe.
Bloomberg's Journey to Improve Resource Utilization in a Multi-Cluster Platform
Discover how Bloomberg optimizes GPU resource management across multiple clusters using Karmada, focusing on intelligent scheduling, high availability, and streamlined job submission for AI training.
Faster Containerized LLM Serving via Knowledge Sharing
Discover how to optimize LLM performance through knowledge sharing systems, reducing processing delays and improving efficiency by enabling multiple LLMs to share cached information on Kubernetes.
Unlocking the Future of GPU Scheduling in Kubernetes with Reinforcement Learning
Explore advanced GPU scheduling in Kubernetes through reinforcement learning, focusing on optimizing multi-GPU setups, reducing resource fragmentation, and improving utilization for large-scale ML projects.
Which GPU Sharing Strategy Is Right for You? A Comprehensive Benchmark Study Using Dynamic Resource Allocation
Explore GPU sharing strategies in Kubernetes using Dynamic Resource Allocation (DRA), comparing Multi-Instance GPUs, MPS, and CUDA Time-Slicing for optimal workload performance.
Load-Aware GPU Fractioning for LLM Inference on Kubernetes
Discover how to optimize GPU utilization for LLM inference on Kubernetes through load-aware fractioning, using MIG slices to improve resource efficiency while maintaining performance standards.
Tackling GPU Shortages and High Costs by Harnessing Hybrid Kubernetes Clusters
Discover how to build hybrid Kubernetes clusters using K3s and P2P VPN to integrate diverse GPU resources, reducing costs while maintaining scalability and performance for AI workloads.
Nextflow and Julia for Scalable Computation
Discover powerful computational tools for scalable data processing, combining Nextflow's workflow management capabilities with Julia's high-performance computing features.
Incremental GPU Slicing in Kubernetes Clusters - Dynamic Resource Management
Discover how to implement dynamic GPU slicing in Kubernetes clusters for efficient resource allocation and cost optimization in large language model deployments, using vendor-agnostic approaches.
Brag Your RAG with MLOps - Building Scalable GenAI Solutions on Kubernetes
Discover how to build and deploy scalable RAG applications using LangChain, Canopy, and PostgreSQL Vector database on Kubernetes, while implementing effective MLOps strategies for GenAI solutions.
SkyRay - Extending KubeRay for Multi-Cluster Multi-Cloud Operations
Discover how SkyRay extends KubeRay's capabilities for multi-cluster cloud operations, enabling seamless Ray cluster management across different cloud environments with policy-driven scheduling and automated migration.
Boosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
Discover how ByteDance optimizes LLM workload performance through enhanced topology-aware scheduling, focusing on die-level affinity, RDMA optimization, and GPU-RDMA integration in Kubernetes environments.
Performance Aware Program Synthesis for Scientific Computing - Enabling Effective Lazy Programmers
Explore performance-aware program synthesis techniques for scientific computing, focusing on efficient code generation and optimization strategies for computational research.
Hardware-aware Algorithms for Language Modeling - FlashAttention and Mamba Architectures
Delve into advanced hardware optimizations for language models, exploring FlashAttention-3's improvements and Mamba's efficient architecture for enhanced performance and linear scaling capabilities.
Interpolation at Arbitrary Points in High-Order Meshes on GPUs
Discover advanced techniques for arbitrary point interpolation in high-order mesh computations, focusing on GPU optimization and parallel processing for efficient scientific simulations.