Best Practices for Deploying LLM Inference, RAG and Fine-Tuning Pipelines on Kubernetes
Master best practices for deploying and managing LLM inference pipelines on Kubernetes, covering optimization techniques, security measures, and efficient pipeline management using tools like KServe.
Bloomberg's Journey to Improve Resource Utilization in a Multi-Cluster Platform
Discover how Bloomberg optimizes GPU resource management across multiple clusters using Karmada, focusing on intelligent scheduling, high availability, and streamlined job submission for AI training.
Faster Containerized LLM Serving via Knowledge Sharing
Discover how to optimize LLM performance through knowledge sharing systems, reducing processing delays and improving efficiency by enabling multiple LLMs to share cached information on Kubernetes.
Unlocking the Future of GPU Scheduling in Kubernetes with Reinforcement Learning
Explore advanced GPU scheduling in Kubernetes through reinforcement learning, focusing on optimizing multi-GPU setups, reducing resource fragmentation, and improving utilization for large-scale ML projects.
Which GPU Sharing Strategy Is Right for You? A Comprehensive Benchmark Study Using Dynamic Resource Allocation
Explore GPU sharing strategies in Kubernetes using Dynamic Resource Allocation (DRA), comparing Multi-Instance GPUs, MPS, and CUDA Time-Slicing for optimal workload performance.
Load-Aware GPU Fractioning for LLM Inference on Kubernetes
Discover how to optimize GPU utilization for LLM inference on Kubernetes through load-aware fractioning, using MIG slices to improve resource efficiency while maintaining performance standards.
Tackling GPU Shortages and High Costs by Harnessing Hybrid Kubernetes Clusters
Discover how to build hybrid Kubernetes clusters using K3s and P2P VPN to integrate diverse GPU resources, reducing costs while maintaining scalability and performance for AI workloads.
Nextflow and Julia for Scalable Computation
Discover powerful computational tools for scalable data processing, combining Nextflow's workflow management capabilities with Julia's high-performance computing features.
Incremental GPU Slicing in Kubernetes Clusters - Dynamic Resource Management
Discover how to implement dynamic GPU slicing in Kubernetes clusters for efficient resource allocation and cost optimization in large language model deployments, using vendor-agnostic approaches.
Brag Your RAG with MLOps - Building Scalable GenAI Solutions on Kubernetes
Discover how to build and deploy scalable RAG applications using LangChain, Canopy, and PostgreSQL Vector database on Kubernetes, while implementing effective MLOps strategies for GenAI solutions.
SkyRay - Extending KubeRay for Multi-Cluster Multi-Cloud Operations
Discover how SkyRay extends KubeRay's capabilities for multi-cluster cloud operations, enabling seamless Ray cluster management across different cloud environments with policy-driven scheduling and automated migration.
Boosting Training and Inference Performance via Topology-Aware Scheduling of Heterogeneous Resources
Discover how ByteDance optimizes LLM workload performance through enhanced topology-aware scheduling, focusing on die-level affinity, RDMA optimization, and GPU-RDMA integration in Kubernetes environments.
Hardware-aware Algorithms for Language Modeling - FlashAttention and Mamba Architectures
Delve into advanced hardware optimizations for language models, exploring FlashAttention-3's improvements and Mamba's efficient architecture for enhanced performance and linear scaling capabilities.
Interpolation at Arbitrary Points in High-Order Meshes on GPUs
Discover advanced techniques for arbitrary point interpolation in high-order mesh computations, focusing on GPU optimization and parallel processing for efficient scientific simulations.
DIZK: A Distributed Zero Knowledge Proof System
Discover how DIZK revolutionizes zero knowledge proof systems through distributed computing, enabling 100x larger computations and faster processing for enhanced blockchain security and privacy applications.
Introduction to Game Shaders and GLSL Programming - Lecture 18
Dive into shader programming fundamentals, from basic GLSL concepts to advanced effects like green screening and spotlights, with hands-on demonstrations using ShaderToy and SFML implementation.
Fine-tuning Llama 3.2 for Mental Health Sentiment Analysis Using TorchTune
Master fine-tuning techniques for Llama 3.2 using torchtune, focusing on sentiment analysis for mental health data with practical implementation steps on a single GPU.
Building Scalable AI Infrastructure with KubeRay and Kubernetes
Master scalable AI infrastructure deployment using KubeRay and Kubernetes, covering gang scheduling, distributed checkpointing, and integration with cloud providers for efficient AI workload management.
AI in Kubernetes - Getting Started with Machine Learning Pipeline Development
Discover how Kubernetes supports AI development with features like autoscaling, GPU management, and ML pipeline orchestration using Flyte for efficient model deployment and serving.
Faster Inference Using Output Predictions with OpenAI and vLLM
Master advanced inference techniques for faster AI model outputs using OpenAI predictions and vLLM speculative decoding, with practical demonstrations and cost optimization strategies.
AIStore as a Fast Tier Storage Solution: Enhancing Petascale Deep Learning Across Cloud Backends
Explore AIStore's innovative approach to handling petascale datasets, offering high-performance storage solutions for deep learning with linear scalability and optimized cloud backend integration.