vLLM on Kubernetes in Production - Deployment and Cost-Saving Strategies
Deploy vLLM on Kubernetes for efficient LLM inference and serving. Learn basics, local setup, and production deployment with GPU-attached nodes via DaemonSet. Includes hands-on demo and cost-saving strategies.
Scaling Video Ad Classification Across Millions of Classes with GenAI
Innovative approach combining LLMs and ML for video ad classification, achieving 15% accuracy uplift and scalability to millions of product classes. Enhances real-time insights for ad-intel company Vivvix.
vLLM - Servidor Especializado en Inferencia de LLMs
Descubre cómo implementar y optimizar servidores de inferencia LLM con vLLM, una tecnología especializada para el procesamiento eficiente de modelos de lenguaje a gran escala.
vLLM Inference and LLM Server Engine for Machine Learning
Discover how to leverage vLLM for efficient large language model inference and serving, including practical implementation techniques and deployment strategies.
Open Source LLMs in the Cloud - Scalable Solutions
Explore scalable deployment strategies for open-source LLMs in cloud environments, comparing Python solutions, native runtimes, and WebAssembly approaches for optimal performance and resource efficiency.
Retrieval Augmented Generation - Techniques and Applications
Explore Retrieval Augmented Generation with Syed Asad. Dive into production issues, embedding models, inference layer experiments, and the evolving landscape of AI engineering in this insightful discussion.
Deploy LLMs More Efficiently with vLLM and Neural Magic
Explore vLLM's efficiency for LLM deployment and Neural Magic's enterprise solutions for cost-effective, scalable AI model services.
Serve a Custom LLM for Over 100 Customers - GPU Selection, Quantization, and API Setup
Learn to deploy and serve custom LLMs efficiently for multiple customers. Master server selection, API software, GPU optimization, and quantization techniques for cost-effective, high-performance model serving.
Tiny Text and Vision Models - Fine-Tuning and API Setup
Explore fine-tuning and API setup for tiny text and vision models, covering multi-modal architectures, LoRA adapters, and deployment strategies for custom APIs.
Fast LLM Serving with vLLM and PagedAttention
Explore vLLM, an open-source library for fast LLM inference using PagedAttention. Learn how it achieves higher throughput and its implementation for efficient AI model serving.
Optimizing LLM Inference with AWS Trainium, Ray, vLLM, and Anyscale
Explore cost-effective LLM inference scaling using AWS accelerators, Ray, vLLM, and Anyscale. Learn to build a complete stack on EKS and leverage advanced cluster management for enterprise-grade GenAI workloads.
Cloud Native Sustainable LLM Inference in Action
Explore sustainable LLM inference using cloud-native tech. Learn about energy monitoring, AI accelerator optimization, and balancing efficiency with environmental impact. Includes live demo of vLLM in a Kubernetes cluster.
Building Your Own ChatGPT-style LLM AI Infrastructure with Kubernetes
Explore building AI infrastructure with Kubernetes, covering open-source LLMs, data engineering, and cost optimization. Gain insights on deployment, scalability, and choosing the right tools for AI-enabled applications.
How to Pick a GPU and Inference Engine for Large Language Models
Explore GPU selection and inference engine optimization for large language models. Compare quantization effects, software options, and cost-efficiency across various GPUs and model sizes.
Load-Aware GPU Fractioning for LLM Inference on Kubernetes
Discover how to optimize GPU utilization for LLM inference on Kubernetes through load-aware fractioning, using MIG slices to improve resource efficiency while maintaining performance standards.
Faster Inference Using Output Predictions with OpenAI and vLLM
Master advanced inference techniques for faster AI model outputs using OpenAI predictions and vLLM speculative decoding, with practical demonstrations and cost optimization strategies.
Architecting the Future of AI: From Cloud-Native Orchestration to Advanced LLMOps
Dive into cloud-native AI architecture, exploring Kubernetes orchestration, Ray distributed computing, and vLLM for efficient large language model deployment and management.