Главная
Study mode:
on
You
History
Saved
In progress 0 courses
compleat 0 courses
#Art & Design #Adobe #ChatGPT #GitHub

#vLLM

Showing: 38 courses
Anyscale

Roblox's Journey to Supporting Multimodality on vLLM - Ray Summit 2024

0 rewiews
Explore Roblox's integration of multimodal language models into vLLM, uncovering technical challenges and insights for developers seeking to enhance their LLM deployments with advanced AI capabilities.
Add to list
1 Lesons
30 minutes
On-Demand
Free-Video
Anyscale

Achieving vLLM Platform Portability with Triton Autotuning - Ray Summit 2024

0 rewiews
Discover IBM Research's innovative approach to vLLM platform portability using Triton autotuning, enhancing LLM serving with improved cross-platform performance and reduced dependencies.
Add to list
1 Lesons
31 minutes
On-Demand
Free-Video
Anyscale

Databricks' vLLM Optimization for Cost-Effective LLM Inference - Ray Summit 2024

0 rewiews
Discover strategies for optimizing vLLM to enhance LLM inference performance, reduce GPU idle time, and accelerate quantization, leading to cost-effective large-scale language model deployment.
Add to list
1 Lesons
28 minutes
On-Demand
Free-Video
Anyscale

Optimizing vLLM for Intel CPUs and XPUs - Ray Summit 2024

0 rewiews
Dive into Intel's optimization strategies for vLLM on CPUs and XPUs, exploring technical advancements, performance data, and collaborative efforts to enhance GenAI inference efficiency.
Add to list
1 Lesons
30 minutes
On-Demand
Free-Video
Anyscale

Handshake's Approach to Content Tagging with vLLM and Anyscale

0 rewiews
Explore Handshake's journey in implementing vLLM-based content tagging using Anyscale, enhancing job feed performance and addressing growing LLM requirements through innovative internal tools and methodologies.
Add to list
1 Lesons
24 minutes
On-Demand
Free-Video
Anyscale

The State of vLLM - Advancements in LLM Inference and Serving

0 rewiews
Explore the latest advancements in vLLM, the open-source LLM inference engine. Gain insights into its growing adoption, new features, performance improvements, and future roadmap for efficient LLM deployment.
Add to list
1 Lesons
35 minutes
On-Demand
Free-Video
Anyscale

Accelerated LLM Inference with Anyscale - Ray Summit 2024

0 rewiews
Dive into Anyscale's latest LLM enterprise features and open-source contributions. Explore advancements in vLLM, including FP8 support and speculative decoding, doubling throughput and latency efficiency for improved inference performance.
Add to list
1 Lesons
30 minutes
On-Demand
Free-Video
Anyscale

Dynamic Scheduling for Large Language Model Serving

0 rewiews
Discover how Llumnix optimizes large language model serving through dynamic request scheduling, cross-instance defragmentation, and innovative KV cache migration techniques for improved performance.
Add to list
1 Lesons
32 minutes
On-Demand
Free-Video
Anyscale

How Roblox Scaled Machine Learning by Leveraging Ray for Efficient Batch Inference

0 rewiews
Discover how Roblox leverages Ray to scale machine learning infrastructure, focusing on multimodal language models integration, batch inference optimization, and practical implementation strategies.
Add to list
1 Lesons
29 minutes
On-Demand
Free-Video
Anyscale

The Evolution of Multi-GPU Inference in vLLM

0 rewiews
Dive into advanced parallelism strategies for distributed LLM inference, exploring tensor, pipeline, and expert parallelism techniques through vLLM's architecture and optimization approaches.
Add to list
1 Lesons
31 minutes
On-Demand
Free-Video
Anyscale

Building Intelligent AI Infrastructure with ORI - Dynamic Query Routing and Model Management

0 rewiews
Discover how ORI dynamically routes AI queries to optimal language models, featuring chain atomic queries, benchmark matching, and ethical frameworks integrated with Ray Serve and vLLM for high-speed inference.
Add to list
1 Lesons
22 minutes
On-Demand
Free-Video
Anyscale

Optimizing vLLM Performance Through Quantization - Model Compression Techniques

0 rewiews
Discover practical techniques for model quantization in vLLM deployments, exploring FP8, INT8, and INT4 methods to enhance performance and reduce memory usage while maintaining efficiency.
Add to list
1 Lesons
38 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Running LLMs in the Cloud - Approaches and Best Practices

0 rewiews
探索云端大语言模型部署策略：比较Python、原生运行时和WebAssembly方法，分析各自优缺点，助您选择最佳实施方案，优化资源效率和可移植性。
Add to list
1 Lesons
11 minutes
On-Demand
Free-Video
Linux Foundation

Running LLMs in the Cloud - Best Practices and Deployment Approaches

0 rewiews
探索云端大语言模型部署策略：Python解决方案、原生运行时和WebAssembly。了解各方法优缺点，获取实用建议，选择最佳部署策略。
Add to list
1 Lesons
11 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Running LLMs in the Cloud - Approaches and Best Practices

0 rewiews
探讨在云原生环境中部署大语言模型的最佳实践，比较Python、原生运行时和WebAssembly三种方法的优缺点，为开发者提供实用建议和清晰路线图。
Add to list
1 Lesons
11 minutes
On-Demand
Free-Video
Anyscale

Akash Network - Powering Ray on an Open Source Super Cloud

0 rewiews
Discover how Akash Network's open-source super cloud platform revolutionizes distributed computing and AI workloads, featuring real-world applications and live demonstrations of Ray clusters with vLLM and Llama-3.1.
Add to list
1 Lesons
16 minutes
On-Demand
Free-Video
Qdrant - Vector Database & Search Engine

Production-Scale Retrieval Augmented Generation for Real-Time News Distillation

0 rewiews
Discover how to process millions of news articles daily using open-source tools like Flowdapt, Qdrant, vLLM, and TEI for real-time RAG applications, with insights on scaling, deployment, and startup advantages.
Add to list
10 Lesons
34 minutes
On-Demand
Free-Video
Trelis Research

Test Time Compute: Verifiers and Parallel Sampling - Part 2

0 rewiews
Explore advanced inference techniques using verifiers, parallel sampling, and prompt optimization to enhance language model performance and accuracy.
Add to list
21 Lesons
1 hour
On-Demand
Free-Video
Linux Foundation

Optimizing Kubernetes Cluster Scaling for Advanced Generative Models

0 rewiews
Explore advanced techniques for scaling Kubernetes clusters to support complex generative models, leveraging tools like Flyte, MLflow, and Kubeflow for efficient deployment and management.
Add to list
1 Lesons
1 hour 22 minutes
On-Demand
Free-Video
Data Centric

Using Open Source LLMs with AutoGen - Deploying Llama 2 70B Tutorial

0 rewiews
Deploy open-source LLMs with AutoGen using vLLM, Runpod, and Postman to simulate OpenAI chat completion endpoints. Learn to utilize this setup as a direct replacement for OpenAI invocations.
Add to list
1 Lesons
19 minutes
On-Demand
Free-Video
Data Centric

Building Local AI Agents with LangGraph and Ollama

0 rewiews
Explore building local AI agents using LangGraph and Ollama. Learn integration techniques, code implementation, and compare with vLLM. Gain insights on Ollama's effectiveness for agent development.
Add to list
5 Lesons
19 minutes
On-Demand
Free-Video
Kubesimplify

vLLM on Kubernetes in Production - Deployment and Cost-Saving Strategies

0 rewiews
Deploy vLLM on Kubernetes for efficient LLM inference and serving. Learn basics, local setup, and production deployment with GPU-attached nodes via DaemonSet. Includes hands-on demo and cost-saving strategies.
Add to list
1 Lesons
28 minutes
On-Demand
Free-Video
Databricks

Scaling Video Ad Classification Across Millions of Classes with GenAI

0 rewiews
Innovative approach combining LLMs and ML for video ad classification, achieving 15% accuracy uplift and scalability to millions of product classes. Enhances real-time insights for ad-intel company Vivvix.
Add to list
1 Lesons
24 minutes
On-Demand
Free-Video
The Machine Learning Engineer

vLLM - Servidor Especializado en Inferencia de LLMs

0 rewiews
Descubre cómo implementar y optimizar servidores de inferencia LLM con vLLM, una tecnología especializada para el procesamiento eficiente de modelos de lenguaje a gran escala.
Add to list
1 Lesons
43 minutes
On-Demand
Free-Video
The Machine Learning Engineer

vLLM Inference and LLM Server Engine for Machine Learning

0 rewiews
Discover how to leverage vLLM for efficient large language model inference and serving, including practical implementation techniques and deployment strategies.
Add to list
1 Lesons
46 minutes
On-Demand
Free-Video
Linux Foundation

Open Source LLMs in the Cloud - Scalable Solutions

0 rewiews
Explore scalable deployment strategies for open-source LLMs in cloud environments, comparing Python solutions, native runtimes, and WebAssembly approaches for optimal performance and resource efficiency.
Add to list
1 Lesons
40 minutes
On-Demand
Free-Video
MLOps.community

Retrieval Augmented Generation - Techniques and Applications

0 rewiews
Explore Retrieval Augmented Generation with Syed Asad. Dive into production issues, embedding models, inference layer experiments, and the evolving landscape of AI engineering in this insightful discussion.
Add to list
20 Lesons
44 minutes
On-Demand
Free-Video
Neural Magic

Deploy LLMs More Efficiently with vLLM and Neural Magic

0 rewiews
Explore vLLM's efficiency for LLM deployment and Neural Magic's enterprise solutions for cost-effective, scalable AI model services.
Add to list
18 Lesons
33 minutes
On-Demand
Free-Video
Trelis Research

Serve a Custom LLM for Over 100 Customers - GPU Selection, Quantization, and API Setup

0 rewiews
Learn to deploy and serve custom LLMs efficiently for multiple customers. Master server selection, API software, GPU optimization, and quantization techniques for cost-effective, high-performance model serving.
Add to list
12 Lesons
52 minutes
On-Demand
Free-Video
Trelis Research

Tiny Text and Vision Models - Fine-Tuning and API Setup

0 rewiews
Explore fine-tuning and API setup for tiny text and vision models, covering multi-modal architectures, LoRA adapters, and deployment strategies for custom APIs.
Add to list
15 Lesons
44 minutes
On-Demand
Free-Video
Anyscale

Fast LLM Serving with vLLM and PagedAttention

0 rewiews
Explore vLLM, an open-source library for fast LLM inference using PagedAttention. Learn how it achieves higher throughput and its implementation for efficient AI model serving.
Add to list
1 Lesons
32 minutes
On-Demand
Free-Video
Anyscale

Optimizing LLM Inference with AWS Trainium, Ray, vLLM, and Anyscale

0 rewiews
Explore cost-effective LLM inference scaling using AWS accelerators, Ray, vLLM, and Anyscale. Learn to build a complete stack on EKS and leverage advanced cluster management for enterprise-grade GenAI workloads.
Add to list
1 Lesons
46 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Cloud Native Sustainable LLM Inference in Action

0 rewiews
Explore sustainable LLM inference using cloud-native tech. Learn about energy monitoring, AI accelerator optimization, and balancing efficiency with environmental impact. Includes live demo of vLLM in a Kubernetes cluster.
Add to list
1 Lesons
1 hour 24 minutes
On-Demand
Free-Video
Tejas Kumar

Building Your Own ChatGPT-style LLM AI Infrastructure with Kubernetes

0 rewiews
Explore building AI infrastructure with Kubernetes, covering open-source LLMs, data engineering, and cost optimization. Gain insights on deployment, scalability, and choosing the right tools for AI-enabled applications.
Add to list
12 Lesons
1 hour 41 minutes
On-Demand
Free-Video
Trelis Research

How to Pick a GPU and Inference Engine for Large Language Models

0 rewiews
Explore GPU selection and inference engine optimization for large language models. Compare quantization effects, software options, and cost-efficiency across various GPUs and model sizes.
Add to list
24 Lesons
1 hour 4 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Load-Aware GPU Fractioning for LLM Inference on Kubernetes

0 rewiews
Discover how to optimize GPU utilization for LLM inference on Kubernetes through load-aware fractioning, using MIG slices to improve resource efficiency while maintaining performance standards.
Add to list
1 Lesons
39 minutes
On-Demand
Free-Video
Trelis Research

Faster Inference Using Output Predictions with OpenAI and vLLM

0 rewiews
Master advanced inference techniques for faster AI model outputs using OpenAI predictions and vLLM speculative decoding, with practical demonstrations and cost optimization strategies.
Add to list
8 Lesons
24 minutes
On-Demand
Free-Video
CNCF [Cloud Native Computing Foundation]

Architecting the Future of AI: From Cloud-Native Orchestration to Advanced LLMOps

0 rewiews
Dive into cloud-native AI architecture, exploring Kubernetes orchestration, Ray distributed computing, and vLLM for efficient large language model deployment and management.
Add to list
1 Lesons
27 minutes
On-Demand
Free-Video