Главная
Study mode:
on
1
Introduction
2
Agenda
3
What is ModelServing
4
Deployment Strategy
5
KServe
6
Pod Per Model
7
ModelMesh
8
ModelMesh Features
9
ModelMesh Architecture
10
Monitoring
11
Prometheus
12
Grafana Dashboard
13
Model Loading
14
Serving Runtime
15
Why KServe
16
Step by Step
17
Example
18
Model Mesh Example
19
In Practice
Description:
Explore the process of creating a custom serving runtime in KServe ModelMesh to serve machine learning models in this 30-minute conference talk. Gain insights into ModelMesh key features, learn how to build a new container image supporting desired frameworks, and understand the deployment strategy. Discover the advantages of KServe and ModelMesh architecture, including monitoring capabilities with Prometheus and Grafana dashboards. Follow along with hands-on demonstrations of loading models in existing model servers and running predictions using custom serving runtimes. Delve into practical examples and step-by-step instructions for implementing ModelMesh in real-world scenarios.

Creating a Custom Serving Runtime in KServe ModelMesh - Hands-On Experience

Linux Foundation
Add to list
0:00 / 0:00