LLMOps: Using Nvidia TensorRT SDK for GPU Inference

LLMOps: Using Nvidia TensorRT SDK for GPU Inference

Explore the process of converting a model to Tensor-RT format in this 34-minute video tutorial. Learn how to compare throughput and inference time by varying batch size and data precision, using both native PyTorch inference and TensorRT runtime. Gain practical insights into optimizing GPU inference for machine learning models, with a focus on LLMOps techniques. Access the accompanying notebook on GitHub to follow along and implement the demonstrated concepts in your own projects.