Evaluating Quality and Improving LLM Products at Scale

Evaluating Quality and Improving LLM Products at Scale

Explore strategies for evaluating and enhancing large language model (LLM) products at scale in this 15-minute conference talk by Austin Bell at the AI in Production Conference. Learn how to measure the impact of prompt changes and pre-processing techniques on LLM output quality, enabling confident deployment of product improvements. Gain insights from Bell's experience as a Staff Software Engineer at Slack, focusing on developing text-based ML and Generative AI products. Discover methods to ensure consistent enhancement of generative products through effective evaluation and measurement techniques.