Shaping AI Benchmarks and Open-Source Development - Percy Liang Interview
Explore advancements in AI benchmarking and the role of open-source in AI development in this 53-minute podcast episode featuring Together AI co-founder and Stanford Associate Professor Percy Liang. Delve into the development of HELM, a robust framework for evaluating language models, and understand how it enhances transparency and effectiveness in AI benchmarks. Gain insights on the importance of open-source models in democratizing AI development and the challenges of English language bias in global AI applications. Learn about selecting AI models using leaderboards, contributing to AI benchmarking, and the complexities of evaluating AI agents. Discover the mission of TogetherAI for open AI development and the fascinating intersection of AI with music creation. This comprehensive discussion offers valuable perspectives on how benchmarks are shaping AI's future, addressing both technological progress and the push for more equitable and inclusive technologies.