Scaling AI at Autodesk with Ray and Metaflow
Explore a 15-minute conference talk from Ray Summit 2024 where Thomas Gale and Peter Meltzer from Autodesk, alongside Savin Goyal from Outerbounds, demonstrate how Autodesk leverages Ray and Metaflow for large-scale 3D generative AI operations. Learn about their innovative integration of Ray's distributed computing with Metaflow's workflow orchestration, including the processing of terabytes of data and training of advanced 3D models. Discover their custom Metaflow flows designed for Ray integration, implementation strategies for both ephemeral and existing clusters, and their open-source contributions like the @metaflow_ray decorator. Gain practical insights into managing complex AI pipelines in production environments, covering everything from data extraction and deduplication to model training, while maintaining effective artifact tracking and metadata management.