Inside TensorFlow - Parameter Server Training
Save Big on Coursera Plus. 7,000+ courses at $160 off. Limited Time Only! Grab it Explore parameter server training, a data-parallel method for scaling up model training across multiple machines, in this TensorFlow video. Learn about adaptive learning rates, synchronous parameter server training, and evaluation techniques. Discover the challenges and benefits of multi-client and single-client setups, and delve into custom training loops with parameter servers. Examine current API limitations, variable sharding, and ongoing developments in runtime, performance, and scalability. Gain insights into distributed functions, large embedding models, and performance comparisons with Estimator. Understand worker profiles, multi-step packing, and fault tolerance strategies for preemptions and failures. Explore large-scale fault tolerance testing, running jobs with preemptible resources, and utilizing the multi-worker testing framework and MLCompass dashboard.