Federated Learning with Formal User-Level Differential Privacy Guarantees
Explore federated learning with formal user-level differential privacy guarantees in this 59-minute invited talk from PPML 2022. Delve into topics such as non-convex learning, cross-device federated learning, differentially private stochastic gradient descent, and DP-Federated Averaging. Examine challenges in amplification by sampling, noise accumulation in prefix sums, and tree aggregation. Investigate DP-Follow-the-regularized leader (DP-FTRL) and its online learning properties. Analyze privacy-utility trade-offs using Stackoverflow as an example, and discover a production model with formal differential privacy. Gain insights into matrix factorization views of prefix sum estimation and DP prefix sum. Conclude with future directions and acknowledgements in this comprehensive exploration of privacy-preserving machine learning techniques.