Gradient Descent, Step-by-Step
Comprehensive step-by-step explanation of Gradient Descent, covering its application in Machine Learning, optimization techniques, and variations like Stochastic Gradient Descent.
Large Scale Machine Learning and Convex Optimization - Lecture 3
Explore advanced stochastic optimization techniques for large-scale machine learning, including novel Newton-based methods and combinations of batch and online algorithms for improved convergence rates.
SecureML - A System for Scalable Privacy-Preserving Machine Learning
Explores scalable privacy-preserving machine learning techniques for linear regression, logistic regression, and neural networks using secure two-party computation, with efficient protocols and MPC-friendly alternatives to nonlinear functions.
Trustworthy Machine Learning: Challenges and Frameworks
Exploring the security challenges in machine learning, this talk discusses strategies for building trustworthy AI systems by analyzing data influence, addressing vulnerabilities, and enhancing interpretability.
Optimisation
Explore optimization techniques for neural networks, including gradient descent, SGD, momentum, adaptive methods, and normalization layers, with real-world applications in MRI technology.
Stochastic Gradient Descent and Backpropagation
Explore stochastic gradient descent, backpropagation, and neural network implementation in PyTorch. Learn practical tricks, basic modules, and gradient computation for effective deep learning.
PyTorch - A Modern Library for Machine Learning with Adam Paszke
Explore PyTorch's capabilities in machine learning, from research to production. Learn about its innovative solutions for efficient inference and its versatility across various ML scenarios.
Insights on Gradient-Based Algorithms in High-Dimensional Learning
Explore gradient-based algorithms in high-dimensional learning, covering statistical physics solutions for performance analysis in nonconvex settings like spiked mixed matrix-tensor model, perceptron, and phase retrieval.
The Information Bottleneck Theory of Deep Neural Networks
Explore the Information Bottleneck Theory's application to deep neural networks, examining statistical learning, information theory, and stochastic gradient descent in brain data analysis.
Implicit Regularization I
Explore implicit regularization in deep learning, covering boosting, complexity control, optimization landscapes, and stochastic gradient descent for effective model training.
On the Foundations of Deep Learning - SGD, Overparametrization, and Generalization
Explore foundations of deep learning, focusing on SGD, overparametrization, and generalization. Delve into optimization dynamics, margin theory, and the impact of network architecture on performance.
Studying Generalization in Deep Learning via PAC-Bayes
Explore PAC-Bayes theory's application to deep learning generalization, focusing on risk bounds, optimal priors, and SGD prediction for neural networks.
Machine Learning: From ABCs to DEFs
Explore machine learning fundamentals, from basic concepts to practical applications, focusing on people, data, and code. Learn to build simple AI models and understand key ML principles.
SGD in the Large - Average-Case Analysis, Asymptotics, and Stepsize Criticality
Explore average-case analysis, asymptotics, and stepsize criticality in stochastic gradient descent, with insights on optimization problems and stochastic momentum techniques.
Stochastic Gradient Descent and Machine Learning - Lecture 1
Explore optimization methods, gradient descent, Newton's method, and convexity in machine learning. Gain insights into iterative algorithms and their applications in statistical physics and data analysis.
Implicit and Explicit Regularization in Deep Neural Networks
Explores implicit and explicit regularization in deep neural networks, connecting learning algorithms to H-infinity control and explaining convergence behavior in over-parametrized models, offering insights into generalization abilities.
Gradient Descent, Stochastic Gradient Descent, and Acceleration - Part 2
Explore advanced gradient-based optimization methods for large-scale machine learning, including stochastic gradient descent and accelerated versions, with convergence analysis and ODE connections.
Gradient Descent and Stochastic Gradient Descent
Explore gradient descent and stochastic gradient descent in deep neural networks, covering convergence rates, learning rate effects, and challenges in optimization algorithms.
Fit Without Fear - An Over-Fitting Perspective on Modern Deep and Shallow Learning
Explore over-fitting in modern machine learning, challenging conventional wisdom and presenting kernel methods as a competitive alternative to deep learning. Insights on interpolation, generalization, and stochastic gradient descent.
Introduction to Optimization
Learn optimization techniques for machine learning, including likelihood estimation, gradient descent, and regularization, with practical examples and applications.
Federated Learning with Formal User-Level Differential Privacy Guarantees
Explore federated learning with differential privacy guarantees, covering key concepts, challenges, and algorithms like DP-SGD and DP-FTRL, with insights on privacy-utility trade-offs and future directions.