Basics of Neural Networks - Lecture 2
Explore neural network fundamentals, including architecture, training, and optimization techniques for efficient machine learning model development.
Prompt Tuning Explained
Explore prompt tuning techniques for optimizing pre-trained language models. Learn about soft prompts, gradient descent, and improving performance on downstream tasks while maintaining model efficiency.
Introduction to Artificial Neural Networks - JSALT 2023
Explore artificial neural networks: fundamentals, architectures, and applications in this comprehensive introduction from JSALT 2023 workshop.
Shuffling-based Stochastic Optimization Methods: Bridging the Theory-practice Gap
Explore shuffling-based stochastic optimization methods and their practical applications in bridging theoretical concepts with real-world implementation.
Implementation of Backtracking Line Search in Deep Neural Networks - Theory and Practice
Explore backtracking line search implementation in deep neural networks, covering theoretical foundations and practical applications for optimizing model performance.
The Elusive Generalization and Easy Optimization in Machine Learning - Part 1
Explore generalization in machine learning and optimization by gradient descent, focusing on recent developments and empirical findings in neural networks that challenge traditional theoretical foundations.
The Elusive Generalization and Easy Optimization - Part 2
Explore generalization in machine learning and optimization of non-convex systems through gradient descent, focusing on recent developments and empirical findings in neural networks.
Towards Understanding Deep Classifiers Through Neural Collapse
Explore neural collapse in deep classifiers, examining its impact on model behavior and performance. Gain insights into this phenomenon's role in understanding and improving deep learning systems.
Online Optimization Meets Federated Learning - Tutorial
Explore state-of-the-art theoretical results in online optimization, federated learning, and their intersection. Gain insights into regret analysis, data heterogeneity, and distributed online optimization.
The Elusive Role of High Dimensions in Modern Optimization and Generalization
Delve into the fascinating role of high-dimensionality in deep learning optimization, exploring how modern neural networks challenge traditional theoretical constraints and achieve unprecedented success.
Distributed Training: Hybrid Parallelism and Gradient Optimization - Lecture 20
Delve into advanced distributed training concepts, covering hybrid parallelism, gradient compression techniques, and strategies for optimizing bandwidth and latency in ML systems.
How Transformers Learn Causal Structure with Gradient Descent
Delve into the mathematical foundations of how transformer models learn and encode causal relationships through self-attention mechanisms and gradient-based training algorithms.