The Inner Workings of Large Language Models - Visualizing Self-Attention Mechanisms
Dive into the mechanics of Large Language Models, exploring self-attention mechanisms, architectural designs, and performance optimization through clear visualizations and practical examples.
Troubleshooting Deep Neural Networks - Full Stack Deep Learning - Spring 2021
Learn a systematic approach to troubleshoot deep neural networks, from starting simple to tuning hyperparameters, with practical strategies for implementation, debugging, evaluation, and improvement.
Recurrent Neural Networks - Full Stack Deep Learning - Spring 2021
Explore Recurrent Neural Networks: architecture, challenges, solutions, and applications. Dive into LSTMs, bidirectionality, attention mechanisms, CTC loss, and non-recurrent sequence models like WaveNet.
Convolutional Neural Networks - Full Stack Deep Learning - Spring 2021
Explore convolutional neural networks, from basic operations to advanced concepts like dilated convolutions and pooling. Learn about the classic LeNet architecture and gain insights into ConvNet implementation.
Deep Learning Fundamentals - Full Stack Deep Learning
Comprehensive overview of deep learning basics, covering neural networks, learning problems, optimization techniques, and architectural considerations for AI practitioners.
Coding a Neural Network - Full Stack Deep Learning - Spring 2021
Code a neural network from scratch using Google Colab, NumPy, PyTorch, and Keras. Learn linear regression and multi-layer perception models through hands-on implementation and visualization.
Troubleshooting Deep Neural Networks - Full Stack Deep Learning - March 2019
Learn effective strategies for diagnosing and fixing issues in deep neural networks, including a decision tree for systematic troubleshooting and performance optimization.
Innovations in Theoretical Computer Science 2020 - Session 9
Cutting-edge research in theoretical computer science, covering topics like distribution testing, random object sampling, probability distribution learning, agreement expansion, neural networks, and influence maximization.
Understanding Names with Neural Networks - Session 2
Explore neural network-based approaches for cross-language name matching, focusing on English/Japanese pairs and comparing traditional methods with advanced deep learning techniques.
Understanding Names with Neural Networks - Session 1
Explore neural network approaches for cross-language name matching, focusing on English/Japanese. Learn about seq2seq models, LSTMs, and CNNs for improved accuracy in transliteration and scoring.
Transformers Explained - Part 1: Generative Music AI
Comprehensive exploration of transformer architectures in AI, covering intuition, theory, and mathematical formalization, with focus on their application in music and audio processing.
NeurIPS 2023 Poster Session 4 - Highlights in Machine Learning Research
Explore cutting-edge machine learning research in temporal action segmentation, test-time adaptation, recurrent neural networks, dictionary learning, equivariant adaptation, cross-task generalization, and adversarial learning.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Explore Mamba, a novel linear-time sequence modeling architecture using selective state spaces. Learn about its advantages over Transformers and its applications in language, audio, and genomics.
Transformers for Generative Music AI - Part 2: Decoder and Music Generation
Explore transformer architecture for music AI, focusing on decoder components, masked attention, and practical tips for generating music. Dive into future research and neuro-symbolic integration for robust music generation.
Melody Generation with Transformers - Generative Music AI
Implement a transformer architecture in TensorFlow for end-to-end melody generation. Learn melody encoding, dataset preprocessing, model building, training, and inference techniques for AI-driven music creation.
Modular Language Models: Addressing Limitations of Dense Training
Explore modular language models: specialized components for distinct domains, enabling customization, parallelization, and sparse activation. Discover a new approach to collaborative LM development.
Explaining Stable Diffusion for Machine Learning Engineers
Dive deep into stable diffusion concepts, exploring diffusion models, training processes, text conditioning, and latent diffusion for machine learning engineers.
Scalable OpenAI GPT Applications in the Cloud
Explore integrating OpenAI GPT models into applications and leveraging cloud services for scalability. Learn implementation methods and strategies for building AI-powered, scalable apps.
Deep Learning Meets Nonparametric Regression: Are Weight-decayed DNNs Locally Adaptive?
Explore deep learning's curve-fitting capabilities, comparing DNNs to classical methods and uncovering insights on their strength, generalization, and the role of sparsity in neural networks.
Prioritized Training Using Reducible Holdout Loss Selection (RHO-LOSS) - Accelerating Machine Learning
Accelerate machine learning training by prioritizing data points that reduce generalization loss, improving efficiency and accuracy across various architectures and datasets.
Challenges in Deep Learning - Open Problems and Recent Observations
Explore key open problems in deep learning, focusing on continual learning and implicit biases. Gain insights from a leading researcher on crucial areas for advancing the field.