CS480-680 Lecture 19 - Attention and Transformer Networks
Explore attention mechanisms, transformer networks, and their applications in machine learning, including multihead attention and normalization techniques.
MIT Introduction to Deep Learning
Comprehensive introduction to deep learning fundamentals, covering perceptrons, neural networks, loss functions, gradient descent, backpropagation, and regularization techniques.
Deep Learning for Scientific Computing - Two Stories on the Gap Between Theory & Practice - Ben Adcock
Explores deep learning in scientific computing, highlighting limitations and recent theoretical advancements in high-dimensional function approximation and inverse problems for imaging, aiming to bridge the gap between theory and practice.
Solving Overfitting in Neural Networks
Learn techniques to identify and prevent overfitting in neural networks, including early stopping, audio data augmentation, dropout, and L1/L2 regularization, with practical implementation in a music genre classifier.
Implementing a Neural Network for Music Genre Classification
Implement a music genre classifier using TensorFlow and MFCC features. Learn key deep learning concepts like binary/multiclass classification, ReLUs, batching, and overfitting while building the neural network.
How to Implement a Simple Neural Network With TensorFlow 2
Learn to implement a simple neural network using TensorFlow 2 and Keras for arithmetic sum, covering model building, training, testing, evaluation, and prediction.
Training a Neural Network - Implementing Backpropagation and Gradient Descent from Scratch
Implement backpropagation and gradient descent from scratch in Python, training a neural network to perform arithmetic sum. Includes step-by-step coding and explanations.
Training a Neural Network - Backward Propagation and Gradient Descent
Explore the theory and mathematics behind neural network training, focusing on backpropagation and gradient descent algorithms for deep learning enthusiasts.
Implementing a Neural Network From Scratch in Python
Step-by-step implementation of a neural network using Python and numpy, focusing on data representation and forward propagation for a Multilayer Perceptron.
Computation in Neural Networks
Explore Multilayer Perceptrons: components, structure, and signal processing. Learn about neurons, weights, layers, and activations in neural networks for a comprehensive understanding of distributed computation.
ComfyUI - Node Based Stable Diffusion UI
Explore ComfyUI, a node-based interface for Stable Diffusion, offering modular AI image generation with advanced techniques like ControlNET, T2I, and Lora for unparalleled creative control.
Blowing Up Transformer Decoder Architecture
Detailed exploration of Transformer Decoder architecture, covering key components like masked self-attention, cross-attention, and position encoding. Includes implementation insights and training considerations.
Integrating Constraints into Deep Learning Architectures with Structured Layers
Explore integrating constraints into deep learning using structured layers, covering deep equilibrium models, implicit layers, and weight-tied networks for enhanced architecture design.
Reinforcement Learning via an Optimization Lens
Explore reinforcement learning through an optimization lens, covering MDPs, Bellman equations, and novel approaches to solving RL problems using linear programming and primal-dual formulations.
Knowledge Is Embedded in Language Neural Networks but Can They Reason?
Explore language models' reasoning capabilities, from n-grams to transformers, and discuss future research directions in neural state machines for improved natural language understanding.
Sample-Complexity of Estimating Convolutional and Recurrent Neural Networks
Explore sample-complexity in estimating neural networks, focusing on CNN and RNN architectures. Learn about generative models, minimax analysis, and key results through theoretical proofs and practical experiments.
The Software GPU - Making Inference Scale in the Real World
Discover how to achieve GPU-level performance for deep learning inference using only software on commodity CPUs, challenging conventional wisdom and offering a cost-effective, flexible solution for scaling AI models.
Learning with Limited Labels - ODSC West 2020
Explore advanced techniques for machine learning with limited labeled data, including domain adaptation, few-shot learning, and self-supervised learning to enhance model performance across diverse domains and novel categories.
Deep Ensembles: A Loss Landscape Perspective
Explore deep ensemble learning through loss landscapes, examining function similarity, subspace sampling, and prediction space analysis for improved model performance.
Exploring the Origins and Prevalence of Texture Bias in Convolutional Neural Networks
Explore texture bias in CNNs: origins, prevalence, and implications. Examine dataset influence, training objectives, and model architectures to understand and mitigate this phenomenon in computer vision.
Deep Dive into the Transformer Encoder Architecture
Detailed exploration of transformer encoder architecture, covering embeddings, positional encodings, self-attention mechanisms, and key components for effective natural language processing.