Transformers for Generative Music AI - Part 2: Decoder and Music Generation
Explore transformer architecture for music AI, focusing on decoder components, masked attention, and practical tips for generating music. Dive into future research and neuro-symbolic integration for robust music generation.
Pytorch Transformers from Scratch - Attention Is All You Need
Implement Pytorch Transformers from scratch, exploring the "Attention is all you need" paper. Learn attention mechanisms, encoder-decoder architecture, and practical application.
The Sensory Neuron as a Transformer - Permutation-Invariant Neural Networks for Reinforcement Learning - Paper Explained
Explore Google Brain's "Attention Neuron" model for reinforcement learning, featuring permutation-invariant neural networks and their application in various experiments and visualizations.
Recreate Google Translate - Model Training
Explore model training and testing for Neural Machine Translation, recreating Google Translate using mT5 and Hugging Face. Practical implementation and demo of multilingual translations.
Non-Parametric Transformers - Paper Explained
Explore non-parametric transformers: innovative AI model for tabular data, relational reasoning, and causal mechanisms. Learn architecture, loss function, and connections to other neural networks.
Rethinking Attention with Performers
Explore a novel approach to efficient attention mechanisms in Transformers using random positive orthogonal features, enabling linear-time approximation of attention matrices for scalable deep learning architectures.
LambdaNetworks- Modeling Long-Range Interactions Without Attention
Explore LambdaNetworks, a novel approach to modeling long-range interactions in AI without traditional attention mechanisms, offering improved efficiency and performance in computer vision tasks.
Hopfield Networks Is All You Need - Paper Explained
Explores the connection between Hopfield networks and transformer attention, analyzing BERT through this lens and applying Hopfield attention to immune repertoire classification.
Avoiding Catastrophe - Active Dendrites Enable Multi-Task Learning in Dynamic Environments
Explore how active dendrites in neural networks mitigate catastrophic forgetting, enabling multi-task learning and adaptation in dynamic environments through context-sensitive gating and sparse representations.
Train Short, Test Long - Attention With Linear Biases Enables Input Length Extrapolation
ALiBi: A novel attention mechanism enabling transformers to process longer sequences than trained on. Uses linear biases instead of position encodings, improving efficiency and extrapolation capabilities.
Not All Memories Are Created Equal - Learning to Forget by Expiring
Explore Facebook AI's Expire-Span, a Transformer XL variant that dynamically assigns expiration dates to signals, enabling efficient handling of long sequences while maintaining performance and reducing resource consumption.
Emerging Properties in Self-Supervised Vision Transformers - Facebook AI Research Explained
Explore Facebook AI's DINO system, combining self-supervised learning with Vision Transformers for impressive image analysis without labels. Learn about its architecture, methodology, and groundbreaking results in computer vision.
Hugging Face Transformers - The Basics - Practical Coding Guides - NLP Models (BERT/RoBERTa)
Learn the basics of Hugging Face Transformers library, including installation, pre-defined pipelines, and implementing models in PyTorch. Explore tokenization, attention masks, and model outputs in this practical coding guide.
Vision Transformers Explained + Fine-Tuning in Python
Explore Vision Transformers (ViT) for image classification, from theory to implementation. Learn about attention mechanisms, patch embeddings, and fine-tuning using Hugging Face in Python.
A Gentle Visual Intro to Transformer Models
Explore key concepts of Transformer models, including encoder, decoder, and attention, through a visual and accessible explanation by ML expert Jay Alammar.
DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis
Learn about DF-GAN, a deep fusion architecture for text-to-image synthesis. Explore its simplified backbone, gradient penalty, and deep fusion block, with quantitative and qualitative results.
Neural Nets for NLP 2019 - Attention
Explore attention mechanisms in neural networks for NLP, covering basic concepts, improvements, specialized varieties, and a case study on the Transformer model.
Neural Nets for NLP 2017 - Attention
Comprehensive exploration of attention mechanisms in neural networks for NLP, covering various types, improvements, and applications, with a focus on the Transformer model.
Neural Nets for NLP 2021 - Attention
Explore attention mechanisms in neural networks for NLP, covering key concepts, improvements, and specialized varieties with practical examples and applications.
CMU Multilingual NLP - Machine Translation-Sequence-to-Sequence Models
Explore conditional language modeling, sequence-to-sequence models, generation methods, attention mechanisms, and transformers in multilingual natural language processing.
Neural Nets for NLP 2020 - Attention
Explore attention mechanisms in neural networks for NLP, covering types, improvements, and specialized varieties. Dive into the "Attention is All You Need" paper and its impact on the field.