Building an AI Rap God - Cloning a Rapper Using Machine Learning
Learn to create an AI-powered rap generator using Python, combining text generation, voice cloning, and lip-syncing technologies to produce custom rap performances.
Build a Transformer for Language Classification in TensorFlow
Build a transformer-based sentiment classifier using HuggingFace's library in TensorFlow. Learn to download data, tokenize inputs, prepare datasets, define the model architecture, and train for multi-class sentiment analysis.
Longformer - The Long-Document Transformer
Explore Longformer's innovative attention mechanism for processing lengthy documents, combining local windowed and global attention to overcome Transformer limitations in handling long sequences.
Reformer - The Efficient Transformer
Discover how Reformer revolutionizes Transformer models, reducing memory usage and enabling processing of longer sequences through innovative techniques like Locality Sensitive Hashing and Reversible Networks.
Sparse Expert Models - Switch Transformers, GLAM, and More With the Authors
Explore sparse expert models in AI, discussing their architecture, routing, history, applications, and future potential. Learn about Switch Transformers, GLAM, and advancements in efficient large-scale language models.
Knowledge Is Embedded in Language Neural Networks but Can They Reason?
Explore language models' reasoning capabilities, from n-grams to transformers, and discuss future research directions in neural state machines for improved natural language understanding.
Scaling Transformer to 1M Tokens and Beyond with RMT - Paper Explained
Explore the Recurrent Memory Transformer technique for scaling transformers to process 1 million tokens, examining its strengths, weaknesses, and potential impact on long-term dependency handling in NLP tasks.
Explainable Structured Machine Learning in Similarity, Graph and Transformer Models
Explores techniques for explaining complex machine learning models, focusing on similarity, graph, and transformer structures. Discusses applications in knowledge evolution, gender bias, and task-solving.
ODSC East 2022 Keynote by Luis Vargas, Ph.D. - The Big Wave of AI at Scale
Explore the trend of larger AI models, their growing capabilities, and impact on language, vision, and multi-modality. Learn about research, engineering efforts, and implications for companies.
Understanding BigBird - Transformers for Longer Sequences
Explore BigBird's block sparse attention for long sequences, comparing it with BERT and Longformer. Learn implementation, complexity, use cases, and performance optimization tips.
Delivery of Deep Transformer NLP Models Using MLflow and AWS SageMaker for Enterprise AI
Learn to deploy deep transformer NLP models using MLflow and AWS SageMaker for enterprise AI, covering the full machine learning lifecycle and CI/CD pipelines for production-quality implementation.
Extremely Low-Bit Quantization for Transformers - tinyML Asia 2021
Explore extremely low-bit quantization techniques for Transformers, focusing on mixed precision strategies to optimize deployment on resource-limited devices without compromising accuracy.
Conversational AI with Transformer Models - Building Blocks and Optimization Techniques
Explore building conversational AI agents using transformer models, covering advantages over RNN/LSTM, knowledge distillation, and model compression techniques for efficient production deployment.
Transformer Models: Understanding Their Architecture and Functionality - Part 3
Demystifying Transformer models with visuals and examples, covering key concepts like attention mechanism, tokenization, embeddings, and fine-tuning for various NLP tasks.
Fast Multipole Attention: A Divide-and-Conquer Attention Mechanism for Long Sequences
Innovative divide-and-conquer strategy for efficient attention in long sequences, reducing complexity while maintaining global receptive field. Potential to enhance large language models with extended context processing.
Training and Optimisation of Large Transformer Models - An ATLAS and CERN Use Case
Explore CERN's implementation of transformer models for Jet Flavour Tagging, utilizing Kubeflow for machine learning and Katib for hyperparameter optimization in large-scale particle physics data analysis.
Reproducing GPT-2 (124M) from Scratch - Implementation and Optimization
Comprehensive tutorial on reproducing GPT-2 (124M) from scratch, covering implementation, optimization, training setup, and results analysis. Includes practical coding and performance tuning tips.
Llama2 7B QLora Fine Tuning with Chain-of-Thought Dataset
Fine-tune Llama2 7B model using QLora with chain-of-thought dataset. Learn step-by-step process, access provided notebooks, and explore CodeLlama implementation for enhanced machine learning capabilities.
In-Context Learning in Deep Learning - How It Works in Large and Small Language Models
Explore in-context learning in LLMs: definition, significance, Bayesian framework, and implementation. Gain insights into this surprising capability of language models and its implications for AI.
Language Models as World Models? - Understanding Representations and Semantic Control
Explore how language models encode world knowledge, their limitations in semantic accuracy, and techniques for improving their representations and controllability.
How Multilingual Is Your NLP Model?
Explore multilingual NLP challenges, language identification techniques, and BERT's capabilities for handling diverse linguistic data in the evolving landscape of global communication.