Discover AI
NVIDIA HYMBA: A Hybrid-Head Architecture for Small Language Models with MetaTokens
Explore NVIDIA's Hymba hybrid architecture combining transformer attention and state-space models, featuring innovative meta tokens and memory optimizations for enhanced language model performance and efficiency.
YouTube32 minutesSelf-pacedFree video
Yannic Kilcher
TokenFormer - Rethinking Transformer Scaling with Tokenized Model Parameters
Delve into an innovative approach to transformer architecture scaling that treats model parameters as tokens, enabling flexible parameter adjustments without complete retraining from scratch.
YouTube28 minutesSelf-pacedFree video
StatQuest with Josh Starmer
Encoder-Only Transformers: Understanding BERT and RAG Architecture
Discover how encoder-only transformers power RAG, sentiment analysis, and classification through clear explanations of word embedding, positional encoding, and attention mechanisms.
YouTube19 minutesSelf-pacedFree video