Emergence and Grokking in Simple Neural Architectures
Dive into emergence and grokking in simple neural architectures. Explore MLPs' remarkable behaviors, feature-learning kernel machines, and their implications for understanding modern deep learning phenomena.
Scaling Insights from Infinite-Width Theory for Next Generation Architectures and Learning Paradigms
Explore scaling insights from infinite-width theory for advanced architectures and learning paradigms. Gain understanding of optimal scaling rules across various models and their impact on training stability and generalization.
The Prefrontal Cortex as a Meta-Reinforcement Learning System
Explore the prefrontal cortex's role in meta-reinforcement learning, connecting neuroscience and AI through recurrent neural networks and cognitive tasks like Harlow's experiment.
Building Makemore - Activations & Gradients, BatchNorm
Explore MLP internals, activations, gradients, and BatchNorm. Learn to diagnose deep networks, understand training challenges, and implement modern techniques for improved performance.
Neural Network Architectures for Images
Explore neural network architectures for image processing, including classification, segmentation, and enhancement. Learn about CNNs, ResNets, autoencoders, and their applications in various imaging tasks.
Neural Networks for Speech Recognition - 2013
Explore the evolution of neural networks in speech recognition, from early experiments to modern hybrid HMM/MLP approaches, with insights from a leading expert in the field.
How LLMs Might Store Facts - Multilayer Perceptrons in Transformers
Explores how large language models potentially store factual information, focusing on multilayer perceptrons in transformers and the concept of superposition in neural networks.