Advanced Algorithms - COMPSCI 224
Explore cutting-edge algorithmic techniques through comprehensive lectures, enhancing problem-solving skills and deepening understanding of complex computational challenges.
Theory of Computation
Explore finite automata, regular languages, context-free grammars, and Turing machines. Learn about computational models, their properties, and fundamental concepts in theoretical computer science.
Introduction to the Theory of Computing - Stanford
Explore fundamental concepts of theoretical computer science, including automata, computability, and complexity theory, with practical applications and algorithmic fairness.
Taskonomy - Disentangling Task Transfer Learning
Explore task transfer learning, higher-order transfers, and task-specific networks. Discover global taxonomies, evaluate computed structures, and assess generalization to novel tasks.
Computing with Dynamical Systems - From Implementations Towards Novel Concepts
Explore dynamical systems in computing, from practical implementations to cutting-edge concepts. Gain insights into complex systems and interdisciplinary physics applications.
Towards Understanding Modern Alchemy - Transformers as a Computational Model
Explore in-context learning of formal languages, comparing Transformers to other models and examining specialized "n-gram heads" for improved performance in language modeling.
Associative Memories as a Building Block in Transformers
Explore associative memories in Transformers, their role in storing knowledge, and how gradients and over-parameterization affect their learning and capacity.
The Emergence of Clusters in Self-Attention Dynamics
Explore the formation of clusters in self-attention dynamics, examining their impact on transformer models and computational processes.
Interpretability Agents: Automating and Scaling Model Interpretation
Explore automated interpretability agents for scaling model interpretation, enabling efficient analysis of AI systems through hypothesis-driven experimentation and modular design.
Understanding and Improving Efficient Language Models
Explore efficient language models, focusing on associative recall's impact on performance. Learn about new hardware-efficient architectures expanding quality-efficiency frontiers in machine learning.
Transformer Expressivity and Formal Logic
Explore the relationship between transformers and formal logic, uncovering their abilities, limitations, and implications for computational models.
Using Recurrence to Achieve Weak to Strong Generalization
Explore recurrent architectures for weak-to-strong generalization in reasoning models, enhancing problem-solving capabilities beyond training data.
Transformers as a Computational Model - What Was Revolutionized by the "Transformer Revolution?"
Explore the paradigm shift in machine learning brought by transformers, examining new research questions and recent findings in AI development.
Exact Solutions to the Geometric Dynamics of Signal Propagation Through Transformers
Explore exact solutions to geometric dynamics of signal propagation in transformers, predicting their behavior as computational models.
Language Generation in the Limit - Transformers as a Computational Model
Explore language generation using basic specifications, contrasting with language learning models. Discover non-trivial guarantees for generating valid new strings from unknown languages.
Computational Benefits and Limitations of Transformers and State-Space Models
Explore mechanisms enabling retrieval, copying, and length generalization in language models. Compare Transformers' excellence with LSTM and state-space models' limitations in sequence modeling tasks.
Iterated Models: Expressive Power, Learning, and Chain of Thought
Explore sequence-to-sequence models, focusing on transformers, their computational power, and learning complexities. Gain insights into iterated functions and chain of thought.
The Parallelism Tradeoff: Understanding Transformer Expressivity Through Circuit Complexity
Explore transformer neural nets' computational power and limitations through circuit complexity analysis, revealing insights into parallelism tradeoffs in large-scale model architectures.
Limitations of Attention Mechanism in Transformers - Implications for Generalization and Optimization
Explore Transformer's reasoning capabilities and limitations in sequential tasks, examining optimization challenges and out-of-distribution generalization issues compared to RNNs.
Transformers as a Computational Model
Explore Transformers as a computational model with Misha Belkin, delving into their architecture, capabilities, and implications for AI research and applications.
Modularity, Attention and Credit Assignment Computations - Anirudh Goyal
Explore modularity, attention, and credit assignment in computational models, focusing on complex generalization, independent mechanisms, and selective activation for improved learning and problem-solving.