Deep Learning for Sequences in Quantitative Finance
Overview of deep learning sequence methods in quantitative finance, covering feature extraction, return forecasting, portfolio allocation, and trading execution. Explores applications and challenges in the investment process.
Efficiently Modeling Long Sequences with Structured State Spaces - Albert Gu
Explore structured state space models for efficient long sequence modeling across modalities, featuring S4's innovative approach to handling extended dependencies in various data types.
Stanford Seminar - Decision Transformer: Reinforcement Learning via Sequence Modeling
Innovative approach to reinforcement learning using sequence modeling and Transformer architecture, offering simplicity and scalability while matching or exceeding state-of-the-art performance in various tasks.
Computational Benefits and Limitations of Transformers and State-Space Models
Explore mechanisms enabling retrieval, copying, and length generalization in language models. Compare Transformers' excellence with LSTM and state-space models' limitations in sequence modeling tasks.
MIT 6.S191 - Recurrent Neural Networks
Explore recurrent neural networks, LSTMs, and attention mechanisms for sequence modeling. Learn backpropagation through time, gradient issues, and practical applications in deep learning.
MIT: Recurrent Neural Networks
Explore deep sequence modeling with recurrent neural networks, including LSTMs and attention mechanisms. Learn to tackle challenges like vanishing gradients and long-term dependencies in sequence tasks.
Sequence Modeling with Neural Networks
Explore sequence modeling with neural networks, focusing on RNNs, their challenges, and solutions like gated cells. Learn applications in music generation and machine translation.
Transformers in Time Series: A Survey
Explore the application of Transformer models in time series analysis, covering recent advancements and potential future directions in this rapidly evolving field.
Autoregressive Long-Context Music Generation with Perceiver AR
Explore autoregressive long-context music generation using Perceiver AR, presented by DeepMind's Cătălina Cangea. Gain insights into cutting-edge AI techniques for creating music.
Mamba AI: Understanding Selective State Space Models as Transformer Alternatives
Explore the groundbreaking MAMBA architecture and its potential to revolutionize AI through selective state space models, offering a compelling alternative to traditional transformers.
Recurrent Neural Networks - Full Stack Deep Learning - Spring 2021
Explore Recurrent Neural Networks: architecture, challenges, solutions, and applications. Dive into LSTMs, bidirectionality, attention mechanisms, CTC loss, and non-recurrent sequence models like WaveNet.
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Explore Mamba, a novel linear-time sequence modeling architecture using selective state spaces. Learn about its advantages over Transformers and its applications in language, audio, and genomics.
Can Wikipedia Help Offline Reinforcement Learning? - Paper Explained
Exploring how pre-training on Wikipedia improves offline reinforcement learning models, enhancing performance, reducing parameters, and revealing connections between language and RL domains.
Infinite Memory Transformer - Research Paper Explained
Explores innovative Transformer model with unbounded memory, enabling processing of arbitrarily long sequences. Discusses continuous attention mechanisms, sticky memories, and potential applications in language modeling.
Recurrent Neural Networks, Transformers, and Attention
Explore recurrent neural networks, transformers, and attention mechanisms in deep learning. Learn design principles, applications, and advanced concepts like LSTM and backpropagation through time.
Retentive Network - A Successor to Transformer for Large Language Models
Explores Retentive Network as an alternative to Transformer, offering training parallelism and efficient inference for large language models. Discusses retention mechanism, computation paradigms, and experimental results.
Decision Transformer - Reinforcement Learning via Sequence Modeling
Explore how Decision Transformer reframes offline reinforcement learning as sequence modeling, leveraging Transformer architecture to generate optimal actions based on desired returns, past states, and actions.
Integrating Constraints into Deep Learning Architectures with Structured Layers
Explore integrating constraints into deep learning using structured layers, covering deep equilibrium models, implicit layers, and weight-tied networks for enhanced architecture design.
Getting Personal - Deep Learning Methods for Large Scale Personalization
Explore deep learning methods for large-scale personalization, covering sequence modeling, computer vision, and practical applications in this insightful talk by Noa Barbiro.
Recurrent Neural Networks and Transformers
Comprehensive exploration of recurrent neural networks, LSTMs, and attention mechanisms in deep learning, covering theory, implementation, and applications for sequence modeling tasks.
Elastic Decision Transformer Explained
Explore cutting-edge Elastic Decision Transformer, combining transformers and reinforcement learning. Learn how it enhances traditional models by adaptively modifying historical context during inference.