Neural Nets for NLP 2021 - Document-Level Models
Explores advanced NLP techniques for document-level tasks, including long-document modeling, entity coreference, and discourse parsing, with a focus on neural network approaches and their applications.
GPT-2 and Transformers
Explore text-based AI models, focusing on GPT-2 and Transformers. Learn about their architecture, capabilities, and potential applications in natural language processing.
Foundation Models - FSDL 2022
Explore Transformers, Large Language Models, prompt engineering, and innovative applications like CLIP and image generation in this comprehensive overview of foundation models.
Transformers - From Zero to Hero
Comprehensive deep dive into transformer architecture, covering key concepts from self-attention to tokenization. Includes hands-on coding for both encoder and decoder implementations.
Lectures on Deepsets, Graph Neural Networks and Transformers with Applications - Lecture 6
Explore advanced deep learning architectures like Deepsets, Graph Neural Networks, and Transformers, with applications in high energy physics data analysis and research.
Deepsets, Graph Neural Networks, and Transformers in Machine Learning - Lecture 7
Explore advanced machine learning techniques for high energy physics, including Deepsets, Graph Neural Networks, and Transformers, to analyze complex particle physics data.
OPT-175B Language Model Tutorial - GPT-3 Alternative Using Hugging Face
Learn about Meta AI's OPT-175B language model, including a live demo and hands-on coding for text generation using Hugging Face Transformers. Explore this open-source alternative to GPT-3.
Object Detection with Transformers - From Training to Deployment
Explore cutting-edge object detection using transformers, from fundamental concepts to advanced techniques. Learn scalable training with Determined AI and deployment using MLflow.
Transformer Parts and Functions
Comprehensive overview of transformer components and functions, including core, windings, bushings, tank, relays, and cooling systems. Explains key parts and their roles in power engineering.
Building and Deploying a Semantic Search Engine on Hugging Face
Build and deploy a semantic search engine on Hugging Face. Learn NLP techniques, implement search algorithms, and create an API. Gain practical skills in context-aware search and sentiment analysis.
RWKV- Reinventing RNNs for the Transformer Era
Explore RWKV, a novel architecture combining Transformer and RNN strengths. Learn about its linear attention mechanism, scalability, and performance in NLP tasks.
Pretraining Task Diversity and the Emergence of Non-Bayesian In-Context Learning for Regression
Explores how task diversity in pretraining affects transformers' in-context learning abilities for regression, revealing a threshold beyond which models can solve novel tasks and outperform Bayesian estimators.
Deepsets, Graph Neural Networks, and Transformers in Machine Learning - Lecture 4
Explore advanced deep learning architectures like Deepsets, Graph Neural Networks, and Transformers, focusing on their applications in high energy physics data analysis and research.
An Observation on Generalization
Explore insights on generalization in large language models and transformers with OpenAI's Ilya Sutskever, delving into the capabilities and implications of advanced AI systems.
Possible Impossibilities and Impossible Possibilities
Explore the limits and potential of large language models, questioning fundamental constraints and alternative paths beyond scale in this thought-provoking discussion on AI's future.
Sparks of Artificial General Intelligence
Exploring GPT-4's advanced capabilities and the potential for smaller models to exhibit AGI-like properties through high-quality data and efficient training methods.
Interpretability via Symbolic Distillation
Explore symbolic distillation techniques for enhancing interpretability in large language models and transformers, focusing on practical applications and theoretical insights.
Scaling Data-Constrained Language Models
Explore strategies for scaling language models in data-constrained environments, addressing data repetition, compute optimization, and mitigating scarcity challenges.
Understanding the Origins and Taxonomy of Neural Scaling Laws
Explore the foundations and classification of neural scaling laws, examining their impact on large language models and transformers in AI development.
Retentive Network - A Successor to Transformer for Large Language Models
Explores Retentive Network as an alternative to Transformer, offering training parallelism and efficient inference for large language models. Discusses retention mechanism, computation paradigms, and experimental results.
ChatGPT Explained - All You Need to Know - Tutorial
Explore ChatGPT's fundamentals, applications, and underlying technologies. Learn about Generative AI, GPT models, and prompt engineering techniques reshaping digital communication and AI interactions.