Finding Training Inefficiencies with CentML DeepView
Optimize deep learning model training with DeepView. Gain insights into performance bottlenecks, enhance resource utilization, and achieve significant throughput improvements through intuitive visualizations and targeted optimizations.
Finetuning a Large Language Model on a Custom Dataset
Discover parameter-efficient finetuning techniques and optimizations for large language models using custom datasets, enhancing model performance at scale.
Deep Learning Models for Genomes and Beyond
Explore deep learning models for genomic analysis and their applications beyond, presented by Olga Troyanskaya in this Broad Institute colloquium on AI in biomedicine.
LLMOPS: Training a Small LLM (GPT-2) - Machine Learning and Data Science
Explore training a small LLM (GPT2) locally, understanding challenges and limitations of LLM development for individuals and small teams.
The Mathematics of Scaling Laws and Model Collapse in AI
Explore the mathematics behind AI scaling laws and model collapse, focusing on the shift from linear to flattened performance relationships in large language models.
A Framework for Designing Non-Diagonal Adaptive Training Methods
Explore a framework for designing non-diagonal adaptive training methods in deep learning optimization, leveraging Fisher-Rao geometry and quasi-Newton approaches for large-scale neural networks.
Protein Structure with Transformers - Lecture 9
Explore protein structure prediction using transformer models, uncovering advanced techniques for understanding and analyzing protein folding and interactions.
The Emergence of Clusters in Self-Attention Dynamics
Explore the formation of clusters in self-attention dynamics, examining their impact on transformer models and computational processes.
Exact Solutions to the Geometric Dynamics of Signal Propagation Through Transformers
Explore exact solutions to geometric dynamics of signal propagation in transformers, predicting their behavior as computational models.
Limitations of Attention Mechanism in Transformers - Implications for Generalization and Optimization
Explore Transformer's reasoning capabilities and limitations in sequential tasks, examining optimization challenges and out-of-distribution generalization issues compared to RNNs.
Transformers, Parallel Computation, and Logarithmic Depth
Explore the computational power of transformers, their connection to parallel processing, and efficiency in solving complex tasks with logarithmic depth.
Transformers as a Computational Model
Explore Transformers as a computational model with Misha Belkin, delving into their architecture, capabilities, and implications for AI research and applications.
Highlights of Recent ICM Developments: GPU Acceleration, Applications, and More
Explore recent ICM developments, including GPU acceleration and applications. Discover advancements in molecular modeling, docking, and AI tools for computational chemistry.
Continuously Improving Large Scale Recommenders with MLOps Tools and Practices
Leverage MLOps tools and practices to operationalize and continuously improve recommendation systems at scale using NVIDIA Merlin and Kubeflow on Google Kubernetes Engine.
Scale and Accelerate Distributed Model Training in Kubernetes Clusters
Orchestrate and accelerate distributed deep learning workloads across multiple GPUs and nodes using Kubernetes, Kubeflow PytorchJob, RDMA, RoCE, and SRIOV for near-linear performance scaling.
MLOps for Deep Learning: Drift Detection and Efficient Retraining in Model Serving
Explore advanced MLOps techniques for deep learning, including drift detection, efficient retraining, and continuous deployment. Enhance model reliability and performance in production environments.
Reproducible Machine Learning - How Not to Repeat the Past
Explore techniques for organizing and reproducing deep learning experiments, including standardized logging, versioning, and templated workflows. Enhance collaboration and minimize duplication in ML projects.
WarpDrive: Orders of Magnitude Faster Multi-Agent Deep Reinforcement Learning on a GPU
Accelerate multi-agent deep reinforcement learning with WarpDrive, an open-source framework enabling end-to-end GPU execution. Explore features like automatic GPU tuning and distributed training for improved efficiency and scalability.
How to Build with AI Today: Artificial Intelligence Overview
Explore AI's evolution, from rule-based systems to GenAI and LLMs. Learn about limitations, RAG solutions, and practical applications in web development. Discover AI's future beyond chatbots.
TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems
Explore TransGNN's fusion of Transformers and Graph Neural Networks for enhanced recommender systems, improving collaborative filtering and user-item interactions.
Deep Automated Mechanism Design for Integrating Ad Auction and Allocation in Feed - T1.3
Explore deep automated mechanism design for integrating ad auction and allocation in feed, focusing on CTR, ads, and click models.