Deconstructing Lottery Tickets - Zeros, Signs, and the Supermask
Explores the Lottery Ticket Hypothesis, analyzing key components of sparse networks and uncovering insights on weight initialization, sign importance, and the concept of Supermasks in neural network training.
Backpropagation and the Brain
Explore a biologically plausible variant of backpropagation and its potential role in brain learning, discussing feedback connections and error signal approximation in cortical networks.
Feature Visualization and the OpenAI Microscope
Explore neural network visualizations using OpenAI Microscope, understanding feature extraction in ImageNet classifiers and techniques for visualizing internal model workings.
Agent57 - Outperforming the Atari Human Benchmark
Explore DeepMind's Agent57, the first AI to surpass human performance in all 57 Atari games. Learn about its innovative approach to reinforcement learning and meta-learning for exploration-exploitation balance.
Axial Attention & MetNet - A Neural Weather Model for Precipitation Forecasting
Explore a neural network model for precise precipitation forecasting using axial attention, offering high-resolution predictions up to 8 hours ahead with improved accuracy over traditional methods.
Deep Learning for Symbolic Mathematics
Explore neural networks solving complex mathematical problems like symbolic integration and differential equations, outperforming traditional Computer Algebra Systems.
Accelerating Deep Learning by Focusing on the Biggest Losers
Explore a technique to speed up neural network training by prioritizing high-loss examples, reducing backpropagation steps, and achieving faster convergence in deep learning models.
Dynamic Routing Between Capsules
Explore Capsule Networks, a novel neural network architecture using capsules and dynamic routing for improved object recognition and handling of spatial relationships.
Manifold Mixup: Better Representations by Interpolating Hidden States
Explore Manifold Mixup, a regularization technique that improves neural network performance by interpolating hidden representations, leading to smoother decision boundaries and more robust predictions.
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Explore how Batch Normalization accelerates deep network training by reducing internal covariate shift, enabling higher learning rates and improved model performance.
Attention Is All You Need
Explore the groundbreaking Transformer architecture, revolutionizing sequence transduction with attention mechanisms. Learn its superior performance in machine translation and generalization to other NLP tasks.
Deep Image Reconstruction from Human Brain Activity
Explores innovative technique for reconstructing visual imagery from brain activity using fMRI and deep neural networks, demonstrating potential for "mind-reading" technology and insights into neural representations.
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
Explore the groundbreaking Lottery Ticket Hypothesis, revealing how neural networks may contain optimal sub-networks from initialization, potentially revolutionizing network efficiency and training.
Implicit MLE- Backpropagating Through Discrete Exponential Family Distributions
Explore backpropagation through discrete exponential family distributions, enabling integration of combinatorial optimizers in neural networks for enhanced learning capabilities.
TensorFlow Binary and Multi-class Classification Tutorial
Learn TensorFlow for binary and multi-class classification, covering data handling, model building, optimization, and evaluation techniques for effective machine learning implementations.
TensorFlow Tutorial 2022
Comprehensive TensorFlow tutorial covering tensors, data handling, neural networks, and model building. Learn to create, compile, and evaluate machine learning models using practical examples.
Learning How to Learn Efficiently - Nando de Freitas, DeepMind
Explore efficient learning techniques for neural networks to solve multiple tasks with limited data, focusing on developing adaptable "prior" networks for rapid task-specific learning.
Neural Networks as Interacting Particle Systems
Explore neural networks as interacting particle systems, examining their mathematical foundations and applications in complex systems modeling across various scientific disciplines.
The Truck Backer-Upper
Explore neural network applications in vehicle control, focusing on the Truck Backer-Upper problem. Learn about state transitions, training strategies, and Bayesian neural networks.
ConvNets in Practice
Explore practical applications of ConvNets in object detection, face recognition, and semantic segmentation. Learn about network architectures, performance comparisons, and real-world implementations in robotics and image processing.
Parameter Sharing - Recurrent and Convolutional Nets
Explore parameter sharing in recurrent and convolutional neural networks, covering hypernetworks, RNNs, LSTMs, attention mechanisms, and ConvNets, with insights on architecture, training, and biological inspiration.