The Creation of Adam' - Michelangelo (ca. 1508 - 1512)
3
The Creation of AGI (by Adam) - ML Community
4
Envisioned excitement curve of this talk
5
What is Black-Box Optimization
6
How does an Evolution Strategy work?
7
Challenges for Modern Evolutionary Optimization?
8
What is the power of JAX for Evolutionary Optimization? Parallel/Accelerated Fitness Rollouts
9
evosax: Accelerated Evolutionary Optimization
10
Discovering New Algorithms via Meta-Learning
11
Discovering New Algorithms via Meta-Evolution
12
Why not use Meta-V instead of Meta-?
13
Discovering Evolutionary Optimizers (&)
14
White-Box Evolution Strategy: Gaussian Search
15
Learned Evolution Strategy (LES) Architecture
16
Meta-Training Details for LES Discovery BBOB Functions
17
Discovering LES: Meta-Training on Low-D BBOB
18
Evaluating LES: Brax Control Tasks
19
Scaling Meta-Distributions Improves LES Discovery
20
What Has The Learned Evolution Strategy Discovered?
21
Self-Referential Meta-Evolution of Learned ES
22
How does a Genetic Algorithm work?
23
Learned Genetic Algorithms (LGA) 9
24
LGA Generalizes to HPO-B & Neuroevolution Tasks
25
LGA Applies Adaptive Elitism & MR Adaptation
26
On Survivorship Bias & The Hardware Lottery (Hooker, 21)
Description:
Watch a 50-minute AutoML seminar exploring the discovery of black-box optimizers through evolutionary meta-learning. Learn how to develop effective update rules for evolution strategies and genetic algorithms using meta-learning approaches. Dive into the implementation of black-box optimizers with self-attention-based architectures that ensure update rule invariance to candidate solution ordering. Understand how meta-evolving on low-dimensional analytic optimization problems leads to discovering new evolutionary optimizers that generalize across unseen optimization challenges, population sizes, and horizons. Explore the performance comparison against neuroevolution baselines in supervised and continuous control tasks. Examine the neural network components, reverse engineering of learned optimizers into explicit heuristic forms, and the transfer of neural network-based operators to white-box optimizers. Discover how a self-referential training approach can be used to train an evolution strategy from scratch using learned update rules for meta-learning loops. The talk includes practical demonstrations using the JAX-based Evolution Strategies library evosax and covers both Learned Evolution Strategy (LES) and Learned Genetic Algorithms (LGA) implementations.
Read more
Discovering Black-Box Optimizers via Evolutionary Meta-Learning