Formal Languages and Automata for Reward Function Specification and Efficient Reinforcement Learning
Explore formal languages and automata for reward function specification and efficient reinforcement learning in this comprehensive lecture by Sheila McIlraith from the University of Toronto. Delve into the challenges of real-world reinforcement learning, focusing on goals and preferences expression. Examine Linear Temporal Logic (LTL) as a compelling method for expressing temporal properties of traces. Discover the concept of reward machines and their application in defining reward functions. Compare various reinforcement learning methods, including Q-Learning, Option-Based Hierarchical RL, and Q-Learning for Reward Machines (QRM). Analyze experimental results from discrete domains, Office World, Minecraft World, and Water World. Investigate techniques for creating reward machines, including construction from formal languages and generation using symbolic planners. Gain insights into reward specification and its application in partially-observable reinforcement learning environments.