Explore the fundamentals of online learning in Markov Decision Processes (MDPs) through this comprehensive lecture by Ambuj Tewari from the University of Michigan. Delve into key concepts such as online learning theory, E-Cube, R-Max, and the general U principle. Gain insights into algorithm design, notation, and MDPs. Understand optimal MDPs, Bellman equations, and Bellman's theorem. Analyze the optimal approach to online learning in MDPs. This talk, part of the Theory of Reinforcement Learning Boot Camp at the Simons Institute, provides a thorough introduction to the subject and addresses important questions in the field.