Explore the intricacies of online learning in Markov Decision Processes (MDPs) in this comprehensive lecture from the Theory of Reinforcement Learning Boot Camp. Delve into topics such as adversarial scenarios, performance measures, oblivious and non-oblivious adversaries, and the challenges of learning with changing transitions. Examine the formal protocol for online learning in fixed MDPs, temporal dependencies, and regret decomposition. Discover the MDP-Expert algorithm, its guarantees, and applications in bandit feedback scenarios. Investigate online linear optimization, mirror descent, and the Online Relative Entropy Policy Search (O-REPS) algorithm. Compare various guarantees and explore function approximation in MDP-E. Gain insights into O-REPS with uncertain models and consider future directions in this field of study.