Dive into the fundamentals of optimization and machine learning in this comprehensive lecture on Stochastic Gradient Descent. Explore five different facets of optimization, including iterative methods, gradient descent, and Newton's method. Gain insights into the cheap gradient principle, fixed points of gradient descent, and the concept of convexity. Examine various examples of convex functions and delve into important theorems and proofs. Learn about subgradients of convex functions and their applications. This in-depth session, part of the Bangalore School on Statistical Physics XIII, provides a solid foundation for understanding the core principles of optimization techniques used in machine learning algorithms.