Explore a comprehensive tutorial on optimization techniques for data analysis and machine learning. Delve into kernel learning, regression, graph analysis, neural networks, and low-rank matrix analysis problems formulated as optimization challenges. Understand the role of regularization in promoting useful solution structures. Learn about primary algorithmic techniques, with a focus on gradient and stochastic gradient methods. Divided into two parts with a Q&A session, this 1-hour 51-minute presentation by Stephen Wright from the University of Wisconsin-Madison covers essential concepts for researchers and practitioners in the field of data science and applied mathematics.