Play all

Awesome song and introduction

Step 0: The data and the loss function

Step 1: Initialize the model with a constant value

Step 2: Build M trees

Step 2.A: Calculate residuals

Step 2.B: Fit a regression tree to the residuals

Step 2.C: Optimize leaf output values

Step 2.D: Update predictions with the new tree

Step 2: Summary of step 2

Step 3: Output the final prediction

The sum on the left hand side should be in parentheses to make it clear that the entire sum is multiplied by 1/2, not just the first term.

. It should be R_jm, not R_ij.

, the leaf in the script is R_1,2 and it should be R_2,1.

. With regression trees, the sample will only go to a single leaf, and this summation simply isolates the one output value of interest from all of the others. However, when I first made this video I …

, the header for the residual column should be r_i,2.

Description:

Dive into the second part of a four-part video series on Gradient Boost, focusing on regression details. Learn how this popular machine learning algorithm predicts continuous values like weight. Explore the original Gradient Boost algorithm step-by-step, including data and loss function initialization, model initialization with a constant value, and the process of building multiple trees. Understand how to calculate residuals, fit regression trees to residuals, optimize leaf output values, and update predictions with new trees. Gain insights into the final prediction output and benefit from detailed explanations of each step in the algorithm. Perfect for those who have watched Part 1 and are familiar with Regression Trees and Gradient Descent concepts.

Gradient Boost Part 2 - Regression Details

StatQuest with Josh Starmer

Add to list

#Computer Science #Machine Learning #Gradient Descent #Predictive Modeling