Linear Regression

Residuals

A Residual value refers to the vertical distance between each of the data points and the line of best fit. The regression matters because it indicates how well our line of best fit reflects the data.

The formula for Residuals is:

• \(\text{Actual}\; Y\) is the data point on the grap
• \(\text{Predicted}\; Y\) is the point in which the residual line and the regression line intersect each other

The quantity of the regression value tells us how well the line of best fit reflects the data.

• If residuals are small, the model is making accurate predictions
• If residuals are large, the model is not fitting the data well

The sign of the regression value also matters in this context:

• If it's a positive residual, The actual value is higher than the predicted value (the model underestimates)
• If it's a negative residual, The actual value is lower than the predicted value (the model overestimates)
• If the Residual is equal to 0, The prediction is exactly correct for that value

It's important to watch out for outliers in your graph. These are values that vary drastically from the line of best fit. These points might negatively influence the line of best fit. In order to prevent this, remove the outlier and recalculate the regression line.

Least-Squares Fit

The least-squares fit is a mathematical method used to find the best-fit line for a set of data points. Ideally, we want the sum of all residuals to be zero and the sum of the squares of the residuals should be as small as possible.

The sum of squared residuals \((SSR\)) can be be calculated as such:

• \(y_{\text{actual}}\) is the observed value from the dataset.
• \(y_{\text{predicted}}\) is the predicted value from the regression line.
• \((y_{\text{actual}} - y_{\text{predicted}})\) is the residual for each data point.
• Summation Symbol (\(\sum\)) means we add up all of the squared residuals in the dataset.

Although the algebra looks daunting, the equation of this line can be determined using:

Where \(\bar{x}\) is the mean of \(x\) and \(\bar{y} \) is the mean of \(y\).

Note: You might also see this referenced as \(y = ax + b \). It's the same equation, with \(a\) as the slope.

Interpolating and Extrapolating

Using the line of best fit, You can then use the equation for this line to make predictions by interpolating and extrapolating the data.

Interpolating means predicting values that lie within the domain/range. These values are within the data that we have collected. In simple terms, we are predicting values anywhere in between the first data point and the last data point of our Scatter Plot.

Extrapolating means predicting values that lie outside the domain or range. These values are outside the data that we've collected. Usually when doing extrapolation, we present our predictions with a disclaimer saying “If the model continues this way, then…”. We do this, as we have no trends to aid us in our predictions, thus we don’t know for sure what will happen.