Articles → MACHINE LEARNING → Linear Regression Algorithm

Linear Regression Algorithm

This article describes the Linear Regression Algorithm.

What Is The Linear Regression Algorithm?

Imagine you are trying to guess someone's weight just by looking at their height. You notice a pattern: -

Taller people tend to weigh more.
Shorter people tend to weigh less.

Now, imagine you take a piece of paper, create a graph, and plot many people’s height and weight as dots on that paper. Take a ruler, and draw a straight line that passes through the middle of these dots. That line represents the best-guess relationship between height and weight.

Formula For Calculating A Straight Line

y = mx + c

Formula Part	Description
y	Average of target (or output variable)
x	Average of the Input variable
m	The slope (how much y changes for each x). The slope is also known as the Coefficient.
c	The intercept (where the line crosses the y-axis), i.e., what is the value of Y when X is zero?

Example

Consider the following data: -

Height	Weight
150 cm	50 kg
160 cm	56 kg
170 cm	63 kg

To begin with, calculate the average of height and weight.

avg_x = average of x values (heights)
avg_y = average of y values (weights)

avg_x = (150 + 160 + 170) / 3 = 160  
avg_y = (50 + 56 + 63) / 3 = 56.33

The next step is to calculate the slope.

m = Σ[(xi - avg_x)(yi - avg_y)] / Σ[(xi - avg_x)²]

x	y	xi - avg_x	yi - avg_y	(xi - avg_x)(yi - avg_y)	(xi - avg_x)²
150	50	-10	-6.33	63.3	100
160	56	0	-0.33	0	0
170	63	10	6.67	66.7	100

Σ[(x - avg_x)(y - avg_y)] = 63.3 + 0 + 66.7 = 130  
Σ[(x - avg_x)²] = 100 + 0 + 100 = 200

So, m is 130 / 200 = 0.65.

Finally, calculate the intercept.

c = avg_y − m × avg_x 
c = 56.33 − 0.65 × 160  
c = 56.33 − 104 = -47.67

Error

To calculate the error, we will use the following formula.

Error = Y (Actual) – Y (Predicted)

With the previous example, the formula for calculating weight (Y) is.

y = (0.65)X +(-47.67) // mx + c

Height	Weight (Actual)	Weight (Predicted) – Calculation	Weight (Predicted)	Error
150	50	0.65×150−47.67	49.83	0.17
160	56	0.65×160−47.67	56.33	-0.33
170	63	0.65×170−47.67	62.83	0.17

Mean Squared Error (MSE)

MSE is the average of the squared differences between the actual values and the predicted values.

The formula is: -

Picture showing the formula of Mean Squared Error

If you take the above example, then the value would be: -

Picture showing the calculation of Mean Squared Error

Lower MSE = better fit.

Root Mean Squared Error (RMSE)

MSE (Mean Squared Error) is the average of the squared prediction errors.

Posted By -	Karan Gupta

Posted On -	Tuesday, May 20, 2025

Query/Feedback

Your Email Id		**

Subject		*

Query/Feedback	Characters remaining 250	**