9. Simple Linear Regression: Simple Linear Regression
Finding the Regression Equation
A regression line is the best-fitting straight line through a set of data points. This section introduces a method by which the slope and the intercept of the regression line can directly be calculated.
#\phantom{0}#
Calculation Regression Coefficient and Intercept
Performing a simple linear regression analysis results in a regression equation of the form:
\[\hat{Y}=b_0 + b_1 \cdot X\]
To calculate the slope #b_1# of the regression line, use of the following formula:
\[b_1 =\cfrac{\sum\limits_{i=1}^n{(X_i-\bar{X})(Y_i-\bar{Y})}}{\sum\limits_{i=1}^n{(X_i-\bar{X})^2}}\]
Once the slope is known, it is possible to calculate the intercept #b_0# with the following formula:
\[b_0 = \bar{Y} - b_1 \cdot \bar{X}\]
Consider the following #5# pairs of data points:
\[\begin{array}{|c|c|}
\hline
X_i&Y_i\\
\hline
1&3\\
2&1\\
3&2\\
4&4\\
5&0\\
\hline
\end{array}\]
Find the regression line corresponding to these points.
\[\begin{array}{rcl}
\bar{X}&=&\cfrac{\sum\limits_{i=1}^n{X_i}}{n} = \dfrac{1+2+3+4+5}{5}=\dfrac{15}{5}=3\\\\
\bar{Y}&=&\cfrac{\sum\limits_{i=1}^n{Y_i}}{n} = \dfrac{3+1+2+4+0}{5}=\dfrac{10}{5}=2
\end{array}\]
Next, find the values of #(X_i-\bar{X}), (Y_i-\bar{Y}), (X_i-\bar{X})^2# and #(X_i-\bar{X})(Y_i-\bar{Y})# for each pair of data points:
\[\begin{array}{|c|c|c|c|c|c|}
\hline
X_i&Y_i&X_i-\bar{X}&Y_i-\bar{Y}&(X_i-\bar{X})^2&(X_i-\bar{X})(Y_i-\bar{Y})\\
\hline
1&3&-2&1&4&-2\\
2&1&-1&-1&1&1\\
3&2&0&0&0&0\\
4&4&1&2&1&2\\
5&0&2&-2&4&-4\\
\hline
\end{array}\]
With this information the slope #b_1# and intercept #b_0# can be calculated:
\[\begin{array}{rcl}
b_1 &=& \cfrac{\sum\limits_{i=1}^n{(X_i-\bar{X})(Y_i-\bar{Y})}}{\sum\limits_{i=1}^n{(X_i-\bar{X})^2}} = \cfrac{-2+1+0+2-4}{4+1+0+1+4} = -0.3\\\\
b_0 &=& \bar{Y} - b_1 \cdot \bar{X} = 2 - (-0.3)\cdot3 = 2.9
\end{array}\]
So the regression equation is:
\[\hat{Y}=b_0 + b_1X=2.9 -0.3X\]
Note that the regression line always passes through the point #(\bar{X},\bar{Y})#. In this case, #(\bar{X},\bar{Y})= (3,2)# and entering #X=3# into the equation gives:
\[\hat{Y}=2.9 -0.3\cdot 3 =2\]