We can solve linear regression (i.e., estimate the regression coefficients) using just linear algebra. Below is the process of 4 steps to do regression analysis via matrix multiplication.
Step 1: Prepare the matrix
\[ Y = XB \]
We actually can expand the function above to another format below. The function below can give you a more detailed idea of what each component is.
\[ Y= \left[ \begin{array} {} y_{11} \\ y_{12} \\ y_{13} \\ ..\\y_{1n} \end{array} \right] = \left[ \begin{array} {} b_0+b_1 x_{11} + b_2 x_{21} \\ b_0+b_1 x_{12}+b_2 x_{22} \\ b_0+b_1 x_{13}+ b_2 x_{23} \\..\\b_0+b_1 x_{1n} + b_2 x_{2n} \end{array} \right] = \left[ \begin{array} {} 1& x_{11} & x_{21} \\ 1 & x_{12} & x_{22} \\ 1 & x_{13} & x_{23} \\..\\1 & x_{1n} & x_{2n} \end{array} \right] \begin{bmatrix} b_0\\ b_1\\ b_2\end{bmatrix} = X B \]
Step 2: X transpose on both sides of equation
We can multiply X transpose (XT) on both sides of equation and get the following. We do so to create a square matrix, which can calculate its inverse matrix in the next step.
\[ X^TY = X^TXB \]
Step 3: Calculate inverse matrix
Since XT X is a square matrix, we can calculate its inverse matrix and time both sides.
\[ (X^T X)^{-1} X^TY =(X^T X)^{-1} X^T X B\]
Since (XT X)-1XT X is an identity matrix, we can write it as follows.
\[ (X^T X)^{-1} X^TY = B\]
Step 4: Finalize the format
If we shift the left and right sides of the equaltion, it becomes the fomat below. Thus, by using the following function, we can calculate the regression coefficients of the linear model.
\[B =(X^TX)^{-1}X^TY\]
Where,
\[ B = \begin{bmatrix} b_0\\ b_1\\ b_2\end{bmatrix} \]
\[ X= \left[ \begin{array} {} 1& x_{11} & x_{21} \\ 1 & x_{12} & x_{22} \\ 1 & x_{13} & x_{23} \\..\\1 & x_{1n} & x_{2n} \end{array} \right] \]
\[ Y= \left[ \begin{array} {} y_{11} \\ y_{12} \\ y_{13} \\ ..\\y_{1n} \end{array} \right] \]