Lab 02
Linear regression
January 23, 2026
Welcome
Goals
- Review derivation of \(\hat{\beta}_0\) based on survey responses
- LaTex in this course
- Lab 02: Linear regression
Deriving \(\hat{\beta}_0\)
Suppose we have a simple linear regression model of response \(Y\) and predictor \(X\)
\[Y = \beta_0 + \beta_1X + \epsilon\]
We want to find the estimators \(\hat{\beta}_0\) and \(\hat{\beta}_1\) that minimize the sum of squares \(\sum_{i=1}^n \epsilon_i^2\)
Deriving \(\hat{\beta}_0\)
\[ \sum_{i=1}^n \epsilon_i^2 = [y_i - (\beta_0 + \beta_1 x_i)]^2 = [y_i - \beta_0 - \beta_1 x_i]^2 \]
Take the first derivative with respect to \(\beta_0\)
\[ \frac{\partial}{\partial \beta_0} \sum_{i=1}^n\epsilon_i^2 = -2 \sum\limits_{i=1}^{n}(y_i - \beta_0 - \beta_1 x_i) \]
Deriving \(\hat{\beta}_0\)
\[ \frac{\partial}{\partial \beta_0} \sum_{i=1}^n\epsilon_i^2 = -2 \sum\limits_{i=1}^{n}(y_i - \beta_0 - \beta_1 x_i) \]
Find the \(\hat{\beta}_0\) that satifies
\[ -2 \sum\limits_{i=1}^{n}(y_i - \hat{\beta}_0 - \beta_1 x_i) = 0 \]
Deriving \(\hat{\beta}_0\)
\[ -2 \sum\limits_{i=1}^{n}(y_i - \hat{\beta}_0 - \beta_1 x_i) = 0 \]
\[ \Rightarrow \sum_{i=1}^n y_i - \sum_{i=1}^n\hat{\beta}_0 - \sum_{i=1}^n \beta_1x_i = 0 \]
\[ \Rightarrow \sum_{i=1}^n y_i - \sum_{i=1}^n \beta_1x_i = \sum_{i=1}^n\hat{\beta}_0 \]
Deriving \(\hat{\beta}_0\)
\[ \Rightarrow \sum_{i=1}^n y_i - \sum_{i=1}^n \beta_1x_i = \sum_{i=1}^n\hat{\beta}_0 \]
\[ \Rightarrow \sum_{i=1}^n y_i - \beta_1\sum_{i=1}^n x_i = n\hat{\beta}_0 \]
\[ \Rightarrow \frac{\sum_{i=1}^n y_i}{n} - \beta_1\frac{\sum_{i=1}^n x_i}{n} = \hat{\beta}_0 \]
\[ \bar{y} - \beta_1 \bar{x} = \hat{\beta}_0 \]
Deriving \(\hat{\beta}_0\)
\[ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} \hspace{10mm} (\text{ We can plug in }\hat{\beta}_1 ) \]
We need to confirm we found the minimum, so we use the second derivative
\[ \begin{aligned} \frac{\partial^2}{\partial \beta_0^2} \sum_{i=1}^n \epsilon_i^2 &= \frac{\partial}{\partial \beta_0}\Big(-2 \sum\limits_{i=1}^{n}(y_i - \beta_0 - \beta_1 x_i)\Big) \\[10pt] & = 2n > 0 \end{aligned} \]
Deriving \(\hat{\beta}_0\)
We found the least-squares estimator for \(\beta_0\)
\[ \hat{\beta}_0 = \bar{y} - \beta_1 \bar{x} \]
Repeat the same process to find \(\hat{\beta}_1\)
LaTex in this class
For this class you will need to be able to…
Properly write mathematical symbols, e.g., \(\beta_1\) not B1, \(R^2\) not R2
Write basic regression equations, e.g., \(\hat{y} = \beta_0 + \beta_1x_1 + \beta_2x_2\)
Write matrix equations: \(\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}\)
Write hypotheses (we’ll start this next week), e.g., \(H_0: \beta = 0\)
You are welcome to but not required to write math proofs using LaTex.
Writing LaTex
Inline: Your mathematics will display within the line of text.
Use
$to start and end your LaTex syntax. You can also use the menu: Insert -> LaTex Math -> Inline Math.Example: The text
The linear regression model is $\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}$producesThe linear regression model is \(\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \boldsymbol{\epsilon}\)
Writing LaTex
Display: Your mathematics will display outside the line of text
Use a
$$to start and end your LaTex syntax. You can also use the menu: Insert -> LaTex Math -> Display Math.Example: The text
The estimated regression equation is $$\hat{\mathbf{y}} = \mathbf{X}\hat{\boldsymbol{\beta}}$$producesThe estimated regression equation is
\[ \hat{\mathbf{y}} = \mathbf{X}\hat{\boldsymbol{\beta}} \]
Tip
Click here for a quick reference of LaTex code.
Lab 02: Linear regression
Today’s lab focuses on using simple and multiple linear regression to understand variability in coffee quality ratings.
