CSTA Logo

Regression by Hand

Scatter plot with several functions fit to the data.

Jedediyah Williams, PhD
jedediyah@gmail.com

How high does a bouncy ball bounce?

How high does a bouncy ball bounce?

↓ Translate the question into a data problem
When we drop a bouncy ball from rest at height \(h_{drop}\), what height \(h_{bounce}\) will it reach on its first bounce?

How high does a bouncy ball bounce?

\(h_{drop}\) (cm)
\(h_{bounce}\)(cm)
20
15
30
21
40
23
50
35
70
40
100
65
Four students in a classroom. One is holding a meter stick against the wall while another drops a bouncy ball in front of the meter stick. One student is watching while another takes a slow motion video of the ball.

How high does a bouncy ball bounce?

\(h_{drop}\) (cm)
\(h_{bounce}\)(cm)
20
15
30
21
40
23
50
35
70
40
100
65
Scatter plot of Bounce Height vs Drop Height. Data is highly linear.

How high does a bouncy ball bounce?

True relationship
\(Y = ? \)

Approximation
\(\hat{Y} = \hat{\beta}_{0} + \hat{\beta}_{1}X \)
Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data.

How high does a bouncy ball bounce?

\[ \hat{Y} = \hat{\beta} X \]

\(\hat{\beta}\) is a model parameter: a tuneable value in our model
Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data.

How high does a bouncy ball bounce?

The residual error between the observed data \((x_i, y_i)\) and the model is:
\[ e_{i} = y_i - \hat{y}_i \]
where \[ \hat{y}_i = \hat{\beta} x_{i} \]
Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data. Vertical bars depict the error between data and the linear model.

How high does a bouncy ball bounce?

The residual error between the observed data \((x_i, y_i)\) and the model is:
\[ e_{i} = y_i - \hat{y}_i \]
The sum of squared error is the sum of squares $e_i^2$
\[ SSE = \sum_{i=1}^{n} e_{i}^{2}\]
Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data. Vertical bars depict the error between data and the linear model.

 

Model Interpretation

What does the parameter in our model mean?
Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data. Vertical bars depict the error between data and the linear model.
\( \hat{h}_{bounce} = \hat{\beta} h_{drop} \)

Model Interpretation

Scatter plot of Bounce Height vs Drop Height. Data is highly linear. A linear model is fit to the data. Vertical bars depict the error between data and the linear model.
\( \hat{h}_{bounce} = \hat{\beta} h_{drop} \)
\(\hat{\beta}\) is the slope, \(\frac{h_{bounce}}{h_{drop}}\)
This is the ratio of bounce height to drop height
and means we predict \(h_{bounce}\) to be 64% of \(h_{drop}\)

Model Validation

How can we verify that our model works,
or that it is useful?

Model Validation


Data Splitting
Flow diagram depicts Data splitting into Training Set and Testing Set.


Model Validation


Training Data Testing Data

Model Validation

Two students dropping a bouncy ball down a three story stairwell.

Where we go from here

Table of final exam grades with several columns.
Multiple Linear Regression
$\hat{Y}=\hat{\beta}_0+\hat{\beta}_1 X_1 + \hat{\beta}_2 X_2 + \hat{\beta}_3 X_3+\dots$
Multiple scatter plots.
Pendulum Exploration
$\hat{T}=\hat{\beta} \sqrt{L}$
Scatter plot of sinusoidal data. A Curve is fit to the data.
Sinusoidal Model by Parameter-Tuning
$\hat{Y}=\hat{\beta}_0+\hat{\beta}_1 \sin(\hat{\beta}_2 X+\hat{\beta}_3)$
C O 2 data. A model is fit to the data that is quadratically increasing.
Model Regression in Python
$\hat{Y}=\hat{\beta}_0+\hat{\beta}_1 X+\hat{\beta}_2 X^2+\hat{\beta}_3 \sin(\hat{\beta}_4 X+\hat{\beta}_5)$

Resources