Least Squares and Applications

The best answer when there is no exact one.

12 problems · about 24 minutes · S-ID.B.6, N-VM.C.11

What this lesson teaches

The student explains least squares fitting and names where linear algebra is applied.

Real data gives more equations than unknowns, so Ax = b usually has no exact solution.
Least squares finds the x making Ax as close to b as possible, by projecting onto the column space.
Every line of best fit and every regression in statistics is this computation.

Straightforward practice. Get the method working first.

5 problems

Residuals of 6 and 8. What is the total of their squares?
Answer 100
Why 100.
What does least squares find?
Answer The x making Ax as close to b as possible, when no exact solution exists.
Why The closest possible answer.
A prediction of 20 against an observed 18. What is the residual?
Answer -2
Why 18 − 20.
Residuals of 3 and 4. What is the total of their squares?
Answer 25
Why 9 + 16.
Least squares minimises the total of the squared what? Enter 1 for residuals, 2 for slopes.
Answer 1
Why The gaps between predicted and observed.

The same ideas with more to keep track of.

3 problems

Every residual is 0. Was the system solvable exactly? 1 yes, 0 no.
Answer 1
Why No gap at all.
The closest reachable vector to b is its projection onto what? Enter 1 for the column space, 2 for the null space.
Answer 1
Why The column space is what A can reach.
Residuals of 5 and 12. What is the total of their squares?
Answer 169
Why 25 + 144.

Mixed problems. Work out what kind of question it is before you start.

4 problems

Order the reasoning behind least squares.
Answer 1. Notice that Ax = b has no exact solution 2. Ask instead for the closest reachable Ax 3. Project b onto the column space of A 4. Solve for the x that produces that projection
Why The impossibility comes first.
Is a line of best fit a least squares solution? 1 yes, 0 no.
Answer 1
Why It is the standard example.
The Overdetermined System: 40 equations in 3 unknowns. Is an exact solution likely? 1 yes, 0 no.
Answer 0
Why No.
The Residual: A prediction of 12 against an observed value of 15. What is the residual?
Answer 3
Why 3.