Errors and Misuses of Statistics

How a correct calculation still misleads.

12 problems · about 21 minutes · TEKS P.S.4.E, S-IC.B.6

What this lesson teaches

The student identifies Type I and Type II errors and common misuses of statistical reasoning.

A Type I error is a false alarm; a Type II error is a missed effect.
Reducing one increases the other, so context decides the balance.
Truncated axes, multiple testing, and confusing significance with importance are the usual misuses.

Straightforward practice. Get the method working first.

5 problems

Convicting an innocent person corresponds to which error? 1 Type I, 2 Type II.
Answer 1
Why Type I.
Why does testing many hypotheses cause trouble?
Answer Some will look significant by chance alone.
Why Chance alone produces false alarms.
A smoke alarm sounds with no fire. Which error? 1 Type I, 2 Type II.
Answer 1
Why A false alarm.
A smoke alarm stays silent during a fire. Which error? 1 or 2.
Answer 2
Why A miss.
Testing 20 ideas at the 0.05 level, how many look significant by chance on average?
Answer 1
Why 20 × 0.05.

The same ideas with more to keep track of.

3 problems

Does a bar chart axis starting above zero exaggerate differences? 1 yes, 0 no.
Answer 1
Why It compresses the visible range.
Does reducing Type I errors increase Type II errors? 1 yes, 0 no.
Answer 1
Why They trade off.
Can a trivial difference be statistically significant with a huge sample? 1 yes, 0 no.
Answer 1
Why Significance is not importance.

Mixed problems. Work out what kind of question it is before you start.

4 problems

Which are genuine statistical warning signs?
Answer A vertical axis that does not start at zero; One striking result from many tests; A causal claim from an observational study
Why One of these is good practice, not a warning sign.
Testing 100 ideas at the 0.05 level, how many false alarms on average?
Answer 5
Why 100 × 0.05.
The False Alarm: Rejecting a true null hypothesis is which error? 1 Type I, 2 Type II.
Answer 1
Why Type I.
The Miss: Failing to detect a real effect is which error? 1 or 2.
Answer 2
Why Type II.