Errors and Misuses of Statistics

How a correct calculation still misleads.

12 problems · about 21 minutes · TEKS P.S.4.E, S-IC.B.6

NameDate

Figure — use these to answer the problems

These bars differ by very little. On an axis starting near their tops, that gap would look enormous.

Brand A48

Brand B50

Brand A48

Brand B50

Figure 1

Straightforward practice. Get the method working first.

5 problems

Convicting an innocent person corresponds to which error? 1 Type I, 2 Type II.
Answer
2
Why does testing many hypotheses cause trouble?
- Some will look significant by chance alone.
- It takes too long.
A smoke alarm sounds with no fire. Which error? 1 Type I, 2 Type II.
Answer
A smoke alarm stays silent during a fire. Which error? 1 or 2.
Answer
Testing 20 ideas at the 0.05 level, how many look significant by chance on average?
Answer

The same ideas with more to keep track of.

3 problems

Does a bar chart axis starting above zero exaggerate differences? 1 yes, 0 no.
Answer
Does reducing Type I errors increase Type II errors? 1 yes, 0 no.
Answer
Can a trivial difference be statistically significant with a huge sample? 1 yes, 0 no.
Answer

Mixed problems. Work out what kind of question it is before you start.

4 problems

9
Which are genuine statistical warning signs?
- A vertical axis that does not start at zero
- One striking result from many tests
- A causal claim from an observational study
- A randomised controlled experiment
- Tick every box that applies.
Testing 100 ideas at the 0.05 level, how many false alarms on average?
Answer
The False Alarm
Rejecting a true null hypothesis is which error? 1 Type I, 2 Type II.
Answer
The Miss
Failing to detect a real effect is which error? 1 or 2.
Answer