A Type I error rejects a true null, a false alarm. A Type II error fails to reject a false null, a missed effect.
Step 1: Let's Learn
Read it, or press Listen and follow the words.
Alpha is the Type I rate
The significance level is exactly the probability of a Type I error when the null is true.
Power
Power is the probability of correctly rejecting a false null. It is 1 minus the Type II error rate.
What raises power
A larger sample, a larger real effect, a larger alpha, or less variability all raise power.
The trade-off
Lowering alpha reduces false alarms and lowers power. Only a larger sample improves both at once.
Which error is worse depends
For a smoke alarm a missed fire is far worse than a false alarm. For a criminal conviction the balance reverses.
Two ways a test can be wrong
A Type I error rejects a true null; a Type II error fails to reject a false one. Both are possible in every test, and reducing one generally increases the other.
Alpha is the Type I error rate
Choosing a significance level of 0.05 accepts a 5% chance of a false positive when the null is true. Lowering alpha reduces false positives and increases the chance of missing a real effect.
Power is the chance of detecting a real effect
Power is 1 minus the Type II error rate. It increases with sample size, with a larger true effect, and with a higher alpha. Sample size is the lever an experimenter actually controls.
Which error is worse depends on the situation
A false positive on a medical screening causes anxiety and further tests; a false negative may cost a life. Choosing alpha is a judgement about consequences, not a statistical decision.
Step 2: Try It Yourself
Tap and try it out.
Observed
Expected
Observed has the most. It has 12 more than Expected.
Step 3: Watch an Example
One step at a time.
Watch Diego Weigh the Two Errors
A drug trial tests whether a new treatment works, with the null saying it does not.
- Step 1
A Type I error approves a treatment that does not actually work.
Step 4: Your Turn
Practice makes it stick.
The False Alarm
Problem 1 of 2
Rejecting a true null hypothesis. Which error? 1 Type I, 2 Type II.
The Power
Problem 2 of 2
The Type II error rate is 0.2. What is the power?
Errors and Detection
1 of 8
Failing to reject a false null. Which error? 1 Type I, 2 Type II.
2 of 8
Alpha is 0.05. What is the Type I error rate?
3 of 8
Type II rate 0.3. What is the power?
4 of 8
Does a larger sample raise power? 1 yes, 0 no.
5 of 8
Does lowering alpha raise power? 1 yes, 0 no.
6 of 8
Does a larger true effect raise power? 1 yes, 0 no.
7 of 8
Which changes increase power?
8 of 8
Power 0.85. What is the Type II error rate?
Step 5: Quick Check
Show what you know.
Question 1 of 2
The Type II error rate is 0.25. What is the power?
Question 2 of 2
Which change improves both error rates at once?
What You Learned
- A Type I error is a false alarm; a Type II error is a missed effect.
- Alpha is the Type I error rate, and power is 1 minus the Type II rate.
- Larger samples, larger effects and larger alpha all raise power.