The null hypothesis says nothing is going on. The alternative says something is.
Step 1: Let's Learn
Read it, or press Listen and follow the words.
The p-value
The probability of data at least this extreme, assuming the null is true.
What it is not
It is not the probability the null is true. That quantity is not what the calculation produces.
Failing to reject
A large p-value means the data is consistent with the null, never that the null is proved.
Alpha is chosen in advance
The significance level is set before seeing the data. Choosing it afterwards to make a result significant is a serious methodological failure, and it is the mechanism behind much unreplicable research.
Failing to reject is not proof
A large p-value means the data are consistent with the null, not that the null is true. Absence of evidence is not evidence of absence, and the exam expects "we do not have convincing evidence" rather than "there is no difference".
Statistical against practical significance
A very large sample can make a trivial difference statistically significant. Whether the difference matters is a separate judgement the statistics cannot make.
Testing many things inflates the error rate
Run twenty independent tests at the 5% level and you expect one false positive by chance alone. Reporting only the significant result out of many is a well-known way to produce a meaningless finding.
Step 2: Try It Yourself
Tap and try it out.
1
Step 3: Watch an Example
One step at a time.
Watch Rosa State a Conclusion
A test gives p = 0.02 at a significance level of 0.05.
- Step 1
0.02 is smaller than 0.05.
Step 4: Your Turn
Practice makes it stick.
The Decision
Problem 1 of 2
p = 0.03 at level 0.05. Reject the null? 1 for yes, 0 for no.
The Other Way
Problem 2 of 2
p = 0.30 at level 0.05. Reject the null? 1 for yes, 0 for no.
Testing
1 of 8
p = 0.01 at level 0.05. Reject? 1 for yes, 0 for no.
2 of 8
p = 0.06 at level 0.05. Reject? 1 for yes, 0 for no.
3 of 8
Does failing to reject prove the null? 1 for yes, 0 for no.
4 of 8
Is the p-value the probability the null is true? 1 for yes, 0 for no.
5 of 8
Order the steps of a significance test.
- 1Check the conditions
- 2Compute the test statistic and p-value
- 3State a conclusion in context
- 4State the hypotheses
6 of 8
A Type I error rejects a true null. At level 0.05, what is its probability?
7 of 8
Lowering the level from 0.05 to 0.01 makes rejecting which? 1 for easier, 2 for harder.
8 of 8
p = 0.001 at level 0.05. Reject? 1 for yes, 0 for no.
Step 5: Quick Check
Show what you know.
Question 1 of 1
p = 0.20 at level 0.05. Reject? 1 for yes, 0 for no.
What You Learned
- The null says nothing is happening; the alternative says something is.
- A p-value is the chance of data this extreme assuming the null is true.
- Failing to reject is never proof of the null.