A significance test asks whether a result is too surprising to blame on chance alone.
Step 1: Let's Learn
Read it, or press Listen and follow the words.
The null hypothesis
The null is the boring claim: no effect, no difference. It is assumed true while testing.
The p-value
It is the probability of a result at least this extreme, assuming the null is true. Small means surprising.
The threshold
A p-value below 0.05 is conventionally called significant. The number is a convention, not a law.
A large p-value proves nothing
Failing to find evidence is not evidence of absence. You fail to reject rather than accept.
Conclude in context
State the decision and answer the original question, naming the variables. A bare number earns nothing.
Assume nothing is happening, then check
A test assumes the null hypothesis, computes how unusual the data would be under it, and decides whether that is surprising enough to abandon it. The structure is proof by contradiction, softened.
What a p-value is
The probability of data at least as extreme as observed, assuming the null is true. It is not the probability the null is true, and treating it as such is the standard misreading.
Failing to reject is not proof
A large p-value means the data are consistent with the null, not that the null is correct. Absence of evidence is not evidence of absence, and the wording of a conclusion should reflect that.
Significant does not mean important
A large enough sample makes any trivial difference statistically significant. Whether the difference matters is a judgement about the situation, which no test can make.
Step 2: Try It Yourself
Tap and try it out.
Observed
Expected
Observed has the most. It has 12 more than Expected.
Step 3: Watch an Example
One step at a time.
Watch Sana Test a Coin
A coin gives 62 heads in 100 tosses, and Sana tests whether it is fair.
- Step 1
The null is that the coin is fair, so heads has probability 0.5.
Step 4: Your Turn
Practice makes it stick.
The Decision
Problem 1 of 2
p-value 0.02 at the 0.05 level. Is the result significant? 1 yes, 0 no.
The Non-Result
Problem 2 of 2
p-value 0.40. Does this prove the null is true? 1 yes, 0 no.
Test and Conclude
1 of 8
p-value 0.01 at the 0.05 level. Significant? 1 yes, 0 no.
2 of 8
p-value 0.20 at the 0.05 level. Significant? 1 yes, 0 no.
3 of 8
Which hypothesis states no effect? 1 null, 2 alternative.
4 of 8
A fair coin, 100 tosses. Expected heads?
5 of 8
The usual significance threshold, as a decimal?
6 of 8
Does a smaller p-value mean more surprising under the null? 1 yes, 0 no.
7 of 8
Put a significance test in order.
- 1Collect data and compute the statistic.
- 2Find the p-value, assuming the null is true.
- 3Compare to the threshold and conclude in context.
- 4State the null and alternative hypotheses.
8 of 8
p-value 0.049 at the 0.05 level. Significant? 1 yes, 0 no.
Step 5: Quick Check
Show what you know.
Question 1 of 2
p-value 0.30 at the 0.05 level. Significant? 1 yes, 0 no.
Question 2 of 2
What exactly does a p-value measure?
What You Learned
- The null says nothing is happening and is assumed while testing.
- A p-value is the chance of data this extreme if the null were true.
- Below 0.05 is conventionally significant, and a large p-value proves nothing.