A hypothesis test asks whether a result is too surprising to blame on chance alone.
Step 1: Let's Learn
Read it, or press Listen and follow the words.
The null hypothesis
The null is the boring claim: no effect, no difference, nothing happening. It is what you assume while testing.
The alternative
The alternative is the claim you are investigating. The test either finds evidence for it or does not.
The p-value
The p-value is the probability of a result at least this extreme, assuming the null is true. Small means surprising.
The threshold
A p-value below 0.05 is conventionally called significant. The number is a convention, not a law of nature.
What it does not say
A large p-value does not prove the null. Failing to find evidence is not the same as showing there is nothing there.
Could chance alone explain this?
A hypothesis test asks whether the observed data would be surprising if nothing were really going on. If it would be, the "nothing" explanation becomes hard to maintain.
The null hypothesis is the sceptical position
It states no effect, no difference, no relationship. The test asks whether the data give enough evidence to abandon it. It is never proved — only rejected or left standing.
The p-value, precisely
The probability of data at least as extreme as observed, assuming the null is true. It is not the probability the null is true, and it is not the probability the result was a fluke.
Significant does not mean important
A statistically significant result may be tiny. With a large enough sample, any trivial difference becomes significant. Whether it matters is a judgement statistics cannot make for you.
Step 2: Try It Yourself
Tap and try it out.
Observed
Expected
Observed has the most. It has 12 more than Expected.
Step 3: Watch an Example
One step at a time.
Watch Diego Test a Coin
A coin gives 62 heads in 100 tosses. Diego suspects it is unfair.
- Step 1
The null hypothesis is that the coin is fair, with p = 0.5.
Step 4: Your Turn
Practice makes it stick.
The Threshold
Problem 1 of 2
p-value 0.03 at the 0.05 level. Is the result significant? 1 yes, 0 no.
The Large p
Problem 2 of 2
p-value 0.40. Does this prove the null hypothesis is true? 1 yes, 0 no.
Test It
1 of 8
p-value 0.01 at the 0.05 level. Significant? 1 yes, 0 no.
2 of 8
p-value 0.20 at the 0.05 level. Significant? 1 yes, 0 no.
3 of 8
Which hypothesis states no effect? 1 null, 2 alternative.
4 of 8
A fair coin, 100 tosses. Expected heads?
5 of 8
The usual significance threshold, as a decimal?
6 of 8
Does a smaller p-value mean more surprising under the null? 1 yes, 0 no.
7 of 8
Put a hypothesis test in order.
- 1Collect data and compute the statistic.
- 2Find the p-value, assuming the null is true.
- 3Compare it to the threshold and conclude in context.
- 4State the null and alternative hypotheses.
8 of 8
p-value 0.049 at the 0.05 level. Significant? 1 yes, 0 no.
Step 5: Quick Check
Show what you know.
Question 1 of 2
p-value 0.002 at the 0.05 level. Significant? 1 yes, 0 no.
Question 2 of 2
What exactly does a p-value measure?
What You Learned
- The null hypothesis says nothing is happening; the alternative is the claim under investigation.
- A p-value is the chance of data this extreme if the null were true.
- Below 0.05 is conventionally significant, and a large p-value proves nothing.