Understand · explore · practise
Write justified contextual conclusions, distinguish statistical evidence from proof or causation and evaluate sampling and repeated-testing limitations.
Before you startComplete one-tailed and two-tailed tests.
01 / A decision needs an evidence statement
Reject: sufficient evidence for the alternative under the model.
Do not reject: insufficient evidence for that alternative at the stated level. Neither decision proves a hypothesis.
Name the population probability in context. Include the test level and any important limitations. A bare “reject” leaves the reader without the meaning of the result.
02 / Insufficient evidence does not establish equality
A test of H₀:p=0.7 against p<0.7 does not reject at 5%. Repair “the success rate is definitely still 70%”.
State what the data do not establish.
There is insufficient evidence at 5%, under the stated model, that the population success probability has fallen below 0.7.
Why is “accept H₀” potentially misleading?
A failure to show a difference is not a proof.
It can suggest the null has been established. “Do not reject H₀” makes the limited evidential conclusion clearer.
A non-significant sample can occur both when H₀ is true and when it is false. This chapter does not require calculating the probability of detecting every possible alternative.
03 / Rejection is not certainty or a measured effect
A sample proportion is 0.82 and H₀:p=0.7 is rejected against p>0.7. Has p=0.82 been proved?
Separate estimate from evidence against the null.
No. The test supports an increase from 0.7 under its assumptions. The sample proportion estimates p but does not fix its exact population value.
Does statistical significance alone establish that an improvement is large enough to matter?
What information is missing?
No. Consider the size of the estimated change, uncertainty and practical context. A test decision alone does not measure importance.
Pause, replay or seek freely. The notes explain the same idea and stay in view.
04 / Keep the conditional probability in the correct direction
Repair “a p-value of 0.03 means a 3% chance H₀ is true”.
Identify what is assumed in the calculation.
The evidence probability is calculated assuming H₀ and the model. It is not a probability assigned to H₀ being true.
An actual significance level is 0.04. Does this mean every rejection has a 4% chance of being wrong?
The conditioning is important.
No. It is the probability of rejection when H₀ is true for that test and model. It does not directly give the probability H₀ is true after a rejection.
05 / Define success before stating the direction
H₀:p=0.08 is equivalent to H₀:q=0.92
The same baseline is expressed using complementary events.
H₁:p<0.08 is equivalent to H₁:q>0.92
Fewer faults means more working items.
For 30 items, Y=30−X
X≤1 faults is the same event as Y≥29 working items.
Translate p>0.08 into a claim about q.
Subtract from one and reverse the inequality.
q<0.92: an increased fault probability corresponds to a decreased working probability.
Why must P(X≤1) equal P(Y≥29) when Y=30−X?
The two descriptions select identical outcomes.
Every sample with at most one fault has at least 29 working items, and conversely. They are the same event.
06 / A precise answer can rest on weak assumptions
A school wants a conclusion about all pupils but samples only volunteers from the maths club. What limitation should be stated?
Who had a chance to enter the sample?
The volunteers may not represent the whole school. A correct calculation does not remove selection bias.
Why might 30 consecutive rainy-day indicators be poorly modelled as independent trials?
Weather persists across days.
Successive conditions may be related, so the binomial variation need not fit. Seasonal changes can also undermine a common probability.
A new teaching method is followed by a significant rise in success probability. Does the test alone prove the method caused it?
Other differences may coincide with the change.
No. Statistical evidence of a difference is not causal proof. The study design and possible confounding factors matter.
07 / Several opportunities to reject increase false-alarm risk
Probability none rejects = 0.96³ = 0.884736
Multiply the three independent nonrejection probabilities.
Probability at least one rejects = 1−0.884736 = 0.115264
This is 11.5264%, not 4% for the whole set.
Probability all three reject = 0.04³ = 0.000064
This answers a different question.
Can the same products be assumed if all tests reuse strongly overlapping data?
Independence has not been established.
No. Shared data can make the decisions dependent; the products are not justified without independence or another suitable model.
Why is repeatedly trying tests and reporting only a significant one misleading?
How many opportunities were there for an extreme result?
It hides the repeated opportunities for rejection and can exaggerate the strength of evidence. The planned analysis and all relevant tests should be transparent.
What should accompany a contextual conclusion?
Think of the model, rule and scope.
The defined parameter and hypotheses, model assumptions, test level and method, relevant probability or critical-region membership, and important sampling limitations.
08 / Say what the evidence supports
State sufficient or insufficient evidence for the original alternative at the chosen level. Avoid claims of certainty, exact parameter values or causation. Define complementary events consistently and mention material model and sampling limitations. Treat repeated testing explicitly.
Section 1 of 8 · A decision needs an evidence statement