Understand · explore · practise
Distinguish association from causal effects, investigate confounding and reverse direction, and explain how random allocation and representative sampling answer different questions.
Before you startScatter diagrams, correlation and sample proportions.
01 / Separate pattern from explanation
Correlation describes association. Causation concerns what would change under an intervention.
A scatterplot alone does not establish that changing x would change y.
A plausible story can help form a hypothesis, but it is not a substitute for evidence. Ask how the data were collected, which alternative explanations remain and what claim the study can actually support.
Each scenario below is fictional. A possible explanation is a hypothesis to investigate, not a finding established by the diagram.
Volunteers attending an optional workshop have higher later test scores than non-attenders.
The association alone does not establish a workshop effect. Prior attainment or motivation may differ between the self-selected groups.
02 / Look for a common cause
Pause, replay or seek freely. The notes explain the same idea and stay in view.
Staff attendance might raise both meeting count and electricity use
It is a candidate common cause.
The observed association does not isolate a meeting effect
Other explanations can fit the same pattern.
Measure relevant context and improve the study design
Naming a possible confounder does not prove it is the explanation.
Workshop volunteers score more highly than non-attenders. Name a plausible confounding factor.
The groups chose whether to attend.
Prior attainment or motivation may influence both attendance and later scores. These possibilities need evidence; they are not established facts about the groups.
More staff attend an office on days with more meetings and higher power use. Does this prove meetings have no effect on power use?
Several mechanisms can coexist.
No. Attendance is a plausible common cause, but meetings may also have an effect. The observational association alone does not separate the contributions.
03 / Consider reverse direction
A gardener records that larger plants receive more water. Does this prove extra water made them larger?
Watering may depend on plant size.
No. The gardener may give larger plants more water; plant type, light and other factors may also matter. An observational association does not determine the direction.
A researcher puts watering on x and plant size on y. Has reverse causation been ruled out?
Axis placement does not establish time order or intervention.
No. The axis choice defines a plotting or prediction direction, not a causal direction.
04 / Check who entered the data
Only people who completed an optional online course appear in a results survey. What limits a claim about all people who start the course?
Non-completers are missing.
Completion and survey response may be associated with outcomes. The observed completers need not represent all starters; report the target group and missingness.
Would collecting many more self-selected volunteers automatically remove this problem?
Size and representativeness answer different questions.
No. A larger biased sample can describe the wrong group more precisely. Improve coverage and selection, not just the count.
05 / Inspect what a total conceals
Within lower level: 50% versus 40%; within higher: 90% versus 80%
Workshop participants have a higher observed success proportion in both groups.
Combined: 38/60 ≈ 63.3% versus 40/60 ≈ 66.7%
The aggregate comparison goes in the other direction.
The groups have different mixtures of starting levels
Neither the aggregate nor these observational subgroup comparisons alone proves a causal effect.
Which workshop group in the example has more lower-level participants: attendees or non-attendees?
Compare 40/60 with 20/60.
Attendees: two thirds versus one third. Different starting-level composition helps explain the reversed overall association.
If one measured starting-level variable is accounted for, is every possible confounder eliminated?
Other measured or unmeasured differences may remain.
No. Adjustment can address specified factors under assumptions; it does not automatically establish a causal effect.
06 / Distinguish allocation from sampling
Random allocation assigns study participants to conditions. Random sampling selects participants from a population.
Neither phrase should be substituted for the other.
Why can random allocation strengthen a comparison of two feedback methods?
It reduces systematic pre-existing differences in expectation.
It makes treatment assignment independent of pre-existing factors by design, supporting a causal comparison when implementation and measurement are sound. Chance imbalance, attrition and other limitations still need consideration.
One school randomly allocates consenting pupils to feedback methods. Does this alone establish the effect in every school?
Allocation does not make the school a representative sample.
No. The experiment may support a causal comparison within its studied setting, while generalisation requires evidence about other pupils, schools and conditions.
07 / Match the claim to the evidence
Rewrite “Each extra meeting causes another 3 units of power use” when 3 is only a gradient fitted to observational office data.
Describe a fitted prediction, not an intervention effect.
The fitted model predicts 3 more units of power use per additional meeting in this observed setting. The gradient alone does not establish that adding a meeting causes that change.
A dataset shows no clear linear correlation. Does that prove changing x cannot affect y?
Consider nonlinear patterns, range and other factors.
No. A nonlinear effect, limited range, confounding or noisy measurements could obscure a simple linear association. Neither a positive finding nor its absence should be overclaimed.
08 / Ask what alternatives remain
Describe the observed pattern, consider common causes, reverse direction and selection, then assess the study design. Distinguish a prediction from an intervention effect and a causal comparison from generalisation to a wider population.
Section 1 of 8 · Separate pattern from explanation