Understand · explore · practise
Distinguish unusual observations from errors and out-of-scope records, document justified changes and recompute summaries correctly. Original scenarios and worked practice.
Before you startOutlier rules, means and summary totals.
01 / Investigate before changing
A valid observation can be unusual. A plausible-looking observation can be wrong. Check evidence about the measurement and study scope instead of using an outlier flag as an automatic deletion command.
Preserve the original record. If a correction or exclusion is justified, record what changed, why, and how the analysed sample changed.
A long journey time is checked against the original record. The journey belongs to the defined study population.
Retain the valid observation. Investigate its context and compare suitable summaries; do not delete it merely because it is unusual.
These are illustrative records, not personal information. Keep an audit trail when analysing real data.
02 / Check the original record
Confirm that the same journey and unit are being compared
Avoid accidentally matching a different record.
Correct 180 to 18 and retain the original in the audit trail
There is direct evidence for the replacement.
Recompute every affected summary
The original arithmetic can be correct for incorrect data.
A value 81 seems high. May you replace it with 18 because that looks like a plausible reversal?
A plausible explanation is not evidence.
No. Check the original record or another reliable source first. If unresolved, label the issue and explain how it is handled rather than inventing a replacement.
A very high but verified rainfall measurement belongs to the study location and date range. Is unusual size alone a reason to exclude it?
The study may need to represent extreme weather too.
No. Retain genuine eligible data unless there is a separately justified analytical rule. Removing it just for being large can bias the results.
03 / Separate units from mistakes
One duration is recorded as 1.5 hours while the analysis uses minutes. What should the comparable numeric value be?
Multiply by 60.
90 minutes. Record the conversion; do not treat 1.5 as minutes or label it an outlier before reconciling units.
Most records are daily rainfall totals, but one is a monthly total. Can you simply divide the monthly value by 30 and call it the missing daily observation?
A monthly average cannot recover a particular day.
No. The variable is different. Find the appropriate daily record or treat the mismatch under a documented rule. Dividing invents information about a specific day.
04 / Use the defined population
The study covers journeys starting on weekdays. A weekend journey was included by mistake. What is a justified action?
Apply the study definition consistently.
Exclude it from this weekday analysis and record the reason. The action is justified by eligibility, whether the duration is ordinary or extreme.
After seeing results, a researcher excludes all long journeys to report a quicker average. What is wrong?
The target population has silently changed.
This selects records based on the desired conclusion. Keep the original scope or explicitly define and justify a different analysis; do not present it as representing all original journeys.
05 / Distinguish missing from zero
Three daily totals are 0, 2, 4 mm and one day is missing. Find the mean over measured days. Is the four-day mean known?
Use the three observed values only for the observed-day mean.
Observed-day mean = 6/3 = 2 mm. The exact four-day mean is unknown. Replacing the missing day by zero would give 1.5 mm but would add an unsupported measurement.
If a missing value is filled using a stated statistical model, may it be described as directly measured?
Modelled and observed values have different origins.
No. Label the imputation and its assumptions, distinguish it from original measurements, and consider its effect on the results.
06 / Recalculate all affected totals
Pause, replay or seek freely. The notes explain the same idea and stay in view.
Before: n = 4, T = 30, U = 380
T is the sum; U is the squared sum.
After: n = 4, T = 20, U = 120
Subtract 18 and 18², then add 8 and 8².
Mean falls from 7.5 to 5; descriptive variance from 38.75 to 5
Both summaries must be recalculated.
If 18 were excluded as ineligible rather than corrected, what would n, T and U become?
Remove the observation and reduce the count.
n = 3, T = 12, U = 56. Mean = 4 and descriptive variance = 8/3. These are different from the corrected four-record results.
After a justified correction, can the old outlier thresholds always be reused?
The quartiles, mean or SD may have changed.
No. Recompute the summaries and thresholds for the revised dataset when the rule is defined from that dataset. Keep a separate fixed reference only if that is explicitly the intended method.
07 / Make the decision inspectable
What should an audit note contain for a corrected transcription error?
Someone else should be able to reconstruct the change.
Identify the record, original and corrected values, units, supporting source, reason and date of the change. Preserve the original rather than silently overwriting it.
A questionable eligible value cannot be verified. How can you show how much the conclusion depends on it?
Compare transparent alternative analyses rather than concealing the issue.
Report the uncertainty and, where appropriate, compare summaries with and without the value, clearly labelling both. This sensitivity analysis does not prove that exclusion is correct.
08 / Keep valid variation
Check source, units and eligibility. Correct only with support, distinguish missing data from zero, retain genuine eligible unusual values, and document justified exclusions. Recompute summaries and make uncertainty visible.
Section 1 of 8 · Investigate before changing