Missing Data

Definition

Missing data are values that should exist but were not recorded. How they went missing matters: if the gaps are unrelated to the values, little is lost, but if missingness depends on the value itself, simply ignoring those rows biases the result.

In plain terms

Statisticians distinguish three cases. Missing completely at random: a sensor dropped a reading by chance. Missing at random: the gap depends on something you observed (younger respondents skip a question more often). Missing not at random: the gap depends on the missing value itself (people with the highest incomes leave the income field blank). Only the first is safe to ignore.

In data work there is a second trap: a missing value must be kept distinct from zero, from “not applicable” and from “the parser failed”. Treating an empty price as 0 makes averages wrong and makes “no data” look like “free”.

Example

Five items, three with a price:

Treating “not observed” as zero cuts the average by 40% without a single price having changed.

Common misunderstandings

• “Fill the gaps with the average and move on.” That shrinks the variation in the data and can hide the very pattern that caused the gaps.
• “Empty and zero are interchangeable.” Zero is a measurement; empty is the absence of one.
• “Data are missing at random by default.” That has to be argued from how the data were collected, not assumed.