Statistics & Probability

Bias that arises when units drop out of a study for reasons related to what is measured, so the units that remain no longer represent the original group.

How common something is in the whole relevant population, before looking at any specific evidence.

A simple method, such as repeating the last value, against which a new method is compared. A result means something only relative to it.

Estimating how much a statistic would vary from sample to sample by resampling the observed data, with replacement, many times.

The average squared difference between forecast probabilities and what happened. Lower is better; always saying 50% scores 0.25.

Events forecast at p% happen about p% of the time. Calibration is separate from accuracy: an overconfident forecaster is not calibrated.

How much two raters agree beyond what chance alone would produce: 1 is perfect, 0 is no better than chance.

Grouping items or people by a shared starting point, such as the month they first appeared, and following each group over time.

A change over time in the relationship between inputs and the outcome, so that a model or rule that once worked slowly stops working.

A range calculated from a sample that expresses how precisely a quantity has been estimated. A 95% interval comes from a procedure that captures the true value in 95% of repetitions.

A third factor that influences both the supposed cause and the outcome, creating or hiding an association between them.

Correlation means two quantities move together; causation means changing one would change the other. Data alone cannot tell which, or whether a third factor drives both.

How closely two vectors point in the same direction, as the cosine of the angle between them. It ignores length.

Estimating how well a method will do on new data by repeatedly leaving part of the data out, training on the rest, and rotating.

Information that would not exist at prediction time finds its way into the data used to build or evaluate a model, so results look better than they can ever be in use.

How big a difference or relationship is in practical terms, as opposed to whether it can be detected statistically.

Floating-point numbers are fast but cannot hold many decimal fractions exactly; money and other exact quantities are better stored as whole numbers of a small unit.

A measure of market concentration: the sum of the squared market shares, in percent, of all sellers. It rises sharply when a few sellers dominate.

Data set aside before any building begins and used once, at the end, to estimate performance on unseen data.

The many niche items that each sell little but together make up a large share of sales, typical of markets where shelf space costs almost nothing.

The mean is the sum divided by the count; the median is the middle value. Extreme values pull the mean but not the median.

Values that should exist but were not recorded. Why they are missing decides whether ignoring them is harmless or biases the result.

Replacing each point of a series with the average of the points around it, to smooth short-term noise. A longer window smooths more but reacts later.

The more tests, rules or variants you try on the same data, the more likely some look significant by chance. The number of tries has to be counted.

An observation far from the rest. It may be an error, a rare but real event, or a sign of two kinds of things mixed together; look before deleting.

When a model learns the noise in its training data instead of the real pattern, and so fails on new data.

How surprising the data would be if the null hypothesis were true. It is not the probability that the hypothesis is true.

Observations of the same units, such as products, firms or people, repeated over time, so change can be studied within each unit.

A percentage point is the difference between two percentages; a percent change is that difference relative to the starting value.

The value below which a given percentage of observations fall. The 50th percentile is the median; the 90th marks the value that 90% do not exceed.

The population is everything you want to know about; the sample is the part you observed. Statistics asks what the second says about the first.

A distribution in which a few items are enormous and a long tail of items are tiny; the average is a poor summary of such data.

How strongly the quantity bought responds to a price change: the percentage change in quantity divided by the percentage change in price.

A rating is a score a customer gives; a review is the written text. The average score reflects the people who chose to rate, not all buyers.

Extreme measurements tend to be followed by less extreme ones, because part of any extreme value is chance. The worst look improved and the best look worse without any cause.

A marketplace's ordering of products within a category by recent sales. It is relative, changes constantly, and is not a count of units sold.

The list or source a sample is actually drawn from. Whatever the frame leaves out cannot appear in the sample, however large the sample is.

A regular pattern that repeats at a fixed period, such as higher sales every December. Without separating it, a normal seasonal rise is mistaken for growth.

The units in your data differ systematically from the group you want to describe, because of how they were chosen or kept. Results hold for the selected group only.

A trend that appears in every subgroup can reverse or vanish when the groups are combined, because group sizes differ and the grouping is linked to both choice and outcome.

How far values typically lie from their average, in the units of the data. Small means tightly clustered; large means spread out.

A selection error from studying only what survived a process and ignoring what dropped out along the way.

Observations of one quantity recorded in time order. Neighbouring values are related and order matters, so time series need methods that respect time.

How many standard deviations a value lies above or below the average of its group. It puts different measures on one scale.