Learn

A number does not carry its own credibility. Three questions about where it came from, what was done to it and what depends on it let you judge it.

Jobs are retried, files are re-imported and parsers are re-run. A pipeline must give the same result each time. How to choose keys, tell a repeat from a new observation, and test it.

An empty cell can mean not observed, not applicable, not yet available, a failed parse or a true zero. Treating them alike corrupts averages, totals and joins. How to keep them apart.

A series of daily observations is only comparable if each was taken the same way. How to fix the conditions, record completeness, keep failures visible and avoid comparing across a change.

Automated collection takes something from the source: bandwidth, attention and goodwill. Prefer authorised routes, read the rules, spend a small budget slowly, and stop when told no.

Rare events produce mostly false alarms even from a good screen, and studying only the winners hides what the losers did too. Both are failures to ask: how common is this, and who is missing?

Every fact carries two times: when it happened and when you could have known it. Mixing them up is a quiet reason many historical results look better than they should.

A test that quietly uses information from after the decision date will look better than anything that could have been achieved. Four ways the future leaks in, a simulation of one of them, and how to close them.

Test twenty rules on the same history and the best one will look impressive even if all twenty are noise. How big the effect is, how to count your attempts, and how to correct for them.

Random splits quietly leak the future when data are ordered in time. How to split by time, step forward through history, and purge and embargo the overlap that still leaks.

Two ways to decide that a result fits a query, and a simple method for judging which ranking is better: measure it against people's judgments.

A score that cannot say why is a number to be argued about. How to build a scoring scheme with hard rules, weights, a proper place for unknown values, reasons with every result and a check against human judgments.

John Boyd's observe–orient–decide–act loop rewards speed, but speed only helps if you learn. A short decision record, written at the time, turns each pass through the loop into something you can review fairly.

Saying 70% is a promise: about seven in ten such calls should come true. How to check that promise with a table, a chart and one number, using twenty forecasts.

Products with feature X sell more. Does X cause the sales? Four questions (reverse direction, a third factor, chance and selection) and a design that can answer them.

Rate the source and the claim separately, count independent origins rather than repeats, and place the evidence on a ladder from anecdote to controlled experiment.

Mention volume, called 声量 in Chinese media monitoring, looks like a plain count. Five choices decide its value, and the same two weeks can be +50%, +23% or no change, depending on how you count.

News travels by copy. To count what was reported, rather than how often it was reprinted, group near-identical texts. A walk-through of shingles, similarity scores and the limits of the method.

A list of the 100 best sellers describes the best sellers. It is a selected sample whose members were chosen by the very thing you may want to measure. What it can and cannot tell you.