Evidlake
Menu

Pre-registration

Also known as: preregistration, analysis plan, research freeze.

Writing down, before seeing results, the question, the data and exactly how it will be analysed, and keeping that record unchanged.

Updated Last reviewed

Synthetic dataExamples on this page use synthetic data created for illustration.

Not adviceThis is educational material. It is not investment, business or other professional advice.

Definition

Pre-registration means writing down, before seeing the results, what question will be asked, what data will be used and exactly how it will be analysed, and keeping that record unchanged. It separates tests of a plan from exploration done afterwards.

In plain terms

Analysis has many forks: which period, which threshold, which items to exclude, which metric. If these are chosen after seeing results, a pattern can nearly always be found (see Multiple Comparisons Problem). Pre-registration closes the forks in advance: the plan is dated and frozen, and the analysis follows it.

It does not forbid exploration. Exploratory analyses are valuable for finding ideas, but they are labelled as such and treated as hypotheses for a new test. Any deviation from the plan is reported. The same discipline works inside a team: a dated, frozen specification before the data cut is opened.

Example

Frozen on 2026-03-01: the metric is the 30-day relative change in rating count; the signal is a rise above 20%; the sample is items listed before 2026-01-01; the test period is March to May.

AnalysisStatus
The plan as frozenConfirmatory: one test
Afterwards: 14-day window, 15% threshold, only top-ranked items, and three more variantsExploratory: six extra tests, to be reported as such

With seven tests, the chance that at least one passes a 5% significance bar by luck alone is 1 − 0.95⁷ ≈ 30%. A “significant” exploratory variant is a lead, not a finding.

Common misunderstandings

  • “Pre-registration forbids exploring the data.” It requires only that exploration is labelled as exploration.
  • “A pre-registered study is automatically correct.” It protects against one source of error, flexible analysis, not against all of them.
  • “It is only for medicine and psychology.” Any field with many analysis choices benefits.
  • Multiple Comparisons Problem

    The more tests, rules or variants you try on the same data, the more likely some look significant by chance. The number of tries has to be counted.

  • Backtesting

    Evaluating a rule or model by replaying it on historical data as if it had been used at the time. Informative only if the replay is honest.

  • Reproducibility

    A result is reproducible when someone else, with the same data and code, gets the same result. It needs the data version, code, settings and environment to be recorded.

References

  1. Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor. The Preregistration Revolution . Proceedings of the National Academy of Sciences, 2018. DOI: 10.1073/pnas.1708274114
  2. Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn. False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant . Psychological Science, 2011. DOI: 10.1177/0956797611417632