Brier Score
Also known as: probability score.
The average squared difference between forecast probabilities and what happened. Lower is better; always saying 50% scores 0.25.
Synthetic dataExamples on this page use synthetic data created for illustration.
Not adviceThis is educational material. It is not investment, business or other professional advice.
Definition
The Brier score is the average squared difference between forecast probabilities and what happened (1 if the event occurred, 0 if not). Lower is better: 0 is perfect, and always forecasting 50% scores 0.25. It rewards forecasts that are both accurate and honestly calibrated.
In plain terms
Introduced by Glenn Brier in 1950 for weather forecasts, it is a proper scoring rule: the best long-run score is earned by stating your true belief, with no benefit from exaggerating or hedging. Squaring punishes confident misses heavily: forecasting 95% for something that does not happen costs far more than forecasting 60%.
A score means little by itself. Compare it with a baseline such as always forecasting the base rate; a skill score, one minus the ratio of the two, says how much better than that baseline the forecasts are.
Example
Four forecasts:
| Forecast probability | What happened | Squared error |
|---|---|---|
| 0.9 | Happened (1) | (0.9 − 1)² = 0.01 |
| 0.7 | Did not (0) | (0.7 − 0)² = 0.49 |
| 0.6 | Happened (1) | (0.6 − 1)² = 0.16 |
| 0.2 | Did not (0) | (0.2 − 0)² = 0.04 |
Brier score = (0.01 + 0.49 + 0.16 + 0.04) / 4 = 0.175. The baseline of always saying 50% scores 0.25, so the skill score is 1 − 0.175 / 0.25 = 0.30: 30% better than the baseline.
Common misunderstandings
- “A lower Brier score means better decisions.” It means better probability statements; decisions also depend on costs.
- “Scores from different sets of events are comparable.” Easy events give low scores whoever forecasts them; compare against a baseline on the same events.
- “Percent correct is just as good.” It ignores confidence; the Brier score punishes the confident miss.
References
- Glenn W. Brier. Verification of Forecasts Expressed in Terms of Probability . Monthly Weather Review, 1950. DOI: 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
- Tilmann Gneiting, Adrian E. Raftery. Strictly Proper Scoring Rules, Prediction, and Estimation . Journal of the American Statistical Association, 2007. DOI: 10.1198/016214506000001437