BM25
Also known as: Okapi BM25, best matching 25.
A ranking function that scores documents for a query from the query words they contain, with a cap on the credit for repetition and an adjustment for length.
Synthetic dataExamples on this page use synthetic data created for illustration.
Definition
BM25 is a ranking function that scores a document for a query from the query words it contains. Like TF-IDF it favours rare words, but it caps the credit for repeating a word and adjusts for document length, which usually ranks better in practice.
In plain terms
Three ideas are combined. Rare words weigh more (the idf part). Repeating a word helps, but with diminishing returns: the tenth occurrence adds much less than the second (saturation, controlled by a parameter usually called k1). And a long document is expected to contain more words by chance, so its score is reduced somewhat (length normalisation, controlled by b).
BM25 has been a strong baseline in search for decades and is still used, often combined with Text Embedding methods. Its limits are the limits of word matching: it cannot connect “car” with “automobile” unless both words appear.
Example
The credit a document gets from one query word, as a function of how many times it occurs (term frequency), with k1 = 1.2 and no length adjustment (the formula tf × (k1 + 1) ÷ (tf + k1)):
| Occurrences | Credit |
|---|---|
| 1 | 1.00 |
| 2 | 1.38 |
| 3 | 1.57 |
| 5 | 1.77 |
| 10 | 1.96 |
| 20 | 2.08 |
Twenty occurrences earn only about twice the credit of one, so stuffing a page with a keyword stops paying off quickly. With a plain count it would earn twenty times as much.
Common misunderstandings
- “BM25 understands what the query means.” It only matches words; a different word for the same thing scores nothing.
- “Its parameters are fixed constants.” The usual values (k1 around 1.2, b around 0.75) are starting points to be tuned on your own judged queries.
- “Embeddings have made it obsolete.” It remains a strong, explainable baseline, and many systems use both.
References
- Stephen Robertson, Hugo Zaragoza. The Probabilistic Relevance Framework: BM25 and Beyond . Foundations and Trends in Information Retrieval, 2009. DOI: 10.1561/1500000019
- Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze. Introduction to Information Retrieval . Cambridge University Press, 2008. ISBN 978-0521865715.