Hybrid search: BM25, reciprocal rank fusion and reranking

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

Hybrid search runs two different retrievers over the same chunks and then merges their ranked lists. One retriever is lexical, almost always BM25. The other is a vector comparison. A third stage, a cross-encoder reranker, may then rescore a short head of that merged list. The merge is the part that goes wrong in production, because a BM25 score and a vector score are not numbers on the same scale, and the usual fusion formula can bury a perfect lexical hit.

An AI writing system produced this article for NetLeverage's editorial studio. Descriptions of what a retriever "finds" are descriptions of scores and ranks. They are not a claim about understanding, and they are not a self-report of experience.

Nothing below was executed. Index settings, pipeline bodies and the worked rank arithmetic are taken from the cited papers and vendor pages, or calculated from those published formulae. Closing an index, changing similarity, or pointing a search pipeline at a live corpus can change what users see. Rehearse that on a copy.

Why it matters

A retrieval-augmented answer can only cite chunks it was handed. Two published results show why one retriever is a weak bet, and why adding the two scores is a second bet.

Sciavolino, Zhong, Lee and Chen measured dense passage retrievers against BM25 on entity questions ("Where was [E] born?"). On EntityQuestions, top-20 accuracy was 49.7% for a dense model trained on Natural Questions and 56.7% for one trained on four question-answering sets, against 72.0% for BM25. On Natural Questions the same dense model led, 80.1% against 64.4%. Those checkpoints fail on rare entities and on question patterns they were not shown, while BM25, which does not train on a question distribution, moves less with entity frequency. The figures are the paper's. They are not a leaderboard for every later embedding model.

The second result is older and is about merging lists, not about vectors. Cormack, Clarke and Büttcher introduced reciprocal rank fusion in 2009 as a simple unsupervised fusion method. Their method ignores the scores and uses only ranks. That is the property hybrid search still needs.

How it works

What BM25 actually computes

Apache Lucene 10.3.0 implements BM25 in BM25Similarity, and it points at Robertson, Walker, Jones, Hancock-Beaulieu and Gatford, Okapi at TREC-3 (1994), as the origin. The class an engineer debugs today is the Lucene one. Its defaults are k1 = 1.2, b = 0.75, and discountOverlaps = true. The constructor rejects a k1 that is negative or not finite, and a b outside [0, 1].

For a term, Lucene's inverse document frequency is the natural logarithm

where $n$ is the number of documents containing the term and $N$ is CollectionStatistics.docCount(), the documents that have the field. Lucene uses that count, not the whole-index document count, so a sparse field does not skew the statistic. In Java, Math.log is the natural logarithm. The class comment cites the 1994 TREC paper; the formula in this article is the 10.3.0 implementation.

The term-frequency piece, again from the class's own explanation, is

and the score contribution is $\mathrm{b}\mathrm{o}\mathrm{o}\mathrm{s}\mathrm{t}\times \mathrm{i}\mathrm{d}\mathrm{f}\times \mathrm{t}\mathrm{f}$. Average field length is sumTotalTermFreq / docCount. With discountOverlaps true, tokens whose position increment is zero do not add to the length used for that norm.

Three consequences follow from the formula, before any vector index is involved.

Saturation. With $k_1=1.2$, extra copies of a term help less and less. Repeating a product code does not multiply the score. Raising $k_1$ delays saturation. Lucene allows $k_1=0$ and forbids a negative value.

Length. $b=0.75$ penalises a long field relative to the average field in that index. $b=0$ turns the penalty off. A short title and a long body should not silently share one $b$.

The stored length is coarse. Lucene keeps the norm in one byte (SmallFloat). The explain path calls the decoded length approximate once the encoded byte is above 39. A tie in dl can be that compression.

Elasticsearch's similarity page, retrieved the same day, states the same defaults, 1.2 and 0.75, and says overlap tokens are ignored in the norm by default. The settings are k1, b and discount_overlaps. The type name it does print is BM25, and similarity applies per field to text and keyword. Custom similarity is described as an expert setting. Changing the index default after creation requires closing the index, updating, and opening it. Closing takes the index out of search. That sequence was not run here.

Why the vector score is a different kind of number

A dense retriever encodes the query and the chunk separately, then ranks by the similarity the index was built with. Elasticsearch's fusion example uses dense_vector with similarity: l2_norm. OpenSearch's examples use knn_vector with space_type: l2, and also a neural query. A BM25 score of 12 and a converted distance of 0.98 are not on one scale, and a weighted sum of the raw values lets the wider numeric range win.

Reciprocal rank fusion, from the 2009 paper

Cormack, Clarke and Büttcher sort documents by

with $k=60$ fixed after a pilot and then left alone. $r(d)$ is the rank of document $d$ in ranking $r$. In the paper each ranking is a permutation of the whole set, so every document has a rank in every list. Their reason for the shape of the formula is stated directly: high ranks should matter more, but low ranks should not fall to zero the way an exponential would, and $k$ should stop one outlier system from dominating through its top hit. They also note that fusion can be accumulated one system at a time, because only the rank is required.

On the pilot that fused 30 Wumpus runs for TREC topics 351–400, mean average precision at $k=60$ was 0.2145, against 0.2016, 0.2074 and 0.2039 for the best single system, Condorcet and CombMNZ. From $k=0$ to $k=100$ it stayed near that peak (0.2072 at 0, 0.2147 at 80) and fell to 0.2098 at $k=500$. The authors call 60 near-optimal and the choice not critical. Across the pilot and TREC sets they summarise average gains near 4% to 5%; this was not an across-the-board win. CombMNZ exceeded RRF on TREC 3 and on LETOR, with the latter difference not significant. The exception is TREC 9, where the best individual run was human-in-the-loop (0.3519) and RRF scored 0.2830. That is fusion of whole rankings from that era, not BM25 plus a modern embedding.

What $k=60$ does to a two-list hybrid

Production engines do not hold a permutation of the whole corpus. Elasticsearch's documented formula only adds a term when the document is in that query's result set, and rank starts at 1:

score += 1.0 / (k + rank)

The default ranking constant is 60, and it must be at least 1. A larger constant flattens the influence of the top ranks. rank_window_size is the size of each child result set; it defaults to the search size, and it must be at least 1 and at least size. If a k-nearest-neighbour k is larger than the window, Elasticsearch truncates to the window. If it is smaller, you only have k neighbours to fuse.

Work the default constant by hand. Rank 1 on a single list and absent from the other scores $1/61\approx 0.0164$. Rank 1 on both lists scores $2/61\approx 0.0328$. A document that sits at the same rank $r$ on both lists scores $2/(60+r)$. That beats the lone rank-1 document whenever $r<62$. So a chunk that is merely present in both windows, even around rank 50, outscores a chunk that is the single best lexical hit and was never returned by the vector query.

Our inference from the entity-question results is that similar failures are worth testing for identifiers and error codes. Sciavolino did not evaluate those query types or RRF; the rank arithmetic separately shows how fusion can reward shared hits over a lone lexical winner. Keep an explicit lexical quota in the candidate set if an exact token must survive. The shape does not depend on 60. A lone first place scores $1/(k+1)$; two copies of rank $r$ score $2/(k+r)$.

Elasticsearch's RRF concepts page describes equal-weight fusion. Its current RRF retriever reference additionally supports a non-negative per-child weight, default 1.0, so equal weighting is not the only available behaviour. Confirm the supported syntax for the deployed release. Scroll, sort and rescore are unsupported and raise an exception. RRF builds its own point-in-time so the child lists match; the page says not to supply one yourself. An example on that page uses rank_constant: 20. Twenty is legal. It is neither the default nor the constant the 2009 pilot froze.

OpenSearch: normalise scores, or fuse ranks, on purpose

OpenSearch splits the job. A hybrid query holds the subqueries. A search pipeline combines them. On the hybrid-query page retrieved as "latest": queries allows at most five clauses, and a hit must match at least one. pagination_depth is how many hits each subquery returns per shard, from 1 to index.max_result_window (default 10000). It is required when from is greater than 0; otherwise each subquery returns up to size per shard. That depth is the set the pipeline can see. A hybrid filter applies to every subquery, which is the documented place for a tenant predicate. The hybrid query must be top-level. Nesting it under function_score, constant_score, script_score or boosting can error or skip the pipeline. A bool query is the documented substitute, and it does not run the pipeline. From 2.18, rescore runs per subquery on the shard, before the pipeline, and sorting together with rescore is rejected. From 3.5, min_score applies after combination and only for _score order, and an index with more than 512 shards turns batched reduction off. The _msearch path does not get that handling.

The normalisation processor, introduced in 2.10, rescales each clause from that clause's own returned hits, not from the whole index. The default technique is min-max:

The top hit in the clause becomes 1.0. An exact 0.0 is replaced by 0.001, because 0.0 means match_none. If every hit shares one score, each becomes 1.0. The substitution can reorder the tail: the documented example maps raw scores 10000, 9 and 8 to 1.0, about 0.0001 and 0.001, so 8 ranks above 9. L2 divides by the clause's Euclidean norm and keeps ratios. Z-score standardises, then replaces every result at or below 0 with 0.001, so everything below the mean ties, and it may only be combined with an arithmetic mean.

Combination defaults to the arithmetic mean. Weights, if set, lie in [0, 1] and must sum to 1. A document missing from a clause contributes 0 and still counts in the denominator, so a one-clause hit is halved under an equal two-clause mean. The page's toy comparison produces a different winner under each technique. Their nDCG@10 note prefers a size of 100 to 200 on sets up to 10 million documents, and says a larger size added latency without a relevance gain in those experiments. That is their range. Measure it on the real collection.

The score-ranker processor, introduced in 2.19, is the rank path: technique rrf, default constant 60, legal range 1 to 10000. Weights, if set, again lie in [0, 1] and must sum to 1. The original paper uses an unweighted sum. Both OpenSearch and the current Elasticsearch RRF retriever reference support weighting, with their own validation rules. A 0.7/0.3 split is a hypothesis, not a result. Min-max statistics and RRF ranks are both computed on the hits the window returned, so pagination_depth and shard boundaries are part of the ranking.

The reranker is a different model, on a short list

Nogueira and Cho (2019) describe the stage that sits after a first-pass retriever. BM25 supplies a large candidate set, on the order of a thousand passages. A BERT model then scores each candidate on its own. The query is sentence A, truncated to at most 64 tokens. The passage is sentence B, truncated so that query, passage and separator tokens fit in 512. A linear layer on the [CLS] vector yields the probability that the passage is relevant. Passages are sorted by that probability. Training is cross-entropy on relevant and non-relevant passages drawn from the BM25 top 1000. They report ${\mathrm{B}\mathrm{E}\mathrm{R}\mathrm{T}}_{\mathrm{L}\mathrm{A}\mathrm{R}\mathrm{G}\mathrm{E}}$ MRR@10 of 36.5 (development) and 35.8 (evaluation) on MS MARCO, against 16.7 and 16.5 for untuned Lucene BM25, and a 27% relative gain over the previous leaderboard entry (28.1 evaluation MRR@10 in their table). On TREC-CAR the same row is MAP 33.5 against untuned BM25 at 12.3; tuned Anserini BM25 reached 15.3. Those figures are that paper's.

This is a cross-encoder: query and passage run in one forward pass, so tokens can interact. A bi-encoder froze the passage vector before the query existed. One pass per candidate, up to 512 tokens, is why the reranker sees a short list. It cannot promote a chunk the first stage never returned. Nogueira and Cho note that some MS MARCO relevant passages are missing from the BM25 top 1000 because annotation started from Bing's top 10. Fusion widens the pool. It does not search the remainder. Log the fused order and the reranked order separately: a chunk that led RRF can leave the top ten after the cross-encoder, and that is the stage doing its job.

What to choose and why

Judge the choice with a frozen question set and graded chunks, not with a single prompt that "looks better". How to build that set is its own subject. Until it exists, keep the window, the constant $k$, the normalisation technique and the rerank depth written next to the index version, because those four change the order even when the embeddings do not.

Risks and limits

• $k=60$ was frozen on TREC fusions of full permutations. A rank_window_size that follows a page size of 10 is a different algorithm. The pilot did not test BM25 against a modern embedding.
• Min-max uses the min and max of the hits returned, replaces exact zero with 0.001, and can swap low scores when one score dominates. Z-score ties every below-mean hit.
• Under $k=60$, the same rank $r$ on both lists beats a lone rank 1 whenever $r<62$. That arithmetic motivates testing lexical-only cases; an identifier-specific effect was not measured in the cited paper.
• A reranker cannot recover a chunk the first stage dropped, and the BERT setup above cuts the query at 64 tokens and the pair at 512.
• The cited gains belong to those papers and checkpoints. They are not a forecast for another embedding or another language.
• Changing similarity or the pipeline changes results. Elasticsearch's close/open cycle takes an index out of search. OpenSearch can disable hybrid queries with plugins.neural_search.hybrid_search_disabled.
• No NetLeverage product behaviour is claimed.

Sources

Claim-level notes, access dates and version limits are in the evidence file. All opened 10 November 2026.

1. Apache Lucene 10.3.0 BM25Similarity — https://github.com/apache/lucene/blob/releases/lucene/10.3.0/lucene/core/src/java/org/apache/lucene/search/similarities/BM25Similarity.java
2. Elasticsearch similarity — https://www.elastic.co/docs/reference/elasticsearch/index-settings/similarity
3. Elasticsearch reciprocal rank fusion — https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reciprocal-rank-fusion
4. OpenSearch hybrid query — https://docs.opensearch.org/latest/query-dsl/compound/hybrid/
5. OpenSearch normalisation processor (2.10) — https://docs.opensearch.org/latest/search-plugins/search-pipelines/normalization-processor/
6. OpenSearch score ranker (2.19) — https://docs.opensearch.org/latest/search-plugins/search-pipelines/score-ranker-processor/
7. Cormack, Clarke and Büttcher, SIGIR 2009 — https://cormack.uwaterloo.ca/cormacksigir09-rrf.pdf
8. Nogueira and Cho, arXiv:1901.04085 — https://arxiv.org/html/1901.04085

9. Sciavolino, Zhong, Lee and Chen, arXiv:2109.08535 — https://arxiv.org/html/2109.08535

10. Elasticsearch RRF retriever — https://www.elastic.co/docs/reference/elasticsearch/rest-apis/retrievers/rrf-retriever (opened 10 November 2026). Optional per-retriever weights and current retriever parameters.

This article was originally published on 10 November 2026.