Hybrid search: how to combine BM25 and vector search without losing the best result

Most hybrid search guides stop at "run BM25 and vector search, merge the results with reciprocal rank fusion, done". But how you combine the two matters as much as using both, and the default combination quietly loses your most precise results.

Here is a real query from Wayfair's public search dataset, run against 43,000 products. The shopper typed a specific rug: "regner power loom red". Keyword search (BM25) ranks the right rug first. Vector search doesn't find it in its top 100. Merge the two lists with reciprocal rank fusion (RRF), the most common way search engines and vector databases merge them, and the one exact match falls to number 56.

Nothing crashed. Across all 480 queries in the dataset, the merged results were better on average than either list on its own. That's exactly why this kind of failure survives: the average goes up, and the shopper who knew what they wanted gets five other red rugs.

In short: "hybrid search" usually means two retrievers whose results are merged afterwards. That's the weakest version. It works better when every candidate is scored on both keywords and meaning before it leaves the index, when query understanding decides how much each signal counts, and when a ranker trained on your own data has the final say. Then measure every change per type of query, because averages hide the queries you broke.

"Hybrid search" means too many things

The term is overloaded. Most of the time it means: run a keyword search and a vector search, then merge the two result lists. I don't think that's true hybrid. You can have two, three or ten retrievers (candidate generators, in search terms) and still fail miserably at ranking what they bring back. More retrievers mean more candidates, not better ranking.

True hybrid search scores every candidate on both signals. At Vantage Discovery, where I led the ML team, that's how we built it. We calculated the keyword score (TF-IDF or BM25) and the embedding score in a single pass, as long as latency stayed under control. Even after the index grew as we onboarded onto the Shopify ecosystem, we kept doing it at the retrieval stage, inside the shards: a keyword shortlist followed by embedding scoring, and we could run it in either order. We didn't wait for the root ranker to decide between two lists. Candidates that scored well on both had already won.

The difference matters because of what the merge step can and can't see. In the setup on the left, a document found only by BM25 arrives with no vector score, and one found only by vector search has no keyword score. The merge has to guess. RRF guesses by ignoring scores altogether, which is how the Regner rug ended up at number 56.

Why combine BM25 and vector search at all

I covered the failure modes of embeddings in why embeddings don't work for your search. In one line: embeddings handle paraphrase and meaning, and keywords handle product codes, rare words, names and exact phrasing. Each finds results the other misses.

In practice, adding embeddings to keyword search did two things for me. It removed the need to maintain long, elaborate synonym lists, because the model already knows that a couch is a sofa. And it reduced, though didn't eliminate, how much we leaned on click-based boosting (sometimes called navboost: promoting results for the queries they were clicked for) to improve retrieval.

But the benefits only show up if the merge is done well.

What happens when you bolt on a second retriever

At Roblox, we added an open-source vector database to search for the avatar marketplace, as a call running in parallel with the existing Lucene-based keyword search. That meant the merge had to do a lot of the heavy lifting.

Overall, the metrics improved. Some results are better than none, especially when your biggest audience is young and has a lot of time to browse. But it broke a lot of other queries: "irrelevant content" started popping up where keyword search had been precise. The average went up, and specific queries went visibly wrong.

That pattern isn't unique to Roblox. It's built into how late merging works.

How reciprocal rank fusion buries the exact match

Reciprocal rank fusion (Cormack, Clarke and Büttcher, SIGIR 2009) ignores scores and uses only each document's rank in each list:

RRF score(d) = Σ 1 / (k + rank of d in each list)

k is a constant, usually 60. A document ranked first in one list gets 1/61. A document that appears in both lists gets both contributions added together.

RRF is the most popular choice for good reasons. It needs no training data, it doesn't care that BM25 scores and cosine similarities live on different scales, and no single outlier score can take over. But rewarding documents that appear in both lists has a cost: a document found by only one retriever is capped at 1/61, however confident that retriever was. And any document in the top 61 of both lists outranks it. Even at rank 61 in both, a document scores 2/121 = 0.0165, just above 1/61 = 0.0164.

For an identifier or a specific product name, the document found by only one retriever is often the one the user wants.

What the research says about RRF vs weighted sums

The alternative to merging ranks is merging scores: normalise each retriever's scores to a common range, then take a weighted sum.

hybrid score(d) = α · norm(vector score) + (1 − α) · norm(BM25 score)

The most careful comparison I know is An Analysis of Fusion Functions for Hybrid Retrieval (Bruch, Gai and Ingber, ACM TOIS 2023). They found that this weighted sum (a convex combination) beat RRF both on the data it was tuned on and on unfamiliar data, and that tuning α needed only a small set of labelled queries. RRF, despite its reputation for needing no tuning, turned out to be sensitive to its k. In other words, the default isn't the best choice.

Normalising is its own problem. BM25 scores have no fixed range: a score of 12 is excellent for one query and weak for another. Cosine similarities usually sit in a narrow band. Min-max scaling per query (best becomes 1, worst becomes 0) is the usual fix, with z-scores and distribution-based score fusion as alternatives. Bruch et al. found the choice of normalisation mattered less than expected.

I measured it: a hybrid search benchmark on WANDS

To check how much of this holds on product search, I ran a small benchmark on WANDS, Wayfair's public dataset: 480 real search queries, 43,000 furniture and home products, and 233,000 human relevance labels. The code is on GitHub, so you can rerun it or try it on your own data.

Setup. BM25 over product name, class, category and description (using the bm25s library). Vector search with two off-the-shelf embedding models: bge-base-en-v1.5, a strong model, and all-MiniLM-L6-v2, a small, fast one. I measured nDCG@10, counting "exact" labels as 2 and "partial" as 1. Each retriever returned its top 100. Every weight, and the learned ranker, was tuned with 5-fold cross-validation, so no query was scored by settings tuned on itself. I compared:

• BM25 only and vector only
• RRF, k = 60, on the two top-100 lists
• Late weighted sum: min-max normalised scores and one tuned α. A document missing from one list gets zero for that side, which is all a late merge knows.
• Single-pass hybrid: the same weighted sum, but every product gets both scores, as in the Vantage setup
• Single-pass with α per query type: a simple classifier splits queries into named or rare-word queries, long descriptive queries, and the rest
• A learned ranker: gradient-boosted trees over the candidates from both lists, with features like both scores, both ranks, how much of the query matches the product name, and review counts

What I took from it:

1. Hybrid search beat both retrievers, with both embedding models. With bge-base, RRF added 2.4 points over vector search alone. With the weaker MiniLM model, vector search fell below BM25 (66.7 vs 68.9), and hybrid search still added 3.6 points over BM25. The two lists overlapped by less than half: of the products labelled exact matches, each retriever's top 100 held about 70%, and the two together held 83%.
2. Giving every candidate both scores beat late merging, with the same formula. Single-pass hybrid beat the late weighted sum by 0.7 points with bge-base and 1.2 with MiniLM. Both differences held up under a bootstrap test, with 95% intervals of 0.2 to 1.1 and 0.7 to 1.8 points. The late weighted sum didn't beat RRF here, unlike in Bruch et al. My reading: when half of each document's scores are missing, a score-based merge loses its advantage.
3. There is no universal α. The best weight for the embedding side was 0.7 with bge-base and 0.6 with MiniLM. Copy someone else's α and you're tuning for their model and their data.
4. Weighting by query type didn't help on this dataset. WANDS queries are mostly descriptive furniture searches with almost no product codes, and my classifier was a crude heuristic. That doesn't make query understanding useless (more on that below), but it does mean you should measure it rather than assume it.
5. The learned ranker was best on average, and also broke the most queries. It added about 1.3 points over RRF, gained most on queries with numbers and sizes, and lost ground on long queries. The next chart shows why averages aren't enough.

Every method made dozens of queries clearly worse while making more of them clearly better. RRF improved 79 queries and hurt 46. The learned ranker improved 106 and hurt 48. That's the Roblox pattern in numbers: the average goes up and specific queries go wrong. And none of the methods rescued the Regner rug from the example at the top. Single-pass hybrid lifted it from 56 to somewhere between 35 and 42, depending on the tuned weight: still far below the fold.

Caveats. This is one dataset, in one domain, with no SKUs or model numbers. A ranker trained on 480 queries is far smaller than a production one. Differences under about half a point are within noise. Treat these numbers as a demonstration of the trade-offs, not as a leaderboard. The benchmark code and full results are public.

Let query understanding decide the weights

Depending on the query, you often need to decide early whether it's lexical (the words themselves matter: a brand, a model number, an exact phrase) or semantic (the meaning matters: "comfy chair for reading in a small bedroom"). That takes a reasonably good query understanding layer with a classifier, so retrieval knows which score should count more for this query.

One weighting does not work for every query, especially for torso and tail queries: the long middle and the long end of your query distribution, where each query is rare and there's little click data to lean on. On WANDS, my crude classifier added nothing, which says more about the classifier than about the idea. A good one is trained on your own queries and judged by its effect on each query type, not by its own accuracy.

Put a ranker on top, and add it in the right order

All of this means that any merge, early or late, needs a machine-learned ranker: something that combines query understanding with the retrieval scores and learns when to favour one or the other. Exact match belongs in it as a strong feature, along with which fields matched, the query type and business signals. That works better than special-case routing, such as sending everything that looks like a code down a separate lookup path. Hard routing breaks on mixed queries like "QN65Q80C wall mount", and leaves you maintaining a growing pile of regular expressions.

If your traffic is varied, or users search in recognisable patterns, engagement data (clicks, add-to-carts, purchases) will help the ranker merge better, provided your retrieval isn't broken. If the right items never come back from retrieval, people can only click the wrong ones, and the ranker learns from those clicks.

Where the rankers live matters too. I'd put a lightweight ranker inside the shards, such as gradient-boosted trees over cheap features, and a more complex neural model only on the final list. But I wouldn't start with the neural model. Add it only once you're confident the lightweight shard ranker has stopped improving. Otherwise the bigger model eats the potential gains too early, and you now have to keep several rankers in tune with each other, a problem you didn't need to have this early on.

Before you merge: check the candidates

A merge can only reorder documents one of the retrievers found. Many hybrid search problems start before the merge.

• Retrieve enough from each side. If each retriever returns only its top 10, the right document often isn't in either list. Take 50 to 200 from each.
• Apply the same filters to both sides. If the keyword query filters by category or availability and the vector query doesn't, the vector side fills the list with results that should never appear.
• Give identifiers their own field. A standard text analyzer may split QN65-Q80C into pieces. Index identifiers in a keyword field with light normalisation (remove hyphens and spaces, fold case) so qn65q80c, QN65-Q80C and QN65 Q80C all match.
• Measure the union before the merge. For each test query, check whether the right document appears in either list. If it's in neither, no merge will save it. Also check how much the two lists overlap. If they share 90% of their documents, hybrid search adds cost and little else.
• Evaluate each retriever on its own. If the two lists are very different, that isn't automatically good news. It can also mean your embeddings are broken: the wrong model for your domain, truncated documents, or a mismatch between how queries and documents were encoded. Run the same relevance evaluation on each retriever independently before you judge the merge.

How to measure whether hybrid search helped

1. Group your test queries by type: identifiers, brand and product names, short head queries, long descriptive queries, and queries with constraints like negation or price.
2. Compare at least four systems: BM25 alone, vector search alone, hybrid search with default settings, and your tuned version. If hybrid search doesn't beat the better single retriever in every group, find out why before shipping.
3. Report nDCG and recall per group, not only the average.
4. Count the queries you broke. For each change, count how many queries got clearly better and how many got clearly worse, as in the chart above. Then read the worst ones.
5. Track the exact-match rank. For identifier and product-name queries, record where the exact product ranks before and after the merge. It should be first, every time.
6. Know where your labels come from. Is your nDCG computed from user engagement, from labels by real people, or from an LLM with a giant prompt? Each has blind spots: engagement only covers what users were shown, people disagree and get tired, and an LLM judge has its own preferences. You fix what you measure, so make sure you aren't measuring the wrong thing.
7. Confirm with an A/B test. Offline metrics tell you where to look. Clicks, add-to-carts and "searched, found nothing, rephrased" rates tell you whether users noticed.

A hybrid search checklist

Which merge to use when:

• No labelled queries yet: RRF, with generous candidate lists from both sides. It's a safe start, but check your exact-match and product-name queries before you ship.
• A few hundred labelled queries: a weighted sum of normalised scores, with α tuned on your own data. Not copied from a blog post, this one included.
• You control the index: score every candidate on both signals in one pass, so nothing reaches the merge with half its scores missing.
• Engagement data and varied traffic: a learned ranker on top, with exact match and query type as features. Lightweight first; a neural model only once that stops improving.

Three checks before you ship:

1. Does the right document appear in at least one list (the union), for each query type?
2. Does hybrid search beat the better single retriever in every query group, not just on average?
3. How many queries got clearly worse, and have you read them?

Hybrid search in Elasticsearch, OpenSearch, Vespa and vector databases

Most engines now support hybrid search, but they differ in the distinction this article is about: whether they merge two separately scored lists, or can score candidates on both signals together. Check the documentation for your version, because these features change quickly.

Merge two result lists (late fusion):

• Elasticsearch: the rrf retriever, and in recent versions a linear retriever for weighted sums with minmax or l2_norm normalisation. Some features depend on your license tier.
• OpenSearch: a hybrid query with a search pipeline: the normalisation processor (since 2.10) for weighted sums, and RRF since 2.19.
• Milvus: native BM25 full-text search since 2.5, combined with dense vectors through hybrid_search and an RRFRanker or WeightedRanker.
• Qdrant: the Query API runs sparse and dense searches as prefetch stages, then merges them with RRF (weighted in recent versions) or distribution-based score fusion.
• Weaviate: a built-in hybrid query with an alpha parameter and a choice of rank-based or relative-score fusion.
• MongoDB Atlas: $rankFusion (MongoDB 8.0 and later) and $scoreFusion (8.2 and later) merge a full-text pipeline with a vector search pipeline.
• Postgres with pgvector: no built-in hybrid search. Run a full-text query and a vector query and merge them in SQL. Postgres's own ts_rank isn't BM25; extensions such as ParadeDB's pg_search add it.

Score both signals together:

• Vespa: a rank profile can combine bm25() and vector closeness() in one expression, and a single query can retrieve from the text index and the vector index together. See its hybrid search tutorial. This is the closest to single-pass hybrid out of the box, and Vespa also supports multi-phase ranking with your own models.
• Pinecone: a single index can store sparse (keyword) and dense vectors on the same record and score them together, with the weighting applied to the query vectors.

Whichever engine you use, the defaults are a starting point. The right weights are something you find by measuring on your own queries.

The short version

Hybrid search is the most reliable fix for embedding-based search, and turning it on takes days. Getting it right takes longer: candidates scored on both signals, query understanding that sets the weights, a ranker that knows what an exact match is, and an evaluation that counts the queries you broke, not just the average.

If your hybrid search went live with the default settings and you're not sure what it's doing to your product-name or long-tail queries, that's what a search relevance audit checks: we measure each query type separately, find where the merge loses good results, and pick the cheapest fix for each.

FAQ

What is hybrid search?

Hybrid search combines keyword search, usually BM25, with vector search based on embeddings, so results can match both the exact words and the meaning of a query. Most implementations run the two searches separately and merge the results. Stronger implementations score every candidate on both signals, then let a ranker order them.

BM25 vs vector search: which is better?

Neither on its own. BM25 is better at exact identifiers, names and rare words; vector search is better at paraphrases and descriptive queries. On Wayfair's WANDS dataset, a strong embedding model beat BM25 while a small one lost to it, and combining them beat both in each case.

How do you combine BM25 and vector search?

The two common methods are reciprocal rank fusion, which merges the two ranked lists by position, and a weighted sum of normalised scores. A weighted sum tuned on a few hundred labelled queries often does better than RRF, especially when every candidate has both scores. A learned ranker on top can improve the results further.

What is reciprocal rank fusion (RRF)?

Reciprocal rank fusion merges ranked lists by giving each document 1 / (k + rank) from every list it appears in, with k usually 60, and adding the results. It needs no score normalisation or training data. Its weakness is that a document found by only one retriever, such as an exact product-name match, can end up below documents that are mediocre in both lists.

Should I use RRF or a weighted sum?

Start with RRF if you have no labelled queries: it's robust and needs no tuning. Once you have a few hundred labelled queries, try a weighted sum of normalised scores, tuned on your own data, and compare them per type of query. Research by Bruch et al. (2023) found the tuned weighted sum generalised better than RRF.

How do you normalize BM25 scores for hybrid search?

BM25 scores have no fixed range, so normalise them per query before combining them with vector similarities. Min-max scaling (best result becomes 1, worst becomes 0) is the most common. Z-scores and distribution-based score fusion are alternatives that are less affected by outliers. The weight you combine them with matters more than the normalisation method.

What is a good alpha for hybrid search?

There isn't a universal one. In my WANDS benchmark, the best weight on the vector side was 0.7 with one embedding model and 0.6 with another. Tune α on your own labelled queries, and ideally per type of query, since product-code searches and descriptive searches need very different weights.

Do you need a reranker with hybrid search?

Usually, yes. A merge only sees one or two numbers per document. A learned ranker or a cross-encoder reranker can use many more signals, such as exact field matches, query type and engagement, and fix the cases where the merge orders candidates badly. Start with a lightweight ranker and add heavier models only when it stops improving.

Does hybrid search help RAG?

Yes, often more than in classic search. RAG questions frequently include exact names, product codes, error messages or section numbers that embeddings match poorly, and a wrong passage at retrieval time becomes a confident wrong answer. Hybrid retrieval with a ranker on top is the standard setup for production RAG.

Which vector databases support hybrid search?

Most do: Elasticsearch, OpenSearch, Milvus, Qdrant, Weaviate, Pinecone, Vespa and MongoDB Atlas all offer hybrid search, and Postgres can do it with pgvector plus full-text search. The difference is in how they combine signals: most merge two separately ranked lists, while Vespa and Pinecone's single-index setup can score keyword and vector signals together.