# Hybrid search and reranking

> Combine keyword and vector search, then let a reranker put the best few on top.

Concept · Embeddings & retrieval · https://levelcode.org/concepts/hybrid-search-reranking

Vector search and keyword search fail in opposite ways:

| | Keyword (BM25) | Vector (embeddings) |
|---|---|---|
| Finds | Exact terms: `ERR_CONN_RESET`, SKU `A-1742`, names, acronyms | Paraphrases: "can't log in" ≈ "authentication failure" |
| Misses | Synonyms and rephrasing | Rare exact tokens and identifiers |

**Hybrid search** runs both and merges the results, so a query gets the benefit of each.

## Keyword side: BM25

BM25 is the standard ranking function behind full-text search ([inverted indexes](https://levelcode.org/concepts/inverted-index)). It scores a document higher when it contains the query terms, more so for rare terms and short documents. Elasticsearch, OpenSearch and most vector databases provide it. PostgreSQL's built-in full-text ranking (`ts_rank`) is a simpler relative; extensions such as ParadeDB's `pg_search` add true BM25.

## Merging: reciprocal rank fusion

The two systems' scores aren't on the same scale, so merge by **rank** instead:

```text
RRF score(doc) = Σ over result lists   1 / (k + rank of doc in that list)     (k ≈ 60)
```

A document ranked well by both lists rises to the top. It's simple, needs no tuning of score weights, and is hard to beat as a default.

## Reranking

Both retrievers are **bi-encoders**: the query and each document are encoded separately, which is fast but shallow. A **reranker**, usually a **cross-encoder**, reads the query and a candidate *together* and scores how well the candidate answers the query. That's much more accurate, but too slow to run over a whole corpus. So you run it in stages:

```text
hybrid retrieve top 50–100  →  rerank  →  keep top 5–10  →  model
```

Reranking often makes the largest single quality improvement in a RAG pipeline after [chunking](https://levelcode.org/concepts/chunking). It also lets you send *fewer* chunks to the model, which saves [context](https://levelcode.org/concepts/context-windows) and cost. Hosted rerank APIs and open cross-encoder models both work; an LLM can rerank too, at a higher cost.

## Habits

- **Measure retrieval separately from generation:** is the right chunk in the top 5 before and after the change?
- **Filter by metadata first** (tenant, permissions, date) so neither retriever wastes slots on documents the user can't see.
- **Keep the reranker's top *k* small.** The model answers better from 5 strong passages than from 20 mixed ones.