Hybrid retrieval, explained: why semantic search alone fails on real customer questions

Semantic search is astonishingly good at paraphrase. “Do you offer refunds?” and “Can I get my money back?” become nearby vectors and retrieve the same passage. That’s the case every demo shows.

Real customer support questions include SKUs, error codes, account IDs, model numbers, order references and part numbers — strings whose meaning is the exact characters, not what they resemble. Semantic search blurs those into their neighbourhood, and the neighbourhood is not the answer.

Hybrid retrieval is what fixes this in production.

Two retrievers

A hybrid system runs the same query through two very different retrievers:

Semantic retriever. Embed the query with the same model used to embed your documents. Ask the vector database (pgvector in our case) for the k passages closest in cosine similarity. Excellent at intent and paraphrase.

Lexical retriever. Search the same corpus with keyword-based full-text search. In Postgres this is `to_tsvector` for stemmed matching plus a trigram index for typo tolerance. Excellent at exact strings, numbers and rare terms.

Neither is complete on its own. You need both.

Reciprocal Rank Fusion

The naive way to combine two ranked lists is to add or average their scores. This doesn’t work because the score distributions are different — semantic similarity is 0..1, keyword ranks are unbounded — and normalising them is fragile.

Reciprocal Rank Fusion (RRF) sidesteps the problem by ignoring scores entirely. It uses positions:

``` score(doc) = Σ 1 / (k + rank(doc, list_i)) ```

for each retriever `list_i` the document appears in. `k` is a smoothing constant, typically 60. A document that both retrievers agree on ends up above documents that top only one list. That’s exactly the property you want: agreement is signal.

We use `k = 60` in Anserra and it’s held up across every dataset we’ve tested.

Why this matters in practice

A specific example. A customer pastes an order number:

“Where is order #29387?”

Semantic retriever: returns three articles about “checking your order status,” a policy about lost shipments, and an FAQ about tracking. All relevant, none carry the customer’s actual order.

Lexical retriever: matches the article that literally contains “29387” — the internal tool description. Wrong article, but it demonstrates the retriever is doing its job.

Fused list: articles that top both are ranked first, so tracking guidance appears before general “order status” content. And because RRF returns the reasons for ranks, we can flag “no lexical match” and route the customer to a human — the specific order isn’t in any indexed document.

Implementation notes

Two engineering details worth capturing:

Postgres does both. With pgvector for embeddings and `to_tsvector` + a GIN trigram index for lexical search, you don’t need a second database. Cross-encoder rerankers are optional add-ons; RRF is usually enough.

Cache the query embedding. Roughly a third of visitor questions are repeats within an hour. Cache the embedding vector by hash of the query text. Retrieval latency drops from ~200ms to ~5ms.

Index the halfvec cast. pgvector’s HNSW index has a hard ceiling at 2000 dimensions for `vector`. `text-embedding-3-large` is 3072. The workaround: build the index on a `halfvec` cast. Half the memory, negligible recall loss, and the query has to cast identically or Postgres skips the index. This is one of the most common misconfigurations I see.

What to measure

Two production metrics tell you whether hybrid retrieval is doing its job:

• Coverage. The share of queries where either retriever returned at least one passage above threshold. Should be > 95%. A coverage drop is your first signal that content is missing.
• Overlap. The share of top-3 results that appear in both retrievers’ top-10. Higher overlap = the query type was easy for both; lower overlap = the query was disagreed on, and you’ll want to sample those manually to catch retrieval failures.

If you want to build this yourself, read our chatbot builder documentation. If you’d rather not, Anserra ships hybrid retrieval by default — no configuration required.