The pgvector halfvec trick: how we index 3072-dim embeddings

This is an engineering note for the small group of people who will Google “pgvector 3072 dimensions” and land here. If you’re that group, hi.

The problem

OpenAI’s text-embedding-3-large returns 3072-dimensional vectors. It’s what most production RAG stacks use in 2026.

pgvector’s HNSW index has a hard ceiling: 2000 dimensions for the vector type. Try to build an HNSW index on vector(3072) and you get:

ERROR: column cannot have more than 2000 dimensions for hnsw index

You need HNSW. IVFFlat is slower and doesn’t scale as well past a few million rows.

The workaround

pgvector 0.7+ adds halfvec — a 16-bit float variant of vector. Its HNSW ceiling is 4000 dimensions. Half the memory, negligible recall loss in every test we’ve run.

Store as vector(3072) but build the index on the halfvec cast:

CREATE INDEX chunks_embedding_idx
  ON chunks
  USING hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops)
  WITH (m = 16, ef_construction = 64);

The important bit: the query has to cast identically. If you write WHERE embedding <=> $1 where $1 is a plain vector, Postgres does a sequential scan and ignores your index.

The correct query:

SELECT id, embedding <=> $1::halfvec(3072) AS distance
FROM chunks
WHERE chatbot_id = $2
ORDER BY embedding::halfvec(3072) <=> $1::halfvec(3072)
LIMIT 20;

Both sides of the operator have to be halfvec. Miss this and you get correct results at 10× the latency — an easy production bug to miss.

Recall in practice

We tested on our own corpus (~50M chunks) and a public benchmark (Wikipedia article sections). The halfvec-indexed HNSW returned the same top-10 in ~97% of queries as the full vector variant. In the remaining 3%, the top-10 differed by one or two positions with substantially similar cosine distances. For RAG, this is invisible.

If your workload is semantic search where minor ranking differences matter (product ranking, search-as-relevance-ordering), test on your own corpus. For RAG feeding an LLM, the difference is smaller than the LLM’s own generation variance.

The parameters

Two knobs that matter: m and ef_construction. Higher = better recall, slower build, more memory.

Our production parameters:

WITH (m = 16, ef_construction = 64)

Query time: SET ef_search = 40. Tune ef_search per session for the recall/latency trade-off you need. We use 40 as default; some workloads bump it to 80 for high-recall use cases.

The gotcha nobody mentions

Halfvec’s precision loss is not uniform. It’s tighter in the range [-1, 1] and loses precision toward the tails. If your embeddings routinely have components > 4 in absolute value, halfvec will quantise them harder.

OpenAI’s text-embedding-3-large almost never produces components outside [-1, 1] because it normalises. If you use a different embedding model, check its output distribution before assuming halfvec is safe.

The one-line summary

Store as vector(3072). Index the ::halfvec(3072) cast. Query with the same cast on both sides. Verify with EXPLAIN that the index is being used.

If you want to see this in production, Anserra’s schema is public. Everything above is what actually runs behind the retriever.