Vector databases and indexes

The problemCopy Anchor Link

Exact nearest-neighbor search compares the query with every vector. With a million vectors of 1,536 numbers that is over a billion operations per query. Approximate nearest neighbor (ANN) indexes skip most of the comparisons and return results that are almost always the true nearest.

Index typesCopy Anchor Link

HNSW builds layers of graphs: start at a sparse top layer, jump toward the query, then refine in denser layers. It is fast and accurate and the common default, but uses memory. IVF clusters the vectors and searches only the few closest clusters. Product quantization compresses vectors into short codes to save memory at some accuracy cost. Flat indexes do exact search and are fine up to roughly 100 thousand vectors.

Choosing oneCopy Anchor Link

Under a million vectors, almost anything works, so choose by what you already run. pgvector adds vector columns to Postgres. Chroma and LanceDB are simple embedded options. Qdrant, Weaviate and Milvus are dedicated open-source servers. Pinecone is a managed service. Supabase (Postgres with pgvector) and MongoDB Atlas add vector search to databases many teams already use. Jina and Cohere sell embedding and reranking models. Look for metadata filtering, hybrid search and how deletes and updates behave.

A worked example: why indexes existCopy Anchor Link

One million vectors of 1,536 numbers: comparing a query with all of them is about 1.5 billion multiplications per search. An HNSW index jumps through a graph of neighbors and checks maybe a few thousand vectors, finding the nearest almost every time. You trade a tiny chance of missing the very best match for a hundred-fold speedup.

How a graph search finds a neighborCopy Anchor Link

An HNSW index links each vector to a few nearby vectors, and keeps sparser copies of the graph on top. A search runs like this:

1. Start at a fixed entry point in the top layer.
2. Move to whichever linked vector is closest to the query, and repeat until no neighbor is closer.
3. Drop to the next, denser layer and continue from that spot.
4. At the bottom layer, keep a short list of the best candidates and return the closest ones.

It is like finding a street by first picking the right city, then the district, then the block. Implementations usually have a setting for how wide that final candidate list is. A wider list is slower and more likely to find the true nearest vector.

A second example: when you do not need oneCopy Anchor Link

Say you have 5,000 chunks of 1,536 numbers. Comparing a query with all of them is 5,000 × 1,536 = 7,680,000 multiplications, which a laptop does in a blink. A plain array and a loop are enough, with no database to install. An index starts to pay off when scanning everything becomes the slow part, so measure before you add one.

What goes wrongCopy Anchor Link

Filtering is the common trap. Some systems find the 10 nearest vectors first and apply the filter afterwards. Ask for 10 results in country = Canada, and if only 2 of the overall top 10 are Canadian, you get 2 results. The symptom is fewer results than you asked for. Confirm it by running the same query without the filter and counting how many of the top 10 pass it.

Other quiet problems: deleted documents that may still appear in some systems until the index is cleaned up, and an index too big for memory, which slows every query. Check how your database handles both before you depend on it.

Measuring the trade-offCopy Anchor Link

Approximate search can miss the true best match. Measure recall: for 100 test queries, compare the index's top 10 with the exact top 10 from a plain scan. If 9.5 of 10 match on average, recall is 95 percent. Then ask whether a lost 5 percent matters for your product. For a help center it usually does not, because the 11th-best passage is rarely needed.

Common mistakeCopy Anchor Link

Choosing by benchmark charts rather than by what your team can operate. Operations cost more than a few milliseconds.

A worked example: sizingCopy Anchor Link

50,000 chunks of 1,536 numbers each, at 4 bytes per number, is about 300 MB of vectors. That fits in memory on a laptop, so no special cluster is needed. At 50 million chunks the same math gives 300 GB, and now index choice, sharding and cost start to matter. Plan for the size you have, not the size you hope for.