What is a vector index and how do HNSW and IVF differ?

The outcome you should expect

Once you put a vector index in front of an embedding collection, the practical outcome is that similarity search stops scaling linearly with data size. Without an index, finding the nearest neighbors to a query vector means comparing it against every row — a brute-force scan that is exact but becomes unusable once a collection passes a few hundred thousand items, because latency grows directly with the row count. An index trades a small amount of accuracy for a large amount of speed by narrowing the search to a promising subset of candidates instead of the whole set.

With HNSW, the outcome is consistently low query latency and high recall across a wide range of collection sizes, because the graph structure lets a search converge on the right neighborhood in a small, roughly logarithmic number of hops rather than a linear scan. This is why HNSW is the default choice inside most managed vector databases and libraries — Pinecone, Qdrant, Weaviate, and FAISS all ship an HNSW option, and for many teams it is the index they never have to think about again once it is tuned. The cost of that convenience is memory: the graph stores explicit links between vectors and their neighbors at multiple levels, so RAM usage per vector is meaningfully higher than a comparably sized IVF index.

With IVF, the outcome is a smaller memory footprint and faster index construction, at the cost of recall that degrades more visibly if the number of probed clusters (nprobe) is set too low. Because IVF first assigns every vector to a cluster centroid and only searches inside the closest clusters at query time, it inherently ignores vectors that live near a cluster boundary but were assigned to a neighboring cluster — a structural blind spot that HNSW's graph traversal does not share. Teams that choose IVF are usually optimizing for cost per vector stored or for very large batch jobs where index rebuild time matters more than shaving milliseconds off each query.

The broader outcome worth expecting is that this choice is rarely permanent or exclusive. Many production systems that start with a pure HNSW index later introduce IVF-style partitioning, product quantization, or a hybrid index (IVF combined with a graph, or HNSW combined with compressed vector storage) once the collection grows past tens of millions of vectors and memory becomes the binding constraint rather than latency. Expect your first choice to be revisited as the dataset grows, not because the first choice was wrong, but because the trade-off that matters shifts from "make queries fast" to "keep the whole index in affordable memory."

What drives that outcome

The difference in behavior between HNSW and IVF comes directly from how each structure decides which vectors to compare against a query, and that structural choice cascades into every downstream property — latency, memory, recall, and how the index behaves as data is added or removed.

HNSW is a graph. Each vector becomes a node connected to a handful of its approximate nearest neighbors, and those connections are organized into layers: a sparse top layer with long-range links for coarse navigation, and progressively denser lower layers for fine-grained refinement. A query starts at an entry point in the top layer, greedily moves to whichever connected node is closest to the query, and repeats that descent layer by layer until it lands in the dense bottom layer, where a local search produces the final candidate list. This is why HNSW search cost grows slowly with collection size — the graph's layered structure keeps the number of hops small even as the total vector count rises. It's also why HNSW handles insertions well: a new vector can be spliced into the graph by connecting it to its approximate neighbors without touching the rest of the structure.

IVF is a partition. During training, a clustering step (commonly k-means) picks a fixed number of centroids and every vector in the collection is assigned to whichever centroid it is closest to, forming an inverted list per cluster — conceptually similar to how an inverted index in text search maps a term to the documents containing it, except here the "term" is a region of vector space. At query time, IVF compares the query only to the centroids, picks the nprobe closest clusters, and then does an exhaustive comparison against every vector inside those clusters. This is why IVF's memory footprint is lower: the coarse structure is just a small set of centroids, and the per-vector overhead is minimal compared to storing explicit graph edges. It's also why IVF is more sensitive to the nprobe setting — probing only one or two clusters is fast but risks missing true neighbors that landed in an adjacent cluster; probing more clusters recovers recall but pushes latency back up toward brute-force territory.

The two designs also drive different failure characteristics. HNSW's graph can degrade if built with too few connections per node (a low "M" parameter), producing a sparse graph that gets stuck in local optima and returns suboptimal neighbors. IVF's clustering can degrade if the training sample used to build the centroids doesn't represent the full data distribution, or if the vector distribution shifts after the index is built, leaving stale cluster boundaries that no longer reflect where the data actually sits.

Benchmarks and realistic ranges

Concrete numbers vary heavily by dimensionality, hardware, and the specific library implementation, but a few realistic ranges hold across most production deployments and are worth internalizing so you can sanity-check your own benchmarks rather than trusting a single vendor's marketing figures.

For query latency, HNSW typically returns results in single-digit to low double-digit milliseconds for collections in the low millions of vectors, because the graph traversal cost grows slowly with size. IVF's latency depends heavily on nprobe: with a small number of probed clusters it can be faster than HNSW, but recall suffers noticeably; pushed toward the recall level HNSW delivers by default, IVF's latency often lands in a similar range or higher, because it ends up doing more raw distance comparisons inside the selected clusters. Neither number is fixed — both indexes can be tuned toward either extreme of the latency-recall trade-off, which is exactly why "which is faster" without a stated recall target is not a well-formed question.

For recall, well-tuned HNSW commonly reaches into the high nineties (percent) at reasonable settings, and this recall tends to be more stable across query distributions because the graph doesn't have hard boundaries the way clusters do. IVF recall is more sensitive to configuration: a low nprobe can leave recall in the 70-85% range, while a high nprobe recovers into the low-to-mid nineties, trading away most of the latency advantage that made IVF attractive in the first place. The practical lesson is that IVF requires more active tuning to hit a given recall target, while HNSW's defaults are usually closer to a reasonable operating point out of the box.

For memory, this is where the two indexes diverge most predictably. HNSW stores explicit edges between vectors at multiple graph layers, so memory overhead per vector is meaningfully above the raw vector size — often on the order of 1.5-3x the raw float vector storage once graph metadata is included. IVF's coarse index is just a small number of centroids, so its overhead is close to the raw vector size with a small constant addition, which is why IVF (especially combined with product quantization) is the more common choice when the vector collection needs to fit in a fixed memory budget rather than a fixed latency budget.

For build time, IVF is generally faster to construct because training centroids and assigning vectors to them is a lighter computation than building a multi-layer graph with quality neighbor connections at every level. This matters most for workloads that rebuild the index frequently — nightly batch reindexing of a document store, for example — where IVF's faster build can outweigh its higher query-time tuning burden.

For scale, both indexes are used well past the tens-of-millions mark in production, but the path to get there differs: HNSW scales by accepting more memory per vector, while IVF scales by combining clustering with compression (product quantization) to keep memory bounded, usually at a further recall cost. Distributed vector databases (Milvus, Weaviate, Qdrant) shard either index type across nodes once a single machine's RAM is no longer sufficient, which is the point at which the HNSW-vs-IVF choice starts to matter less than the sharding and replication strategy sitting above it.

Risks, edge cases, and failure modes

The most common operational mistake is picking an index type based on a benchmark from a different dimensionality or dataset shape than the one actually being deployed. Recall and latency figures for 128-dimensional vectors do not transfer cleanly to 1536-dimensional embeddings from a modern text embedding model — higher dimensionality tends to erode the effectiveness of both graph and cluster structures (a consequence of the "curse of dimensionality" reducing the contrast between near and far neighbors), so always benchmark on your actual embedding model's output, not a published number from a paper using a different one.

A second risk is treating the index build as a one-time event. IVF's cluster centroids are trained on a sample of the data at build time; if the underlying data distribution drifts significantly afterward — new topics, a different user base, a changed embedding model — the centroids stop matching the data well, and recall silently degrades without any error being raised. HNSW is more forgiving of drift because new vectors are inserted directly into the graph rather than assigned to a fixed set of pre-trained regions, but a graph that only grows via insertion and never gets rebuilt can accumulate structural imbalance over time, particularly if a large fraction of the original vectors are later deleted.

Deletion itself is an edge case both index types handle awkwardly. Most implementations of HNSW support deletion by marking a node as removed rather than physically restructuring the graph, which means deleted vectors keep consuming memory and can even be traversed as intermediate hops during search until a full rebuild. IVF handles deletion similarly poorly, since removing a vector from a cluster's list doesn't rebalance the cluster or retrain centroids. If your application does frequent deletes — a chat history that ages out, a product catalog with high churn — plan for periodic full rebuilds rather than assuming either index handles churn gracefully forever.

A subtler failure mode is conflating approximate nearest neighbor search with exact search when correctness matters. Both HNSW and IVF are approximate by design — they trade a controlled amount of missed neighbors for speed. For most retrieval-augmented generation and recommendation use cases, a 95-99% recall rate is entirely acceptable because a slightly imperfect top-k list rarely changes the downstream outcome. But for use cases like deduplication, fraud matching, or compliance search where every true match must be found, an approximate index is the wrong tool, and a brute-force exact search (or an exact index like a flat index with GPU acceleration) is worth the extra latency.

Finally, watch for the trap of over-indexing on a single metric during evaluation. A team that tunes purely for recall@10 in an offline benchmark can end up with a configuration (very high nprobe, or a densely connected HNSW graph) that performs well in testing but is too slow or too memory-hungry once real query volume and concurrent load hit the system. Load-test the chosen configuration under realistic concurrency, not just single-query latency, since both index types share memory and CPU resources across simultaneous queries and can degrade non-linearly under load even when single-query numbers look fine.

A practical rollout plan

Moving from "we need a vector index" to a tuned, production-ready deployment is a sequence of decisions, not a single configuration choice, and skipping steps is the most common reason teams end up re-architecting six months in.

Start by establishing the actual constraint you're optimizing for before picking an index. If the workload is a live user-facing feature — search-as-you-type, a chat assistant's retrieval step, a recommendation feed — latency and recall under real-time load are the binding constraints, which points toward HNSW as the default starting point. If the workload is a nightly or periodic batch job — deduplicating a document store, computing similarity for an offline report — build time and memory cost matter more than shaving milliseconds, which points toward IVF, optionally combined with product quantization if the vector count is large enough that raw storage cost is material.

Next, benchmark on your real embedding model's actual output, at your real dimensionality, with a representative sample of your real query distribution — not a generic benchmark dataset. Measure recall@k against a ground-truth exact search on a held-out sample, then measure latency and memory at the settings that hit your target recall, not at default settings. This single step catches most of the mismatches that come from assuming a published benchmark generalizes to a different embedding model.

Then plan for growth and drift explicitly rather than assuming the index configuration is permanent. Decide up front how often the index will be rebuilt (nightly, weekly, or triggered by a drift-detection threshold), how deletions will be handled (soft-delete with periodic compaction is the common pattern for both HNSW and IVF), and at what collection size you expect to need sharding or a managed vector database rather than a single-node library. Bake this into the rollout as a scheduled maintenance task, not a reactive fix after recall complaints show up.

Finally, roll out with a fallback and a monitoring signal. Keep the ability to fall back to exact search on a small subset of traffic (or run it in shadow mode) so you have a live recall baseline to compare against, and monitor query latency percentiles (not just the average) in production, since tail latency is where index misconfiguration shows up first under real concurrent load.

Related questions

What is product quantization and why is it paired with IVF?

Product quantization compresses each vector by splitting it into subvectors and replacing each with a codebook centroid ID, shrinking storage by an order of magnitude or more. It's commonly layered on top of IVF because IVF's cluster structure already narrows the search space, so the added quantization error has a smaller impact on final recall.

Does a vector index replace the need for a traditional database index?

No — a vector index handles similarity search over embeddings, while traditional indexes (B-trees, hash indexes) handle exact-match and range queries over structured fields. Most production systems use both together, often via metadata filtering combined with vector search in the same query.

Can I combine HNSW and IVF in one system?

Yes, some libraries and databases support hybrid structures, such as an IVF partition where each cluster internally uses a small graph, or a global HNSW graph layered over compressed vectors. These hybrids aim to capture HNSW's recall with closer-to-IVF memory usage.

How does embedding dimensionality affect the HNSW vs IVF decision?

Higher-dimensional embeddings (1024+ dimensions, common with modern text models) increase memory pressure for both indexes, but disproportionately for HNSW's graph edges. This pushes some teams toward IVF with quantization once dimensionality and vector count both scale up.

What happens to recall if I resize an IVF index without retraining centroids?

Recall degrades because the centroids were trained on the data distribution at build time; adding a large volume of new, differently-distributed vectors without retraining leaves centroids that no longer represent the true clusters, silently increasing missed neighbors.

FAQ

Is HNSW always better than IVF? No. HNSW generally wins on latency and recall for a given memory budget, but IVF wins on memory efficiency and build speed. The right choice depends on whether your workload is latency-sensitive and RAM-rich, or memory-constrained and tolerant of a slower build cycle.

Do I need to choose one index type forever? No. Many teams start with one index type and migrate or hybridize as scale and constraints change — most commonly moving from a plain HNSW graph toward IVF with quantization once vector count grows large enough that memory becomes the dominant cost.

What is "recall" in the context of a vector index? Recall measures what fraction of the true nearest neighbors (as found by an exact, brute-force search) the approximate index actually returns. Both HNSW and IVF are approximate by design, so recall is always less than 100% unless configured very conservatively.

Why does IVF need a "training" step and HNSW doesn't? IVF's clustering (typically k-means) needs a representative sample of the data to place centroids sensibly, which is a training step. HNSW builds its graph incrementally by connecting each new vector to its approximate neighbors as it's inserted, with no separate training phase.

Can vector indexes handle filtered search, like "find similar items in this category only"? Yes, most modern vector database implementations support combining a vector search with metadata filters, though the mechanics differ — some apply the filter before the vector search narrows candidates, others apply it after, which can affect both recall and latency.

Is a vector index the same thing as a vector database? No. A vector index is the underlying data structure (HNSW, IVF, and variants) that performs the similarity search. A vector database is the surrounding system — storage, metadata filtering, sharding, replication, and an API — that wraps one or more index types for production use.

Sources

• ANN-Benchmarks
• FAISS wiki — Faiss indexes
• hnswlib GitHub repository
• Pinecone: HNSW explained
• Milvus documentation: vector index types
• Qdrant documentation: indexing
• Weaviate documentation: vector indexing
• Microsoft Research: DiskANN

Related on PULSE

• [What is retrieval-augmented generation and when should you use it?](/knowledge/ai0412)
• [How do embedding models turn text into vectors?](/knowledge/ai0405)
• [What is cosine similarity and how is it used in search?](/knowledge/ai0418)
• [How do vector databases handle sharding at scale?](/knowledge/ai0431)
• [What is product quantization and why does it save memory?](/knowledge/ai0427)