How do you choose a vector database for a production RAG system in 2027?

The outcome you should expect

Teams that run this decision properly end up with something unglamorous: a retrieval layer nobody thinks about. That is the actual target. A production RAG system fails on retrieval far more often than it fails on generation, and the failure is rarely "the database went down." It is a slow drift where recall degrades after a re-index, where a metadata filter silently narrows the candidate pool to twelve documents, where the p99 balloons at 9am because everyone opens the internal assistant at once, or where the monthly bill triples after somebody backfills three years of archived tickets.

The realistic outcome of a good selection is that your retrieval step contributes a small, predictable slice of end-to-end latency — typically the smallest slice. In most RAG pipelines the embedding call for the user's query and the generation call from the LLM dominate wall-clock time. Vector search itself, on a well-configured index, is usually tens of milliseconds. If your retrieval is taking longer than your generation, something is misconfigured, over-filtered, or badly sharded, and switching vendors will not fix it.

The second outcome is cost you can forecast. Vector storage cost scales with dimension count times vector count times replication factor, plus whatever the index structure adds on top. That is arithmetic, not magic. A team that has done the selection work can answer "what happens to our bill if the corpus doubles?" in one sentence. A team that has not will discover the answer in a billing alert.

The third outcome is optionality. The embedding model you choose in a given quarter will not be the one you use two years later — the field moves too fast. Your database choice should not make a re-embed a company-wide project. That means favoring engines where you can build a second collection alongside the first, dual-write, shadow-read, compare, and cut over. Any system where re-embedding means downtime is a system you will resent.

What you should *not* expect is a dramatic quality jump from switching engines. Approximate nearest neighbor implementations across mature systems converge to similar recall at similar latency budgets, because they mostly implement variations of the same handful of algorithms. The differentiators are operational: filtering semantics, multi-tenancy, hybrid search maturity, backup and restore, and how the thing behaves at 3am. Chasing benchmark deltas between vendors while your chunking strategy is unexamined is optimizing the wrong layer by an order of magnitude.

What drives that outcome

Five variables drive nearly every vector database decision, and they interact in ways that make single-metric comparisons useless.

Corpus size and vector dimension. This is the primary fork. A million 768-dimension float32 vectors is roughly 3 GB of raw vector data before index overhead; the same count at 1536 dimensions is about 6 GB. Graph indexes like HNSW add meaningful overhead on top — the neighbor lists are not free — and traditionally want to live in RAM for good latency. Multiply by dimension, count, and replicas and you have your infrastructure floor. This is why dimension reduction matters more than vendor choice: cutting from 1536 to 768 dimensions via a model that supports truncated embeddings roughly halves your memory bill, and quantization can cut it much further.

Filter complexity. Pure nearest-neighbor search is the easy case. Real RAG queries look like "find similar chunks, but only from documents this user's role can see, published after a date, in these three product lines." How an engine handles that determines whether it works for you at all. Pre-filtering restricts the candidate set before the search, which is exact but can be slow if the filter is unselective. Post-filtering searches first and discards, which is fast but can return far fewer results than requested when the filter is highly selective. Filtered-graph approaches try to walk the index while respecting predicates. If your access-control model is complex, evaluate this dimension before anything else — it eliminates more candidates than latency ever will.

Write pattern. A static corpus re-indexed nightly is a fundamentally different problem than a system ingesting documents continuously with a freshness requirement measured in seconds. Some engines handle streaming upserts gracefully; others degrade as segments accumulate until compaction runs. Ask specifically: what happens to query latency during compaction or index rebuild, and can I delete a document and have it disappear from results immediately?

Hosting and data-residency constraints. This is often decided for you. Regulated environments, air-gapped deployments, and contractual data-residency requirements rule out managed services regardless of their technical merits. Conversely, a four-person team with no platform engineer should not be operating a distributed database cluster, whatever the licensing savings look like on a spreadsheet.

Ecosystem fit. Every serious engine has integrations for the common orchestration frameworks. What varies is depth — whether the integration exposes the filtering and hybrid-search features you actually need, or just the basic upsert-and-query surface. Check the specific client library against the specific feature, not the marketing page.

Benchmarks and realistic ranges

Public benchmarks are directionally useful and specifically misleading. They run on standardized datasets with uniform distributions, no metadata filters, no multi-tenancy, and a query pattern that looks nothing like yours. Use them to rule things out, never to rule things in.

Here are the ranges worth calibrating against, stated loosely enough to be honest.

Latency. On a properly sized graph index with the working set in memory, single-digit to low-double-digit millisecond query times are normal for corpora in the low millions. Tens of milliseconds is normal into the tens of millions with sharding. When you see hundreds of milliseconds, the usual culprits are: the index doesn't fit in RAM and you're hitting disk, your filter is forcing a near-exhaustive scan, you're requesting a very large top-k, or you're measuring network round-trip from a different region rather than the search itself. That last one catches people constantly — a managed service in a different cloud region than your application adds real time before any search happens.

Recall. ANN indexes trade recall for speed via tunable parameters — the graph construction parameters and the search-time candidate list size in HNSW-style indexes, or the number of probes in cluster-based indexes. Higher search-time effort means better recall and worse latency, monotonically. The right operating point is application-specific: a legal-discovery system and a support-chat assistant have different tolerances for a missed document. Measure recall against exact brute-force search on a sample of your own data. Ten thousand vectors and a thousand queries is enough to see the curve, and you can run brute force on that scale trivially.

Cost. Managed vector services generally price on some combination of stored vector volume, read units, and write units, with the storage component dominating for large static corpora and the query component dominating for high-traffic small ones. Self-hosting converts that into instance cost — you're renting RAM, mostly — plus the very real cost of the engineer who maintains it. The honest comparison includes that engineer. A rule that holds up: below a few million vectors, managed pricing is usually noise in your budget and not worth optimizing; above that, run the arithmetic seriously, because the delta compounds monthly.

Quantization economics. This is where the biggest wins live and where the least attention goes. Scalar quantization to int8 cuts memory roughly fourfold with modest recall loss on most embedding models. Binary quantization is far more aggressive and, combined with a rescoring pass over full-precision vectors for the top candidates, can preserve most of the quality at a fraction of the memory. If your infrastructure bill is the constraint, evaluate quantization before evaluating vendors — it frequently changes which tier of solution you need.

Adjacent workloads worth budgeting for. RAG retrieval is rarely the only vector workload a company ends up with. Semantic deduplication of an incoming document stream, near-duplicate detection in support tickets, recommendation and "related items" surfaces, clustering for analytics, and image or multimodal search all use the same substrate. If you can see two or three of those coming, pick an engine that handles multiple collections and mixed workloads cleanly rather than one tuned narrowly for a single RAG index. The consolidation savings are real, and so is the reduction in systems your team has to understand.

Risks, edge cases, and failure modes

The filter-collapse failure. A query with a highly selective metadata filter can return far fewer results than requested, or wildly worse ones, depending on the engine's filtering strategy. The symptom is a RAG answer that says "I don't have information about that" for a document you can see in the database. Test explicitly with your most selective realistic filter and confirm the result count matches what an exact query would return.

The re-index blackout. Some setups require a full rebuild after certain configuration changes, and during that rebuild queries either fail or degrade badly. Ask before you commit: which operations trigger a rebuild, how long does it take at my scale, and what is the query behavior during it? Then plan for blue-green collections so the answer stops mattering.

Silent recall regression. You change the search-effort parameter to shave latency, latency improves, everyone is happy, and retrieval quality quietly drops. Nothing alerts, because nothing is monitored. The fix is a small golden set — fifty to a few hundred query-document pairs you know the right answers to — run on a schedule against production. This is the single highest-value piece of RAG monitoring and almost nobody builds it.

Embedding-model drift. Vectors from different models are not comparable. Mixing them in one index produces retrieval that looks like it works and doesn't. Every collection should record which model and which version produced it, and a re-embed should build a new collection rather than mutating the old one in place.

Multi-tenancy leakage. If a namespace or partition is your only isolation boundary, one missing filter parameter in one code path exposes one customer's documents to another. This is the highest-severity failure on this list. Enforce tenant scoping at a layer the application cannot forget to apply, and test it adversarially.

Chunking as the hidden variable. Retrieval quality is dominated by how you split documents, not by which engine searches them. Chunks too small lose context; too large dilute the embedding. Overlap helps continuity and inflates storage. Teams routinely spend weeks benchmarking databases while running a naive fixed-size splitter that is costing them far more quality than any vendor difference. Fix the chunking first.

The single-vector-per-document trap. Embedding a long document as one vector averages away everything specific in it. For documents of any length, chunk and embed at the chunk level, then aggregate at retrieval time. Related: hybrid retrieval that combines lexical matching with vector similarity consistently outperforms either alone on real corpora, particularly for queries containing product names, error codes, acronyms, and other tokens where exact matching matters and semantic similarity is nearly useless.

Cost surprises from dimension creep. Upgrading to a larger embedding model with more dimensions multiplies storage, memory, and often latency. Model many of these upgrades as infrastructure decisions, not model decisions, and check whether the model supports dimension truncation before assuming you need the full width.

A practical rollout plan

Treat this as a two-to-four week evaluation, not an afternoon of reading comparison posts.

Week one — define the workload numerically. Write down expected vector count at launch and at twelve months, embedding dimension, queries per second at peak and at median, the top-k you'll retrieve, your filter predicates in their most complex realistic form, your write pattern and freshness requirement, and your hard constraints on hosting and residency. Most vendor conversations collapse to three candidates once this document exists.

Week one, second half — assemble the evaluation set. Pull a representative sample of your real corpus, ideally ten thousand to a hundred thousand chunks including the messy ones: tables, code blocks, scanned-PDF text, multi-language content. Then write fifty to two hundred real queries with the passages that should be retrieved for each. This ground truth is the most valuable artifact of the whole exercise and it outlives every vendor decision you make.

Week two — measure the floor. Run exact brute-force search on your sample to establish maximum achievable recall. This is your ceiling and it also tells you something important: if brute force at full recall gives disappointing answers, your problem is chunking or embedding, not the database. Stop the vendor evaluation and go fix that.

Week two through three — bake off the shortlist. Load the same data into each candidate with equivalent settings. Measure recall against your ground truth, p50/p95/p99 latency under concurrent load resembling production, latency with your real filters applied, ingest throughput, and behavior during a bulk update. Do this from the network location your application will actually run in.

Week three — probe the operational surface. Kill a node. Restore from a backup and time it. Run a schema or index change while queries are in flight. Push past your quota and see what the error looks like. Read the client library source for the retry and timeout defaults, because they will be wrong for you. These tests find more disqualifying problems than the performance tests do.

Week four — pilot behind a flag. Deploy the winner for a slice of real traffic with the golden-set evaluation running on a schedule and dashboards on retrieval latency, result counts, empty-result rate, and cost. Keep the old path warm. Cut over when the golden set is stable for a week.

One more thing worth planning explicitly: the exit. Write your retrieval code behind a thin interface with upsert, query, and delete, and keep the source of truth for chunks and their metadata in your own primary datastore. The vector index should be a derived artifact you can rebuild from scratch. Teams that do this can migrate in days. Teams that treat the vector store as the system of record cannot migrate at all, and they end up negotiating from a position of no leverage.

Related questions

Do I need a dedicated vector database at all?

Often no. If your corpus is small and you already run PostgreSQL, the pgvector extension handles vector search alongside your relational data, which makes complex metadata filtering a normal SQL join. Add a specialized engine when scale, query volume, or filtering behavior demonstrably breaks that setup.

How much does chunking strategy matter compared to database choice?

Considerably more. Chunk boundaries determine what can be retrieved at all; the database only determines how fast you find it. Test two or three chunking strategies against a ground-truth query set before you shortlist any vendor — the quality delta is usually larger by a wide margin.

Should I use hybrid search or pure vector search?

Hybrid, in most production settings. Lexical matching catches exact tokens — SKUs, error codes, function names, proper nouns — where semantic similarity performs poorly. Combining the two ranked lists via reciprocal rank fusion is straightforward and reliably improves retrieval on real, messy corpora.

What happens when I change embedding models?

Every vector must be regenerated; old and new vectors are not comparable and cannot coexist in one index. Plan for a parallel collection, a dual-write period, a shadow-read comparison against your golden set, and a clean cutover. Budget the full re-embedding cost before choosing a larger model.

How do I monitor retrieval quality in production?

Maintain a golden set of queries with known-correct passages and run it on a schedule against live infrastructure, alerting on recall drops. Also track empty-result rate, average result count after filtering, and p99 latency — those three catch most silent regressions early.

FAQ

How many vectors before I should leave Postgres?

There is no hard line, but the pressure usually shows up somewhere in the low millions, and it shows up as latency under concurrency rather than as a hard failure. The practical trigger is when your index no longer fits comfortably in your database instance's memory and you find yourself sizing that instance around vector search rather than around your transactional workload. At that point you are running a vector database that happens to also serve your application tables, which is the worst of both worlds.

Is managed or self-hosted the better choice?

It depends almost entirely on whether you have platform engineering capacity, not on cost. Managed services trade money for operational load. If nobody on your team wants to own index tuning, capacity planning, backup verification, and upgrade windows, self-hosting will cost more than it saves — the savings show up on the infrastructure line and the expense shows up in engineer-hours that never get counted. Teams with an existing platform function and regulated data usually land on self-hosted for good reasons.

Does GPU acceleration help RAG retrieval?

Rarely at typical RAG scale. GPU-accelerated index construction meaningfully speeds up building large indexes, and GPU search matters for very high-throughput or billion-scale workloads. For a corpus of a few million chunks serving human-paced queries, CPU search with a well-tuned graph index is fast enough that GPUs add cost and operational complexity without changing the user experience.

How should I handle documents that update frequently?

Key your vectors to a stable chunk identifier derived from the source document, then upsert by that key rather than deleting and re-adding. Verify your engine's delete semantics carefully — some mark deletions as tombstones that persist in results until compaction, which produces stale retrieval that is genuinely hard to debug. Test the delete-then-immediately-query path explicitly.

What top-k should I retrieve?

Retrieve more than you send to the model, then rerank. A common pattern is pulling twenty to fifty candidates from the vector index, passing them through a cross-encoder reranker, and sending the top three to eight to the language model. Reranking recovers quality that approximate search loses and costs far less than widening the context window, which degrades answer quality once it gets large.

Can one database serve RAG, recommendations, and semantic search together?

Usually yes, and it is often the right call. These workloads share the same primitive. Use separate collections with separate index configurations rather than separate systems. Verify that mixed read patterns don't cause noisy-neighbor interference at your scale, and keep an eye on whether one workload's write volume is degrading another's query latency.

Sources

• pgvector — open-source vector similarity search for Postgres
• ANN-Benchmarks — standardized approximate nearest neighbor benchmarks
• Faiss documentation — index types and guidelines for choosing one
• Qdrant documentation — filtering, quantization, and deployment
• Weaviate documentation — hybrid search and multi-tenancy concepts
• Milvus documentation — index types and scaling architecture
• Pinecone documentation — indexes, namespaces, and metadata filtering
• Elasticsearch kNN search documentation
• Original HNSW paper — Malkov & Yashunin, hierarchical navigable small world graphs
• BEIR — heterogeneous benchmark for zero-shot information retrieval

Related on PULSE

• [The 10 Best AI Tools for Shopping Cart Development in 2027](/knowledge/ai0245)
• [The 10 Best AI Tools for Favicon and Icon Design in 2027](/knowledge/ai0253)
• [The 10 Best AI Tools for UI Mockups in 2027](/knowledge/ai0251)
• [The 10 Best AI Tools for Landing Page Design in 2027](/knowledge/ai0248)
• [The 10 Best AI Tools for Product Page Design in 2027](/knowledge/ai0244)