What is a feature store and do you still need one for LLM apps?

What it is and why it matters

A feature store sits between raw data sources and the model that consumes them. Its job is to take signals scattered across a data warehouse, an event stream, and application databases, transform them into a consistent numeric or vector representation, and make that representation available in two places at once: an offline store used for training and batch analysis, and an online store used for millisecond-latency lookups during live inference. The core promise is "train-serve consistency" — the feature a model saw during training should be computed the exact same way when that model (or an LLM calling out to retrieve context) asks for it in production.

For classic machine learning, this problem was well understood: a fraud model needs "transactions in the last 10 minutes" to match between backtesting and the live scoring path, or the model silently degrades. LLM applications inherit the same problem in a different shape. When an LLM app pulls in a user's account tier, their last three support tickets, or a freshly embedded chunk of a knowledge base to ground an answer, that is a feature-serving problem even though nobody on the team calls it that. The store still needs to guarantee freshness (how old is this context allowed to be), consistency (does the retrieval step see the same value the prompt-construction step sees), and low latency (fetching context can't add seconds to a chat response).

The reason this still matters in the LLM era, rather than being made obsolete by vector databases, is that most production LLM apps mix two feature types: dense embeddings (for semantic retrieval) and structured features (account status, plan tier, recency counters, click history). A pure vector database is excellent at the first and weak at the second — it was not built for time-windowed aggregations, point-in-time correctness, or governance over who can read which feature. Whether you need the full discipline of a dedicated feature store, or can get by with a lighter combination of a cache and a vector index, depends on how many structured signals your app actually uses and how strict your consistency requirements are.

The step-by-step process

The mechanics of getting a feature into an LLM prompt at inference time follow a consistent pipeline, whether the underlying tool is a dedicated feature store, a vector database, or a hand-rolled cache.

1. Ingest. Raw events (page views, transactions, support tickets, document uploads) land in a warehouse or stream.
2. Transform. Feature definitions turn raw rows into usable signals: a rolling count, a normalized score, or a text chunk that gets embedded into a vector.
3. Materialize. The transformed value is written to two destinations — an offline table for training and evaluation, and a low-latency online store (a key-value or vector index) for serving.
4. Serve. At inference time, the application sends a lookup key (a user ID, a session ID, a query) and receives back the current feature value or the nearest-neighbor matches within single-digit to low-double-digit milliseconds.
5. Assemble. The retrieved features and/or retrieved document chunks are inserted into the prompt template or the tool-call payload sent to the LLM.
6. Log and monitor. The exact features that were served get logged alongside the model's output, so you can debug a bad answer by reconstructing exactly what the model saw.

The step most teams skip is the last one, and it's the one that saves the most debugging time later — without it, "why did the model say that" becomes unanswerable after the fact.

Costs, timelines, and typical ranges

Cost and setup time scale with how much of this pipeline you build versus buy. A team wiring a cache-plus-vector-index pattern together themselves (an in-memory store for structured features, a vector index for embeddings) can typically get a working prototype live within a sprint or two, because both pieces are individually simple; the cost is almost entirely the compute and storage for the underlying infrastructure, which is modest at small data volumes and grows with request volume and stored vector count.

Adopting a dedicated, self-hosted open-source feature store adds real setup time — usually measured in weeks rather than days — because you're standing up a serving layer, a metadata/registry service, and pipelines to keep the offline and online stores synchronized. The ongoing cost is primarily the infrastructure you already run (a low-latency key-value store, compute for materialization jobs) plus the engineering time to operate it: on-call for a serving-layer outage, monitoring for stale features, and periodic schema migrations as your feature set grows.

Adopting a managed, vendor-hosted feature store shifts most of that operational burden to the vendor. Setup time is generally the fastest of the three options because there's no cluster to provision, but you pay for that convenience through usage-based pricing (typically billed per read/write request or per unit of provisioned throughput) that scales with traffic. For a chat application with a modest number of concurrent users, this is often the cheapest path in engineering hours even if the line-item bill is higher than a self-hosted deployment. For an application serving millions of requests a day, the calculus can flip — enough volume makes a self-hosted, fixed-infrastructure-cost approach cheaper on a per-request basis, even after accounting for the team needed to run it.

A rough rule of thumb across all three paths: budget more time for the pipeline that keeps offline and online features consistent than for the serving layer itself. Serving a feature quickly is a solved problem; making sure the feature that gets served matches the one you tested against almost never is, and that mismatch is where most timelines slip.

Where teams get it wrong

The single most common mistake is treating a vector database as a full substitute for a feature store on day one, then discovering the gap only once the app has structured features that don't fit a vector schema — a subscription tier, a rate limit counter, a support-escalation flag. Retrofitting a second storage system after the app already has real users is far more disruptive than deciding up front whether you need one or two systems.

A second recurring failure is duplicate computation: teams re-embed the same document chunk or re-compute the same user profile vector on every request instead of caching it, because nothing in their stack enforces reuse. This inflates both latency and inference-adjacent cost (embedding calls are not free) for no benefit, since the underlying text hasn't changed.

A third is silent feature drift — an embedding model gets upgraded, or a transformation script changes, and old vectors sitting in the store are now on a different footing than newly written ones. Without versioning on the feature definition itself, similarity search quietly gets worse over time and nobody notices until relevance complaints show up. The fix is to version every feature definition (not just the code that computes it) and to re-backfill rather than mixing vector generations in the same index.

A fourth mistake is ignoring point-in-time correctness: pulling in "today's" user data to evaluate a change against historical logs, rather than the data that would have existed when the historical request was actually made. This inflates offline evaluation scores because the "model" is effectively cheating with future information, and the gap shows up as a nasty surprise the moment the feature ships to real traffic.

Finally, teams frequently underestimate latency budgets. A feature lookup that takes 50–100ms is invisible in an offline batch job and very visible in a live chat response, where every added round trip stacks on top of the LLM's own generation time. Testing feature-serving latency against a synthetic benchmark rather than the actual end-to-end request path (including network hops between the app server, the store, and the model endpoint) is a common way this gets missed until production.

Decision framework: when to choose what

Whether you still need a feature store — and if so, which shape of one — comes down to three questions asked in order: does the app use retrieval or personalization at all, does it need real-time (not batch) freshness, and how many distinct structured feature types does it manage. An app with zero personalization and no external knowledge lookup can usually skip this layer entirely and call the LLM directly. An app that only needs embeddings for retrieval, with no time-windowed aggregations or governance requirements, can often get by with a vector database alone. An app that combines embeddings with structured, frequently changing signals — and needs those two things to stay consistent with each other — is the case where a dedicated feature store, whether self-hosted or managed, earns its complexity.

Related questions

Is a vector database the same thing as a feature store?

No. A vector database specializes in storing and searching embeddings by similarity. A feature store manages a broader set of feature types — structured, time-windowed, and vector — with versioning, offline/online consistency, and governance across both.

Do I need a feature store for a simple chatbot with no memory?

Usually not. If the app has no user-specific context, no retrieval step, and no personalization, it can call the LLM directly with a static or lightly parameterized prompt.

How is a feature store different from a data warehouse?

A warehouse is optimized for large batch analytical queries; a feature store adds a low-latency online-serving layer plus guarantees that the value served at inference time matches what was used in training or evaluation.

Can I add a feature store after launching without one?

Yes, but it's disruptive once real traffic exists — you'll need to backfill historical features for evaluation and migrate live lookups without breaking the running app, so deciding early is cheaper than retrofitting later.

What's the difference between an online and an offline feature store?

The offline store holds historical feature values for training and backtesting at high volume with relaxed latency; the online store holds current values optimized for low-latency point lookups during live inference.

FAQ

What exactly does a feature store add on top of a vector database? Feature versioning, point-in-time correctness for training data, support for non-vector structured features like counters and flags, and monitoring for drift — capabilities a vector index alone typically doesn't provide.

Does every RAG application need a feature store? No. A RAG app that only retrieves document chunks by similarity, with no structured personalization signals, can run on a vector database by itself. The need increases as structured, frequently updated context gets layered on top.

What latency should I target for feature serving in a chat app? Aim for single-digit to low-double-digit milliseconds for anything fetched synchronously in the request path, since it stacks directly on top of the model's own generation time and is felt by the user.

Is a self-hosted or managed feature store better for a small team? A managed option generally gets a small team to production faster with less operational burden, at the cost of usage-based pricing; a self-hosted option gives more control and can be cheaper at high, steady volume once someone is available to operate it.

Can a simple cache plus a vector index replace a full feature store? For a small number of feature types and modest governance needs, yes — this hybrid is a common starting point. It breaks down as the number of data sources, feature types, and consistency requirements grows.

What's the biggest risk of skipping a feature store entirely? Feature drift and duplicate computation: embeddings or structured signals quietly become inconsistent across model or code updates, and the same values get recomputed repeatedly instead of cached, which degrades both quality and cost over time.

Sources

• Feast documentation
• Tecton feature platform overview
• Hopsworks feature store documentation
• Databricks Feature Store documentation
• Amazon SageMaker Feature Store documentation
• Vertex AI Feature Store documentation
• Redis vector search documentation
• Qdrant documentation
• Pinecone documentation
• Weaviate documentation

Related on PULSE

• [How does retrieval-augmented generation actually work under the hood?](/knowledge/ai0201)
• [What's the difference between a vector database and a traditional database?](/knowledge/ai0212)
• [How do you keep an LLM app's knowledge base up to date?](/knowledge/ai0223)
• [What does MLOps mean for teams shipping LLM-powered products?](/knowledge/ai0234)
• [How do you debug a bad answer from a RAG pipeline?](/knowledge/ai0219)