What is a model registry and why does it matter for governance?

What a model registry actually is, and why governance hinges on it

Strip away the tooling and a model registry is a catalog with three obligations. First, it names things: a model has a stable identity ("churn-scorer") that outlives any single artifact, and versions accumulate under that name rather than as model_final_v3_REAL.pkl on someone's laptop. Second, it binds metadata to each version: the training run that produced it, the dataset snapshot, the code commit, the evaluation metrics, the person who registered it. Third, it tracks state: which version is in staging, which is serving live traffic, which has been archived or rolled back.

That third obligation is where governance lives. Without a registry, "deploying a model" is a file copy — an act with no approver, no record, and no way to answer the question an auditor or an incident reviewer will eventually ask: *which exact model made this decision on this date, and who authorized it?* With a registry, the promotion from one stage to the next is an API call that can be permissioned, gated on automated checks, logged immutably, and reversed. The artifact does not change. The accountability around it does.

The distinction worth internalizing: experiment tracking and model registry solve adjacent but different problems. Experiment tracking is a lab notebook — it logs hundreds or thousands of runs, most of which are dead ends, and its value is comparison and reproducibility during development. A registry is the customs checkpoint — it handles the small subset of models that are candidates for production, and its value is control. MLflow bundles both, which is convenient and occasionally confusing, because teams assume that logging a run means the model is governed. It does not. A run in the tracking store is a record; a version in the registry is a commitment.

Governance pressure has been increasing from three directions simultaneously, which is why registries stopped being an MLOps nicety. Financial-services model risk management guidance — the Federal Reserve's SR 11-7 framing, which predates the modern ML stack by over a decade — already required institutions to maintain model inventories, document assumptions, and validate independently of the developers. The EU AI Act extends record-keeping and technical-documentation obligations to a much broader set of systems classified as high risk. And the NIST AI Risk Management Framework, while voluntary, has become the de facto vocabulary US enterprises use when a customer's security questionnaire asks how they manage AI risk. All three assume something that can only be answered by a registry: you know what models you have.

There is also a mundane, non-regulatory reason that convinces engineering leaders faster than any framework does. When a model starts behaving badly at 2 a.m., the first question is "what changed?" A registry answers it in thirty seconds — version 14 was promoted at 18:40, here is its diff against version 13, here is the rollback. Without one, that question takes hours of archaeology across Slack, S3 buckets, and someone's memory.

The step-by-step promotion process

The mechanics matter more than the tool. A registry that nobody routes through is a decorative database. The workable shape is a pipeline where registration is automatic and promotion is deliberate.

Step one: register automatically at the end of training. The training job itself calls the registry API and creates a new version. This should never be a human action, because humans skip it under deadline pressure. Attach everything at registration time: dataset identifier or hash, git commit SHA, container image digest, hyperparameters, and the full evaluation report. Metadata attached later is metadata that is missing when you need it.

Step two: run the automated gate. Before a version can be proposed for staging, it must clear machine checks — held-out accuracy above a floor, no regression beyond a tolerance against the current production model, latency within budget on representative payloads, fairness or subgroup-performance metrics computed and recorded, and a schema check confirming the input contract has not silently drifted. The critical design decision here is that these checks *write their results back to the registry as version metadata*. A check that runs in CI and vanishes into a build log is not governance evidence.

Step three: shadow or challenger evaluation. The candidate serves alongside production without affecting decisions, and you compare predictions on live traffic for a defined window — typically a few days to two weeks, long enough to cover a full business cycle. Offline evaluation on a static test set has a well-known failure mode: the test set is a snapshot of a world that has moved on. Shadow mode catches the gap between the two.

Step four: human approval. Someone accountable — a model owner, a risk reviewer, or both for high-impact models — reviews the evidence and approves the transition. The approval is recorded with an identity and a timestamp. For anything credit-, hiring-, or safety-adjacent, this is often the point where an independent validator (not the model's author) has to sign, which is exactly what model risk guidance has demanded of banks for years.

Step five: promote and deploy. The serving layer reads the registry, not a hardcoded artifact path. This is the single most-skipped step and the one that quietly destroys the value of everything above it. If your inference service loads s3://models/prod/model.pkl and a separate process happens to also update the registry, then the registry is documentation, not control. Serving must resolve "the version currently aliased as production" through the registry at deploy time.

Step six: monitor, and close the loop back. Drift detectors, performance monitors, and incident reports should write back against the *version identity* in the registry. That way the history of a model version includes not just how it was born but how it behaved.

Two refinements worth adding once the basic loop works. Use aliases rather than hard stage names — pointing a mutable label like champion at an immutable version number is cleaner than mutating a version's stage field, because the alias movement itself becomes the auditable event and rollback is a single pointer change. And treat deprecation as a first-class transition, not an afterthought: models that are no longer served but whose past decisions are still being contested need to stay retrievable, not deleted.

Costs, timelines, and what implementation actually takes

The license cost of a registry is rarely the real number. Open-source options — MLflow's registry, DVC's Git-backed approach, Kubeflow's ML Metadata — carry no per-seat fee, and the managed registries built into the major clouds (SageMaker, Vertex AI, Azure ML) are priced as a small component of platform usage rather than as standalone products. Vendor-hosted platforms like Weights & Biases, Neptune, and Comet sell per-seat plans with free tiers that are generally adequate for evaluation. Prices move, so check current pricing pages rather than trusting any figure quoted in an article; the structural point is that no mainstream registry is expensive enough for licensing to be the deciding factor.

What costs real money is everything around it. Budget along four lines.

Engineering integration. Wiring the training pipeline to auto-register, building the automated gate, and — the big one — changing the serving layer to resolve versions from the registry. For a team with one or two models already in production, this is typically a few engineer-weeks. For an organization with dozens of models deployed through heterogeneous paths, it is a multi-quarter program, because the work is not the registry; it is standardizing the deployment paths that feed it.

Operations, if self-hosting. A self-hosted registry needs a backing database, artifact storage, backups tested by actual restore, authentication wired to your identity provider, and upgrades. That is a real fraction of a platform engineer's ongoing time. The cloud-managed registries exist precisely to convert that into a line item.

Storage. Usually trivial and occasionally not. Gradient-boosted models and classical scikit-learn artifacts are megabytes; you will never notice them. Large transformer checkpoints are gigabytes each, and a team that registers every checkpoint from every run will discover a storage bill. The fix is policy, not infrastructure: register candidates, not every epoch, and set retention rules that keep production-served versions indefinitely while aging out rejected candidates.

Process time. Approval gates add latency by design. A well-run gate adds hours to a day for routine retrains; a poorly-run one adds two weeks because the only approver is on vacation and there is no delegate. Track this number — time from registration to production — as an explicit metric. If it climbs, the governance layer is failing at its second job, which is to make safe things fast, not just to make unsafe things hard.

For sequencing, a realistic phased plan looks like this. Weeks 1–2: stand up the registry, register existing production models retroactively so the inventory is complete, and accept that the metadata for old models will be incomplete. Weeks 3–6: wire auto-registration into training pipelines and build the automated gate for one high-value model as a pattern. Weeks 7–12: convert serving to resolve from the registry, which is where you will find the surprises. Quarter 2 onward: extend to the long tail of models, add approval requirements calibrated by risk tier, and connect monitoring write-back.

The single highest-return early action is the retroactive inventory. Most organizations discover, at this step, that they have more models in production than anyone believed — including forgotten batch scoring jobs and a model someone deployed before leaving the company. That discovery alone frequently justifies the project.

Where teams get it wrong

Registering models the serving layer does not read. This is the dominant failure and it deserves repeating because it is so easy to ship. The registry fills up, dashboards look healthy, and production continues loading artifacts by path. The tell is simple: ask whether rolling back a model requires a code change or a registry change. If it requires a code deploy, the registry is not in the control path.

Treating approval as a rubber stamp. A gate that always passes is a gate that teaches everyone the gate is noise. If your approval step has never blocked a promotion, either your models have been uniformly excellent or nobody is actually reading the evidence. Define what would cause a rejection *before* you need to reject something, and make the rejection path as smooth as the approval path so that saying no is not socially expensive.

Uniform governance for wildly non-uniform risk. Applying the same three-approver workflow to a credit-decisioning model and an internal ticket-routing classifier does two kinds of damage: it under-governs the first by making review routine, and it over-governs the second until teams route around the registry entirely. Tier your models. A common cut is three levels — models affecting individuals' access to credit, employment, housing, or health get independent validation; models with material revenue impact get owner approval plus automated gates; internal-efficiency models get automated gates and registration only.

Capturing lineage that does not actually reproduce. Recording "trained on customer data" is not lineage. Lineage means a specific, immutable reference: a dataset version, a snapshot identifier, a query plus an as-of timestamp against a warehouse that supports time travel. The test is whether an engineer who has never seen the model can regenerate its training set from the metadata alone. Most teams fail this test on their first attempt, and discovering that during a routine drill is dramatically better than discovering it during a regulatory exam.

Forgetting that the feature pipeline is part of the model. A registry that versions the model artifact but not the transformation code that produces its inputs is versioning half the system. Training-serving skew — where the feature computed offline differs subtly from the one computed online — remains one of the most common production failure modes, and it is invisible if only the model binary is registered. This is why feature stores and registries increasingly reference each other, and why teams without a feature store should at minimum pin the transformation code's commit into the model version's metadata.

Ignoring third-party and foundation models. The inventory obligation does not stop at models you trained. If a product feature calls a hosted LLM, that dependency has a version, a provider, a set of prompts, and behavior that can change under you. Many governance programs have a blind spot exactly here, having built the registry around a training-pipeline mental model. Register the external dependency, its version pin, and the prompt or configuration as an artifact — the auditor's question is about the decision system, not about who trained the weights.

Deleting the past. Retention policies written for cost optimization will happily delete a model version that is the subject of a dispute two years later. Set retention around legal and regulatory exposure, not storage bills, and keep the metadata even when you age out the binary.

A decision framework for choosing a registry

Most of this decision is made for you by facts you already know, which is why the evaluation should take days, not months.

Start with where inference runs. If your models are deployed through a single cloud's ML platform, that cloud's registry is the default and the burden of proof is on the alternative — it inherits your identity provider, your audit logging, and your network policy for free, and those integrations are the expensive part of any governance implementation. Fighting that gravity to get a slightly nicer UI is a bad trade.

Then ask whether you are multi-cloud or hybrid. A cloud-native registry becomes a liability the moment half your inference happens elsewhere. Here a vendor-neutral option — self-hosted MLflow, a Git-based approach, or a hosted platform that spans clouds — earns its operational overhead.

Then ask how your organization already approves changes. If everything meaningful at your company flows through pull requests, a Git-backed registry means governance uses machinery your engineers already trust and your auditors already accept. If approvals live in a ticketing system with non-engineer reviewers, a registry with a real UI and webhook integrations will fit better; asking a compliance officer to approve a merge request is a losing proposition.

Then check the regulatory floor. Independent validation, retained evidence, SSO with enforced roles, immutable audit logs, and data residency are either present or they are not. This filter eliminates options quickly and unsentimentally.

Finally, weigh team capacity. A self-hosted registry is free in licensing and not free in attention. A three-person team without platform engineering support should take the managed option almost every time.

One caution on this framework: the cost of switching registries is low compared to the cost of switching *deployment patterns*. The metadata is exportable and the artifacts are portable. What is not portable is the habit of resolving production models through the registry rather than by path. Build that habit first, on whatever tool is nearest, and treat the specific vendor as a decision you are allowed to revisit.

How registries connect to the wider governance stack

A registry is one component and it becomes far more useful when the neighbors are in place.

Feature stores solve the input half of reproducibility. They give features versioned definitions and, critically, guarantee that the online and offline computations agree. A registry that references feature-set versions closes the training-serving skew gap that the model artifact alone cannot.

Data catalogs and lineage tools extend the chain upstream. The registry knows a model trained on dataset X; the catalog knows dataset X derives from three source tables, one of which is being deprecated next quarter. Joined together, you can answer the impact question in reverse: *which models are affected if this table changes?* That query is what turns lineage from documentation into an operational capability.

Monitoring supplies the after-the-fact half of the record. Drift and performance signals keyed to a model version turn the registry entry into a living history rather than a birth certificate.

Model cards are the human-readable projection of registry metadata — intended use, known limitations, evaluation across relevant subgroups, and contact ownership. The version that helps is the one generated from registry data rather than written by hand, because a hand-written card is accurate exactly once.

It is worth noting that this pattern is not unique to machine learning. Container registries solved the same problem for application images: immutable, addressable versions with signed provenance and a promotion path between environments. Artifact repositories did it for libraries. Terraform state and module registries did it for infrastructure. Machine learning arrived at the answer later and with more difficulty, largely because model behavior depends on data that is harder to pin than a package version. The organizational lesson transfers cleanly, though — every one of those ecosystems learned that governance sticks only when the deploy path physically cannot bypass it.

The generative-AI wave has pushed this further rather than making it obsolete. Teams now need to register prompt versions, retrieval configurations, fine-tuned adapters, and evaluation suites alongside model weights, because those components change behavior as much as weights do. The vocabulary shifts; the underlying requirement — know what is running, know how it got there, be able to put it back — does not.

Related questions

How is a model registry different from a feature store?

A registry versions and governs models; a feature store versions and serves the inputs models consume. They solve complementary halves of reproducibility. A registry alone cannot prevent training-serving skew, because the skew originates in feature computation. Mature stacks reference feature-set versions from within model-version metadata.

Do we need a registry if we only have two models?

Yes, though a lightweight one. The value at small scale is less about approval workflow and more about answering "what changed?" during an incident and having lineage when a customer or auditor asks. Two models becomes six faster than expected, and retrofitting governance costs more than starting with it.

Can a registry satisfy EU AI Act or model risk management requirements by itself?

No. A registry supplies the technical evidence — inventory, versioning, lineage, evaluation records, approval logs — that those frameworks require. It does not supply the policies, risk classification, independent validation function, or human oversight procedures. It is necessary infrastructure for compliance, not compliance itself.

What should trigger an automatic rollback?

Define triggers before deployment, not during an incident. Common ones: prediction-distribution drift beyond a threshold, error rate or latency breaching SLO, a sharp drop in a downstream business metric attributable to the model, or a failed scheduled fairness re-check. Rollback should be an alias change, executable in seconds.

How do we govern third-party and foundation models we did not train?

Register them as versioned dependencies. Capture the provider, the pinned model version, prompt and configuration artifacts, and your own evaluation results against your use case. Provider-side changes are outside your control, which makes recording what you were running when — and evaluating on a schedule — more important, not less.

FAQ

What is a model registry?

A model registry is a centralized system that catalogs machine learning models, tracks every version, binds each version to the data, code, and metrics that produced it, and manages lifecycle state such as staging, production, and archived. It is the authoritative inventory of what models exist and which are live.

Why does a model registry matter for governance?

Because governance requires answering three questions reliably: what models are running, how did each get approved, and can we reverse a bad one. A registry makes promotion a permissioned, logged, reversible event instead of an untracked file copy, which is the prerequisite for audit, incident response, and regulatory evidence.

Does using a registry slow down model deployment?

It can, and the amount is a design choice. Automated gates add minutes; human approval gates add hours to days depending on reviewer availability. Well-implemented registries usually make deployment *faster* on net, because rollback becomes trivial and teams stop treating each release as a high-stakes event. Measure registration-to-production time and treat regressions in it as a defect.

What is the difference between a model registry and experiment tracking?

Experiment tracking logs every training run during development, including the many that fail — it optimizes for comparison and reproducibility. A registry handles the small subset of models that are production candidates and optimizes for control: versioning, approval, and lifecycle state. Some platforms bundle both, but logging a run does not mean a model is governed.

Should we build a registry or adopt an existing one?

Adopt. The problem is well-solved by open-source and cloud-native options, and a homegrown registry means owning schema evolution, access control, audit logging, and API stability forever. Custom build only makes sense when regulatory constraints genuinely rule out every available option, which is rarer than teams initially assume.

How long does implementation take?

Standing up a registry and back-filling an inventory of existing production models takes one to two weeks. Wiring auto-registration and automated gates for a first model takes another month. Converting serving to resolve versions from the registry — the step that makes it governance rather than documentation — is where timelines vary most, from weeks to a quarter depending on how many deployment paths exist.

Sources

• MLflow Model Registry documentation
• Amazon SageMaker Model Registry
• Google Cloud Vertex AI Model Registry
• Azure Machine Learning model management concepts
• NIST AI Risk Management Framework
• Federal Reserve SR 11-7: Guidance on Model Risk Management
• EU Artificial Intelligence Act — official text portal
• Model Cards for Model Reporting (Mitchell et al., arXiv)
• Kubeflow ML Metadata documentation
• Hugging Face Hub model documentation

Related on PULSE

• [How do you build data pipelines for continuous model training?](/knowledge/ai403)
• [What is model serving and how is it different from a REST API?](/knowledge/ai381)
• [How do you handle model rollbacks safely in production?](/knowledge/ai429)
• [The 10 Best AI Model Monitoring Tools in 2027](/knowledge/ai346)
• [What is confidential computing and why does it matter for AI?](/knowledge/ai413)