How do you handle data deduplication when merging two companies' CRMs in 2027?

The Tuesday morning both pipelines showed the same deal

A mid-market software company closes an acquisition on a Friday. The acquired company runs Salesforce with eleven years of accumulated history; the acquirer runs HubSpot with four years and a much cleaner schema. Leadership wants one system by end of quarter, and the RevOps lead is told the integration team will "just run a dedupe job."

The following Tuesday, after a naive import, the combined pipeline report shows $94M. Finance says the real number is closer to $71M. The gap is not fraud and it is not a bug — it is three overlapping categories of duplication that a single dedupe pass never separates.

The first category is intra-source duplication. The acquired org had eleven years of unmanaged lead capture: the same account exists as "Acme Corp," "ACME Corporation," "Acme Corp." (trailing period), and "Acme" with a subsidiary's address. Roughly 8–15% of accounts in a decade-old CRM with no dedupe governance are internal duplicates. That mess has to be resolved *inside* each system before either system is compared to the other, because cross-system matching against a dirty source multiplies the ambiguity — four bad records on one side times two on the other is eight candidate pairs to adjudicate instead of one.

The second category is cross-source overlap — genuine shared customers. Two companies selling into adjacent markets typically share 10–30% of their account base; the same customer bought from both. These are the records that inflate pipeline, because the same buying entity now carries two open opportunities that a naive rollup adds together. Some of that is real (two separate product lines, two separate deals) and some is a single deal double-counted by two teams who both think they own it.

The third category is entity-hierarchy collapse. "Acme Corp" in one CRM is the global parent; "Acme Manufacturing — Ohio" in the other is a division that buys independently. Those are not duplicates. Merging them destroys territory assignment, breaks commission attribution, and makes the account team's book of business unrecognizable. Over-merging is more expensive than under-merging, and it is much harder to reverse.

The practitioner's move is to name these three problems separately and give each its own pass, its own success metric, and its own approval gate. A single "dedupe the CRM" work item guarantees all three get conflated, and the conflation is what produces the $23M phantom.

How the matching engine actually decides two records are one

Deduplication in a CRM merge is not one algorithm — it is a cascade. Each rung is cheaper and more certain than the one below it, and every record that a higher rung resolves is removed from the candidate pool before the expensive rungs run.

Rung one: deterministic keys. Exact match on a normalized, high-cardinality identifier. For accounts in 2027 the practical hierarchy is: tax identifier (EIN, VAT number, company registration number) → DUNS or equivalent firmographic ID → normalized web domain → normalized phone. Domain normalization means stripping protocol, www, trailing slash, and lowercasing — https://WWW.Acme.com/ and acme.com are the same key. Deterministic matching typically resolves 55–75% of true account pairs at essentially zero false-positive rate, and it should be the only rung allowed to auto-merge without review.

Rung two: standardization, then blocking. Before any fuzzy comparison, run both sides through the same normalization: uppercase, strip legal suffixes (Inc, LLC, Ltd, GmbH, Pty, S.A.), collapse whitespace and punctuation, expand or strip common abbreviations, and parse addresses to a postal standard. Then *block* — partition candidates into buckets that share a cheap key (first four characters of the normalized name, or postal code, or domain root) so the fuzzy comparison runs within buckets instead of across the full Cartesian product. Without blocking, 200,000 records against 300,000 is 60 billion comparisons. With blocking on a well-chosen key, it is a few million.

Rung three: probabilistic scoring. Inside each block, compute per-field similarity — Jaro-Winkler or Levenshtein on names, token-set ratio for word-order differences, geographic distance on parsed addresses — and combine them into a weighted score. Weight by discriminating power: a matching rare surname is worth far more than a matching city. Then apply two thresholds, not one. Above the high threshold (commonly ~0.90 on a 0–1 scale), auto-merge. Below the low threshold (~0.70), auto-reject. Between them, route to a human review queue. That middle band is where the real work is, and it is normally 3–8% of the candidate pairs.

Rung four: relationship evidence. When name and address are ambiguous, contact overlap breaks the tie. If two account records share three or more contacts with the same email domain and similar titles, they are almost certainly one entity. Conversely, if two similarly-named accounts share zero contacts and sit 900 miles apart, they are probably distinct divisions.

The output of the cascade is not a merged database. It is a match decision table: source A ID, source B ID, decision, confidence, rung that decided it, timestamp, and reviewer if human. That table is the artifact you keep. It is what lets you unwind a bad merge six weeks later, and it is what an auditor asks for when the revenue numbers move.

Survivorship: deciding which value wins, field by field

The most common design error is treating a merge as "pick the surviving record." Real merges pick a surviving *value per field*, because neither source is uniformly better.

Write survivorship rules as an explicit table before you touch data. A workable default for two merging companies:

• Most-recently-modified wins for volatile fields: phone, address, employee count, job title, current owner. Recency is a decent proxy for truth on anything that changes.
• Source-of-truth wins for fields where one system is authoritatively better. If the acquirer's billing system feeds its CRM, the acquirer's billing address always wins regardless of recency. Name the winning source per field, not globally.
• Longest non-null wins for free-text description fields, with a length cap so a pasted email thread does not become the account description.
• Never overwrite non-null with null. This single rule prevents the most common silent data loss in CRM merges — a sparse record on one side blanking out a populated field on the other.
• Sum or union, never pick for additive and set-valued data: activity history, notes, attachments, campaign membership, product interest. These append; they never survive-or-die.

Opportunities need their own rules because they carry revenue. Two open opportunities on the same merged account are only duplicates if they represent the same buying decision — same product family, overlapping close dates, comparable amount. When they are duplicates, keep the one with the more advanced stage and the more recent activity, and move the loser's activity history onto the survivor rather than deleting it. When they are distinct, leave both and let the account rollup show two deals. Getting this wrong in either direction is what breaks the forecast, so opportunity dedupe should always be a human-reviewed queue, never an automated threshold.

Contacts have a subtlety worth naming: email is a strong deterministic key, but people change jobs. The same email on two records is a near-certain match; the same *person* at a different email is a different record with a personal-identity link, not a merge. And a shared generic address — info@, sales@, support@ — is not a key at all. Put those on a stoplist before the deterministic rung runs, or you will merge four hundred unrelated contacts into one.

Ownership and territory assignment deserve a named decision rather than a rule. When two sellers each own a version of the same account, the merge tool will pick one and the other seller will find out from a dashboard. Route those to a sales-leadership queue with the compensation implication visible, and communicate reassignments before the merge lands, not after.

Real numbers: what these projects actually cost and how long they take

Concrete planning ranges, for two mid-market CRMs in the low-hundred-thousands of records each:

Duplicate rates. Intra-source duplication in an ungoverned CRM commonly runs 8–15% of accounts and 10–25% of contacts; contacts are worse because lead capture forms have no gate. A CRM with active dedupe governance sits closer to 2–5%. Cross-source account overlap between two companies in adjacent markets is typically 10–30%; in the same market segment it can exceed 40%.

Match performance. Deterministic keys resolve 55–75% of true pairs. Probabilistic scoring on top of that pushes recall into the high 80s or low 90s. The manual review band — pairs between the two thresholds — is normally 3–8% of candidates, and a trained reviewer clears roughly 200–400 pairs per day with a good side-by-side UI, less if they have to check contact history for each one.

Timeline. Profiling and rule design: 2–4 weeks. Intra-source cleanup on each side: 3–6 weeks, running in parallel. Cross-source matching, review, and staged merge: 6–12 weeks. Post-merge reconciliation and stabilization: 4 weeks. A realistic end-to-end range for two mid-market CRMs is four to seven months. Compressing below three months means either skipping review of the ambiguous band or skipping reconciliation, and both show up later as revenue reporting you cannot defend.

Effort. Budget one dedicated RevOps or data lead full-time for the duration, plus 0.5–1 FTE of review capacity during the review window, plus part-time engineering for the extract and load. Tooling — an MDM or dedicated identity-resolution platform versus building on the CRM's native merge APIs — is a real fork: native tooling is cheaper and slower and caps out around a few hundred thousand records; dedicated tooling costs more and earns it above roughly half a million records or when you need ongoing match-as-a-service after cutover.

Quality targets to hold yourself to. Precision on auto-merged pairs should be ≥ 99% — an incorrect auto-merge is far costlier than a missed one, because it destroys data that a human then has to reconstruct. Recall in the 90–95% range is acceptable; the residual duplicates get caught by ongoing governance. Post-merge, pipeline total should reconcile to finance within 1–2%, and any variance above that gets explained line by line before cutover is declared complete.

Waves. Never merge everything at once. A workable sequence: wave one is a 500–2,000 record pilot on a low-risk segment (closed-lost, or a non-core geography) to validate the rules end to end. Wave two is accounts with no open pipeline. Wave three is accounts with open pipeline, coordinated with the sellers who own them. Wave four is the long tail and the manual-review residue. Each wave gets its own reconciliation before the next begins, and a wave that fails reconciliation stops the sequence.

Trade-offs: what you give up with each approach

There is no configuration that is simultaneously fast, cheap, and safe. The four real choices:

Aggressive auto-merge versus conservative review. A high auto-merge rate finishes in weeks and produces false merges you will discover for the next two years — usually when a seller says a customer's history vanished. A conservative posture produces a review queue that can run into tens of thousands of pairs and a timeline measured in quarters. The asymmetry matters: an unmerged duplicate is visible, annoying, and fixable in thirty seconds. A wrongly merged pair is invisible, corrupts history, and may be unrecoverable once the audit window closes. Bias conservative and pay for review capacity.

Merge-into-one versus federate-then-migrate. Consolidating both orgs into a single CRM instance gives one pipeline, one set of reports, and one admin surface — at the cost of a hard cutover and a long freeze. Federating — keeping both systems live, syncing a shared identity layer, and reporting from a warehouse on top — lets both sales teams keep working without disruption and defers the hard decisions. Federation is the right call when the acquisition thesis keeps the businesses operationally separate, or when one side's sales motion is genuinely different. It is the wrong call when leadership wants one funnel by next quarter, because federation quietly becomes permanent and you end up maintaining two schemas plus a sync layer forever.

Native CRM tooling versus dedicated identity resolution. Native merge and dedupe features are already licensed, already permissioned, and already audited. They are also usually limited to exact or near-exact matching, cap the number of records per merge operation, and give you little control over survivorship. Dedicated tooling gives real probabilistic matching, configurable survivorship, and a reviewable match table — and adds procurement, integration, and a new system to run. The crossover is roughly at half a million total records, or earlier if you need ongoing match-as-a-service rather than a one-time project.

Clean before migration versus clean after. Cleaning in the source systems means working in a familiar environment with live users who can validate — but you are cleaning a system you are about to retire, which feels wasteful and often gets deprioritized. Cleaning in a staging layer between extract and load is faster and reversible, but the validators are further from the data and the cleaned records have never been seen by the people who know them. The practical answer is split: intra-source duplicates get cleaned in place, where the account owners can confirm; cross-source matching happens in staging, where you can iterate without touching production.

Pitfalls that turn a merge into a two-year cleanup

Merging without a freeze. If both systems stay writable during extract and match, records change underneath you and the match table goes stale. Freeze creation of new accounts and contacts during the merge window, or run a delta capture that replays post-freeze changes onto the merged records afterward. Skipping this produces records that exist in the target but not the match table — orphans nobody can explain.

Treating parent-child hierarchies as duplicates. Subsidiaries, divisions, franchise locations, and international entities look like duplicates to a name matcher and are not. Before matching, load whatever hierarchy data exists on both sides and mark known parent-child pairs as *linked, never merge*. Franchises are the classic trap: two hundred locations with near-identical names and different addresses, each a separate buying entity with a separate contract.

Losing activity history. Emails, calls, meetings, and notes are the record of the relationship, and they are what sellers actually mourn. Every merge must append the loser's activities to the survivor. Verify this on the pilot wave by counting activities before and after — the total must not drop.

No rollback path. Many CRM merge operations are irreversible in the platform. Before any wave, snapshot both source systems and store the match table with enough detail to reconstruct the pre-merge state. "We can restore from backup" is not a rollback plan if the backup predates three weeks of legitimate new work.

Ignoring integrations downstream. Marketing automation, CPQ, billing, support, and the data warehouse all hold their own copies of CRM IDs. When two accounts merge, one ID dies, and every downstream system holding that dead ID silently breaks — orphaned billing records, campaign members pointing at nothing, warehouse joins dropping rows. Inventory every system that stores a CRM ID before wave one, and publish a merged-ID mapping table that each of them can consume.

Consent and privacy carried over blindly. Marketing consent, do-not-contact flags, and regional privacy status are not survivorship fields you resolve by recency. The restrictive value always wins: if either record says do-not-contact, the merged record says do-not-contact. Under GDPR and similar regimes, consent obtained by one legal entity does not automatically transfer to another on acquisition, so the merged marketable universe is often smaller than the sum of its parts. Get legal to sign off on the consent survivorship rule specifically, in writing, before the first wave.

Declaring victory at cutover. Duplicates regenerate. Without ongoing governance — real-time duplicate blocking on record creation, required domain on account create, a weekly duplicate report, and a named owner — a freshly merged CRM is back to 5% duplication within a year. Budget for the governance layer as part of the project, not as a follow-on someone will fund later.

Not communicating with sellers. The people whose accounts are being merged are the best validators you have and the loudest critics if surprised. Show the affected reps their merge candidates before the wave runs, give them a mechanism to flag "these are not the same," and tell them when their book will change. A merge that is technically perfect and socially ambushing still fails.

Related questions

How long should the CRM freeze last during a merge?

Ideally 48–72 hours per wave, not the whole project. Freeze only the objects being merged in that wave, keep the rest writable, and use delta capture to replay changes made during the window rather than extending the freeze.

Should we deduplicate leads before or after they convert?

Before. Deduplicating at the lead stage is cheaper because leads carry less history and no revenue. Once a lead converts to account-contact-opportunity, the merge touches pipeline and requires seller coordination.

What if the two companies use different CRM platforms entirely?

The matching logic is platform-agnostic — it runs on extracts, not on live objects. The platform difference adds schema mapping and picklist reconciliation work upstream, and it usually pushes you toward a staging layer rather than native merge tooling.

Who signs off that the merged pipeline number is correct?

Finance, not RevOps. Reconcile merged pipeline and closed-won against the finance system of record before declaring cutover complete, and get an explicit written sign-off on the variance.

FAQ

How do you handle data deduplication when merging two companies' CRMs in 2027? Run it as a staged program, not a job. Profile both systems, clean intra-source duplicates in place, then match cross-source using deterministic keys first and probabilistic scoring second, with a human review band in the middle. Apply field-level survivorship rules agreed in advance, merge in waves with reconciliation between each, and stand up duplicate-prevention governance before you declare cutover.

What is the single biggest driver of how hard this will be? Data hygiene in the older system. A CRM with a decade of ungoverned lead capture and no required fields will have several times the duplicate rate and far worse key coverage than a governed one, and that gap drives review volume, timeline, and cost more than record count does.

Can AI do the matching in 2027? Machine-learned and embedding-based matching genuinely helps in the ambiguous middle band — it is better than hand-tuned string distance at catching name variants, transliterations, and rebrands. It does not remove the need for deterministic keys on the high-confidence rung, and it does not remove human review, because a model's confidence score is not an audit trail. Use it to rank and pre-sort the review queue, not to auto-merge.

How do we decide which CRM survives? Choose on schema quality, integration surface, and admin capacity rather than on which company acquired the other. The system with cleaner objects, fewer custom fields carrying business logic, and fewer brittle downstream integrations is cheaper to migrate into, regardless of whose logo is on it.

What do we do with the records that stay in the review queue at the end? Leave them unmerged and flagged. An unresolved duplicate is a known, visible, low-cost problem; a wrongly resolved one is a hidden, expensive one. Hand the residue to the governance owner as a standing backlog and work it down after cutover with the sellers who own those accounts.

How do we prove the merge did not lose revenue data? Reconcile three totals before and after every wave: open pipeline by stage, closed-won by period, and total activity count. Each must tie within a pre-agreed tolerance — typically 1–2% for pipeline and zero for activity count, since activities append rather than merge.

Sources

• https://www.salesforce.com/products/data-cloud/
• https://help.salesforce.com/s/articleView?id=sf.managing_duplicates_overview.htm
• https://knowledge.hubspot.com/records/manage-duplicate-records
• https://learn.microsoft.com/en-us/dynamics365/customerengagement/on-premises/admin/detect-duplicate-data
• https://gdpr.eu/
• https://www.dnb.com/resources/what-is-a-duns-number.html
• https://www.gartner.com/en/information-technology/glossary/master-data-management-mdm
• https://www.nist.gov/programs-projects/data-science-evaluation
• https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/lawful-basis/consent/

Related on PULSE

• [Top 10 Go-Fast Boats 2027](/knowledge/bt451)
• [Top 10 Boats for Lake Erie 2027](/knowledge/bt450)
• [Top 10 Boats with Cabins 2027](/knowledge/bt449)
• [Top 10 Boat Brands for Saltwater 2027](/knowledge/bt448)