Duplicate customer record merge

Customer records accumulate by addition, and once a real person has three records in the warehouse, they stay there. The first record was created when the customer signed up; the second appeared when a third-party enrichment landed an email variant that did not match; the third came from a checkout that was a guest checkout the team forgot to stitch later. Each is wrong but each is also working: the records receive their own communications, hold their own segment memberships, and contribute to their own analytics. The cost is real (three contact-list entries, three campaign sends, three conversion records that do not add up to one real customer) but distributed thinly enough that nobody escalates it until the duplicate rate compounds into a meeting.

The merge is the structural response: identify the duplicates, link them under a canonical identity, reconcile the consent and segment state, and record the decision so a wrong merge can be undone. The matching rules are the discipline: a permissive rule merges records that should have stayed separate (different people sharing an address), a strict rule misses merges that should have happened. The reversibility is the safety net: a merge in production is not reversible by default, and a recipe shipped without an explicit rollback path produces wrong merges that the team cannot undo.

This recipe builds the matching, the merge, and the reversibility as one system.

A measurable reduction in duplicate customer records, with the savings showing up in lower contact-list size, fewer campaign sends per real customer, and analytics that finally agrees on the customer count. The metric is the duplicate record reduction rate: the share of records the merge identified as duplicates and successfully consolidated, against a baseline of estimated duplicates measured before the merge ran. The value shows up in spend (fewer messages to the same real person across the duplicate records), in customer experience (the customer does not receive the same campaign three times because they exist as three records), and in operational discipline (the team has a working answer to "how many customers do we have").

The second outcome is the matching-rule precision: the share of merges the audit confirmed as correct, against the share that were reversed. Both should be measured. A high reversal rate means the rule is too permissive; an unmeasured reversal rate means the team does not know whether the merges are doing damage.

A matching pipeline that runs the rules over the candidate-pair space (or a smarter sub-space chosen by blocking heuristics so the team is not comparing every record to every other record), produces the confidence scores, and routes high-confidence pairs to automatic merge and ambiguous pairs to a manual review queue. Composable stacks suit this because the matching lives in the warehouse and the merge writes to the identity graph; packaged stacks may have a vendor-provided merge with rules the team cannot inspect, and the question is whether the vendor's precision is auditable against the team's standard. The capability that decides whether the recipe is safe is the rollback path: a merge that cannot be undone is a merge the team cannot ship, and the reversibility is the engineering work that takes the recipe from research project to production system.

Work in this order. Every merge has to be reversible before the first one runs.

1. Model the merge as a graph edge. Duplicates become edges to a canonical identity with the source records preserved, so a rollback deletes the edge and re-separates any post-merge data. A merge that deletes a source record makes a wrong merge permanent.
2. Write the consent reconciliation rule before the first merge. The stricter consent wins: a record that opted out stays opted out after the merge.
3. Validate matching rules against held-out data. Partial name plus shared address collapses two members of one household into one record.
4. Tune the precision threshold to the stakes. A financial-services merge needs a different target than a low-stakes ecommerce one.
5. Log every merge with the rule that fired and the reversal path.
6. Monitor merge rate, false-merge rate and rule precision, so drifting rules show up as an alert rather than a wave of bad merges.

Identifier graph write covers steps 1 and 2, anonymous-to-known stitching steps 3 and 4, audit trail generation at the decision point step 5, and signal quality monitoring step 6.