How do we test a deduplication rule before running it on the whole database?
Run it in report-only mode first. Have it output every merge it would perform without writing anything, then hand-check a random sample of proposed merges for precision and hand-search a sample of known duplicates for recall. Tune, regenerate the report, and only enable writes once precision holds at your automatic threshold on a fresh sample. Then roll out in waves, keeping the merge log.
Report mode is the whole test
A dedupe rule is a piece of software, and like any piece of software it should produce output you can inspect before it produces side effects. The report is a simple list: record A, record B, match score, which signals fired, and the field-level values each side would contribute.
Reading fifty rows of that list teaches you more about your data than any dashboard. You will find the property manager numbers, the shared family emails, and the technician who types the office address into every job.
Precision and recall, without the jargon
Precision answers: of the merges it wants to make, how many are actually the same customer? You measure it by sampling proposed merges and checking them.
Recall answers: of the duplicates that really exist, how many did it find? You measure it by taking a set of duplicates you already know about, or by manually searching a slice of the database, and seeing how many the rule caught. Precision protects the database; recall determines whether the exercise was worth doing. You will not get both to a hundred percent, and a rule with high precision and mediocre recall is the correct starting point.
Sample properly or you will fool yourself
The instinct is to review the top of the list, which is all exact phone and address matches and all correct. That tells you nothing. Stratify: sample from each score band, including the band just above your proposed automatic threshold, because that band is where the first false merges appear.
Also sample by record age and by entry path. Records created by web forms fail differently than records created by technicians on a phone in a driveway, and a rule that works on one may be wrong on the other.
Roll out in waves
Do not merge two hundred thousand records in one job. Run the highest-confidence band first, let it settle for a week, and check whether anything downstream broke: reporting counts, marketing audiences, open jobs, sync loops into other systems. Then take the next band.
Staged rollout with a reversible log is the same discipline that keeps any integration project from turning into an incident, and it is why custom work on live customer data should always ship behind a report before it ships behind a write.
Topics: deduplication · testing · data hygiene · rollout
Have a version of this question about your own business?
The useful answer usually depends on which systems you run and how they're connected. That's a conversation, not a blog post.