Home › Guides › Bulk deduplication

Bulk deduplication in HubSpot: the full method

By Anthony Abreu · Founder of Inspectable · HubSpot Revenue Operations & Sales Hub certified
Updated

Thirty duplicates can be handled by hand. Three hundred cannot: every decision taken one by one, without a rule, ends up being taken at random, and random merges badly. This guide describes the method that holds at scale: measure, define a key, write the decision rules, execute, trace. It extends the complete guide to HubSpot duplicates, which presents the three approaches, by detailing the one that remains when the other two have given way.

The moment the method changes

HubSpot's native tool reviews duplicates pair by pair, for contacts and companies, and its documentation page describes no bulk merge: it is a review tool, not a cleanup tool. The Excel export method holds up to about fifty groups. Beyond that, it is not just the time that explodes: it is consistency. The record kept on Monday by one criterion, on Thursday by another, and nobody can say any more why a given record disappeared. The threshold is not a magic number, it is the moment your decisions stop being reproducible.

Measure before you touch anything

A deduplication project starts with a number, not with a merge. Export the full base with the identity columns (email, first name, last name, company, phone), the Record ID, the owner and the last activity date, then count: how many groups, of what size, with what share of active records. That count decides everything else: ten groups point back to the manual method, three hundred impose this one, and a majority of inactive groups moves the subject towards retention rather than merging.

The inspectability diagnostic: five indicators measured separately, because a single overall score never tells you where to start.
The inspectability diagnostic: five indicators measured separately, because a single overall score never tells you where to start. Measurement comes before method.

The matching key, beyond the email

HubSpot deduplicates on the exact email; a bulk deduplication therefore looks precisely for what the exact email cannot see. The base key remains first name + last name + company, normalised: lowercase, no spaces, no accents. It then gets refined: the normalised phone number matches records with different emails, and the local part of the address ("j.smith" versus "john.smith" on the same domain) catches variants of the same person.

The wider the key, the more it catches, and the more it gets wrong. Two namesakes in the same company look exactly like a duplicate. A wide key is for preselecting candidates, never for deciding: the decision belongs to a rule, or to a human.

Written rules for choosing the record to keep

This is the heart of the method, and what separates it from improvisation: the selection criteria are written down before starting, then applied to every group in the same order. For example: activity history first, presence of an owner next, field completeness after that, and age as the tie breaker. The exact order matters less than its existence: written rules turn three hundred decisions into a single one, taken once, and let you answer later the question "why this one?". The groups no rule can settle are set aside for human arbitration, and that is a result, not a failure.

The central trap: an import never merges

Reimporting a "deduplicated" file merges nothing. A HubSpot import updates the records it matches and creates the ones it does not. Under no circumstances does it merge two existing records: the duplicates survive the import, intact.

It is the most expensive mistake on this subject, because it looks like a solution: you clean the file in Excel, delete the extra rows, reimport, and believe you are done. In HubSpot, the records "deleted from the file" are still there. Merging is a separate operation, done in HubSpot, group by group. The cleaned file is not a command, it is a roadmap: the list of merges to execute, each with the record to keep.

Executing the merges, in order

The execution is prepared like a large import: automations paused on the properties involved, a backup export before starting, and a trial batch of about ten groups to check that the rules produce what you expect. Then the merges proceed, roadmap in sight, starting with the safe groups and keeping the ambiguous ones for the end. Two mechanics to know before clicking, detailed in the guide on what a HubSpot merge keeps: the primary record wins on conflicting values, and HubSpot refuses to merge beyond 250 accumulated merges on the same records.

The trace that protects you

Six months later, someone will ask where a record went. The only good answer is a file: group by group, the record kept, the records merged, the rule applied, the date. It is also what lets you resume the project after an interruption without redoing the sorting. On Inspectable's demonstration base, entirely synthetic, that file documents 36 groups detected across 1,400 contacts: 30 mergeable, documented record by record for validation before any action, and 9 fully inactive groups isolated for arbitration rather than merged by default. The whole method fits in one sentence: no merge without a trace, no trace without a rule.

Sources

Every page below has been opened and read. These are primary sources: the authority that sets the rule, or the software vendor.

  • Use data quality toolsHubSpot Knowledge Base. The native tool reviews duplicates pair by pair, contacts and companies; the page describes no bulk merge.
  • Merge recordsHubSpot Knowledge Base. What a merge keeps, its absence of undo and the 250 accumulated merges limit.

Your roadmap, ready to validate

That is exactly what the Inspectable audit produces: your duplicate groups detected offline from a simple export, the record to keep chosen by explicit rules, and every decision traced in a file you validate before acting. The free mini-audit gives you the starting number within 24h.

Get my free mini-audit

Frequently asked questions

Does reimporting a deduplicated file merge the records?

No, and that is the central trap of the subject: an import updates or creates, it never merges. The duplicates survive the reimport, intact. The cleaned file serves as the roadmap for the merges, which are done in HubSpot.

Can the whole deduplication be automated?

Detection and choosing the record to keep, yes, with explicit rules. The decision on ambiguous groups, no: two namesakes in the same company look exactly like a duplicate. Those groups are set aside for human arbitration.

What takes the most time?

Not the merges: the decisions. Every group requires choosing the record to keep, and it is the number of decisions that sizes the project. Hence the value of written rules, which settle the vast majority in advance.

Should inactive duplicates be merged?

No. Merging two dead records manufactures a clean dead record. On the demonstration base, 9 groups out of 36 were fully inactive: they belong to a retention decision, keep or delete, not to a merge.