overflatematching

Overflatematching is a term used in record linkage and data integration to describe the erroneous practice of identifying two distinct records as the same entity. It occurs when similarity criteria are too broad, attributes lack sufficient distinguishing power, or thresholds are set too leniently. The result is an inflated set of duplicates or merged records that actually refer to different entities.

Causes of overflatematching include relying on non-unique or common attributes (such as gender and approximate birth year), using weak identifiers, or employing overly permissive matching rules. Aggressive blocking or indexing strategies that pair many records together, combined with high tolerance for attribute differences, can also foster overmatching. Data quality issues, such as misspellings, inconsistent formatting, and incomplete fields, further increase the risk.

Consequences include distorted counts of individuals or entities, biased analytics, degraded data quality, and incorrect histories or contact lists. In healthcare, such errors can lead to duplicated patient records and misattributed treatments; in customer databases, they can inflate contact lists or misrepresent relationships.

Mitigation strategies emphasize precision over recall. Approaches include probabilistic matching with calibrated thresholds, using multiple independent attributes, and incorporating clerical review for borderline matches. Validation against labeled gold-standard data, reporting of precision and recall, and restricting matching to a well-defined blocking strategy help reduce overmatching. Maintaining audit trails and conducting periodic linkage quality assessments are also important.

See also: record linkage, deduplication, probabilistic matching, data quality.