FellegiSuntermodellen

The FellegiSunter model, also known as the Fellegi-Sunter method or the record linkage model, is a statistical framework for identifying matching records in two or more files that refer to the same entity. Developed by Ivan Fellegi and Philip Sunter in 1969, it is widely used in data management, statistics, and various fields for tasks like deduplication, data integration, and privacy protection.

The core idea of the model is to compare pairs of records, one from each file, and assign a probability of agreement or disagreement for each field within the records. These probabilities are then combined using a weighted sum to produce a matching score. Records are classified as matches, non-matches, or possible matches based on this score and predefined thresholds.

The model's strength lies in its probabilistic approach, which allows for uncertainty in the matching process. It accounts for both agreement and disagreement patterns across fields, considering the relative importance of each field through weights. These weights can be estimated empirically or set based on expert knowledge.

Key components of the Fellegi-Sunter model include the comparison vector, which represents the agreement/disagreement patterns for a record pair, and the matching weights. The model explicitly addresses the issue of clerical review by distinguishing between definite matches and potential matches that require further investigation. Despite its statistical rigor, the model's performance is sensitive to data quality and the accuracy of the underlying probability estimates.