KNNImputation

KNNImputation, short for k-nearest neighbors imputation, is a method for filling in missing values in a dataset by leveraging the values of similar observations. It is non-parametric and relies on the idea that observations with similar patterns across other features are likely to share similar values for the missing feature.

The typical procedure: for each instance with a missing entry in a given feature, compute distances to all other instances using the features that are observed for both; select the k closest neighbors; impute the missing value as the mean (or a weighted mean by inverse distance) of those neighbors' values for that feature. The process can be performed feature by feature or jointly, and some implementations allow weighting neighbors by distance.

Considerations: It benefits from proper feature scaling, can handle numerical features, and tends to preserve local structure of the data better than simple global imputations. It can be integrated into data processing pipelines and used after standardization. The choice of k and the distance metric influence results; larger k smooths more; smaller k preserves local variation but risks higher variance. Computational cost grows with dataset size.

Limitations: It may be inefficient on very large datasets; its assumptions may be violated if missingness depends on the unobserved value (MNAR); it can blur distributions, overimpute when near-constant features, and is sensitive to irrelevant features. It is less effective for categorical variables unless converted to numeric encodings; other methods such as MICE or matrix factorization may be preferred in some contexts.