instancebased

Instancebased, often written as instance-based learning, is a family of machine learning methods that postpone generalization until a query is made. Instead of building a parametric model from training data, these methods retain the training instances and use them directly to make predictions.

During prediction, a distance or similarity measure is computed between the new instance and each stored example. The most common approach is to identify the k nearest neighbors and base the prediction on their labels: for classification, by majority vote or a weighted vote that favors closer neighbors; for regression, by averaging the neighbors’ target values, often with distance-based weights. Because model construction is minimal, learning is fast, but prediction can be slow and memory-intensive, especially with large datasets.

Key characteristic is its laziness and flexibility. Instance-based methods make few assumptions about the form of the decision boundary, enabling them to capture complex patterns. However, they rely on representative, clean data and effective feature scaling. They also suffer from the curse of dimensionality: distance measures become less informative as the number of features grows, and performance can degrade with noisy or irrelevant features.

Common variants include k-nearest neighbors (k-NN), which is the standard example, and distance-weighted or locally weighted versions. Case-based reasoning and memory-based reasoning are related approaches used in AI. To improve efficiency, indexing structures such as KD-trees or ball trees and approximate nearest-neighbor search are often employed.

See also: k-NN, lazy learning, case-based reasoning, distance metric learning.