KnearestneighborsKlassifikation

K-nearest neighbors (KNN) is a simple, instance-based supervised learning algorithm used for classification and regression. For a new observation, KNN identifies the k training samples closest to it in feature space and assigns a label (classification) or a value (regression) based on those neighbors.

The method requires choosing a number k and a distance metric, commonly Euclidean, Manhattan, or Minkowski. To predict, distances between the query point and all training samples are computed, the k nearest are selected, and a decision is made: in classification, by majority vote (or weighted voting); in regression, by averaging the neighbor responses (often with weights inversely proportional to distance).

KNN is lazy and instance-based, meaning it defers computation until prediction and stores the full training set. There is no explicit training phase other than possibly normalizing features. Predictions can be slow on large datasets, and performance depends on feature scaling and the relevance of features; irrelevant or noisy features can degrade accuracy.

Practical considerations include feature scaling, choosing k via cross-validation, and selecting a distance metric. High dimensionality (the curse of dimensionality) diminishes the usefulness of distance. KNN can be enhanced with weighted distances, approximate nearest neighbor search, or combining with dimensionality reduction. It remains a common baseline method due to its simplicity and interpretability.