KFoldCrossValidation

K-fold cross-validation is a resampling method used to evaluate the performance of a machine learning model on a limited data set. The data are divided into k folds of roughly equal size. For each of the k iterations, one fold is held out as the test set while the model is trained on the remaining k−1 folds. The process is repeated so that every fold serves as the test set exactly once. The performance scores from the k iterations are then aggregated, typically by computing the mean and often the standard deviation, to provide an overall estimate of model generalization.

Common choices for k are 5 or 10, balancing bias and variance in the estimate. In classification tasks with imbalanced class distributions, stratified k-fold ensures that each fold reflects the overall class proportions. For regression, stratification can be approximated by binning the target values. Time-series data require different approaches, since observations are ordered and independent assumptions are violated; standard k-fold can lead to lookahead bias, so methods like forward-chaining or time-series split are preferred.

Applications include estimating model performance, comparing different algorithms, and tuning hyperparameters. Nested cross-validation can be used when both model assessment and hyperparameter selection are needed, avoiding optimistic bias. While k-fold cross-validation reduces bias in performance estimates and utilizes data efficiently, it can be computationally intensive, especially with large models or many hyperparameters. The results depend on how the data are shuffled and partitioned, so fixing a random seed can improve reproducibility.