KFold

K-fold, short for k-fold cross-validation, is a resampling method used to assess how a predictive model will generalize to an independent dataset. It partitions the available data into k roughly equal-sized folds. The model is trained on k−1 folds and evaluated on the remaining fold. This process repeats until every fold has served as the validation set. The performance metrics from each iteration are combined, typically by computing the mean (and sometimes the standard deviation) to estimate generalization performance.

Common choices for k are 5 or 10, which offer a balance between bias and variance in the performance estimate. Leave-one-out cross-validation is a special case where k equals the number of samples; while it uses nearly all data for training in each iteration, it can be computationally demanding and may yield high-variance estimates for some tasks. Stratified k-fold preserves the class distribution in each fold for classification problems, improving reliability on imbalanced datasets. Grouped k-fold prevents leakage by ensuring that samples from the same group (such as all measurements from a single subject) are not split across training and validation sets.

Practical considerations include performing data preprocessing (scaling, encoding, feature selection) within each training set to avoid information leakage from the validation data. Nested cross-validation is often used for hyperparameter tuning, with an inner loop for tuning and an outer loop for performance estimation. Overall, k-fold cross-validation provides a more robust and data-efficient estimate of model performance than a single train-test split, especially when data are limited.