Metafeature

Metafeature, in machine learning and data mining, refers to a feature that describes properties of a dataset or a predictive task rather than a direct input to a learning model. Metafeatures are used in meta-learning and AutoML to inform model selection, hyperparameter tuning, and algorithm configuration by predicting which models or settings are likely to perform well on a given dataset.

Metafeatures can be categorized into several types. Simple statistical metafeatures include measures such as mean, variance, skewness, and kurtosis of feature values. Structural metafeatures describe dataset properties, such as the number of instances, the number of features, the ratio of numeric to categorical features, and the amount of missing data. Informational metafeatures involve information-theoretic quantities like entropy and mutual information. Landmarking features are derived from the quick performance of simple predictive models on the dataset, serving as proxies for baseline difficulty.

Extraction of metafeatures typically occurs after data preprocessing, and may require handling missing values or encoding categorical variables. Landmarking involves training lightweight models for a short period to obtain rapid performance indicators, which can be combined with other metafeatures to form a metafeature vector.

In practice, metafeatures support meta-learning tasks such as recommending algorithms, tuning hyperparameters, or estimating expected performance, thereby guiding AutoML systems and reducing search space. They also enable analysis of dataset similarity and transfer learning across related tasks. Limitations include the computational cost of calculating metafeatures, sensitivity to preprocessing choices, potential leakage if not carefully controlled, and the challenge of selecting informative features in high-dimensional metafeature spaces.