Featurization

Featurization is the process of converting raw data into a structured set of numerical features that can be consumed by machine learning models. It encompasses feature extraction, encoding, normalization, and sometimes dimensionality reduction. The aim is to represent information in a way that preserves predictive signals while reducing noise and complexity, thereby improving learning efficiency and generalization.

Techniques vary by data modality. For structured/tabular data, featurization often involves handling missing values, scaling numerical features, and encoding categorical variables through one-hot encoding, ordinal encoding, or learned embeddings. For text, common methods include bag-of-words, term frequency–inverse document frequency (TF-IDF), and increasingly, contextual word embeddings. For images, features may be handcrafted descriptors such as SIFT or HOG, or activations from convolutional neural networks. For audio, features such as MFCCs and spectral descriptors are typical, and for time series, windowed statistics or spectral features are used. Automatic feature extraction and representation learning increasingly reduce manual engineering.

Featurization is distinct from feature selection, which aims to reduce dimensionality by removing irrelevant or redundant features after they have been created. It is also different from raw data collection or domain-specific engineering tasks, though domain knowledge often informs the choice of featurization techniques.

In practice, featurization is implemented as part of data preprocessing pipelines and evaluated via cross-validation. Its effectiveness influences model accuracy, training efficiency, interpretability, and robustness to noise and distribution shift.