crossfeatures

Crossfeatures, or cross features, are interaction terms created by combining two or more input features to capture joint effects in predictive models. They are a common technique in feature engineering used to augment a model's ability to represent nonlinear relationships without changing the underlying modeling algorithm.

Crossing numeric features can involve multiplying or otherwise combining values to produce a new feature that reflects an interaction, such as age multiplied by income. Crossing categorical features often means creating a new categorical value that encodes the combination, such as country x device type, then encoding that cross with one-hot encoding or target encoding.

Common approaches include polynomial features that generate all polynomial and interaction terms up to a chosen degree, and explicit cross feature construction followed by encoding. For high-cardinality categories, hashing tricks or embedding-based approaches can help manage dimensionality; an alternative is target encoding for the cross.

Cross features can improve the performance of linear models (e.g., logistic regression, linear regression) by allowing them to model interactions, but they can also increase dimensionality and risk overfitting. Tree-based models can learn interactions automatically, reducing the need for explicit crossing.

Practitioners typically start with domain knowledge, test a small set of crosses, and evaluate with cross-validation, using regularization and feature selection to control complexity. Crossfeatures sit within broader feature engineering practices and are often used in datasets where interactions between variables are believed to drive the target outcome.