overparametrisering

Overparametrisering is a term used primarily in the field of machine learning and statistical modeling to describe a situation where a model has more parameters than can be justified by the amount of data available for training. In simpler terms, it refers to the use of a model that is too complex for the given dataset, leading to potential issues in the model's performance and generalization.

When a model is overparameterized, it may fit the training data extremely well, capturing not only the underlying patterns but also the noise and outliers. This is often referred to as overfitting. While an overparameterized model can achieve very low training error, it typically performs poorly on new, unseen data because it has essentially memorized the training set rather than learning the generalizable patterns.

The problem of overparametrisering is particularly relevant in high-dimensional settings, where the number of features (and thus potential parameters) can be very large compared to the number of observations. Techniques to mitigate overparametrisering include regularization methods like L1 and L2 regularization, which penalize large parameter values, and model selection methods that choose simpler models based on criteria like the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC).

In recent years, the concept of overparametrisering has been revisited in the context of deep learning, where models with millions of parameters are common. Interestingly, some deep learning models can generalize well despite being overparameterized, a phenomenon that is still not fully understood and is an active area of research. Understanding and managing overparametrisering is crucial for building robust and generalizable machine learning models.