Hiperparaméter

Hyperparameters are configuration variables that are external to the model and whose values cannot be estimated from the data. They are used to control the learning process of a machine learning model. Unlike model parameters, which are learned during training, hyperparameters are set before training begins.

Examples of hyperparameters include the learning rate in gradient descent, the number of layers in a neural network, the number of clusters in k-means clustering, and the regularization parameter. The choice of hyperparameters can significantly impact the performance of a machine learning model.

Tuning hyperparameters is a crucial step in building effective machine learning models. This process involves selecting a set of hyperparameter values that result in the best performance on unseen data. Common hyperparameter tuning techniques include grid search, random search, and Bayesian optimization.

Grid search exhaustively tries all possible combinations of a predefined set of hyperparameter values. Random search samples hyperparameter values randomly from predefined distributions, which can be more efficient than grid search, especially in high-dimensional hyperparameter spaces. Bayesian optimization uses probabilistic models to guide the search for optimal hyperparameters, aiming to find good values more quickly.

The goal of hyperparameter tuning is to find a configuration that generalizes well to new data, avoiding overfitting (performing well on training data but poorly on unseen data) and underfitting (performing poorly on both training and unseen data).