hyperparametern

Hyperparametern, in German usage, denotes the plural of hyperparameter. A hyperparameter is a configuration value external to a machine learning model that is not learned from the data during training. Hyperparameters influence learning dynamics, model capacity, and computational cost, and must be selected before training.

Common hyperparameters include the learning rate, batch size, number of layers and neurons per layer, regularization strength, dropout rate, activation function, optimizer type, and data preprocessing options. Hyperparameters can be continuous (e.g., learning rate, weight decay) or discrete (e.g., number of layers, batch size). Some affect training duration, others affect the model's bias-variance trade-off.

Hyperparameter tuning is the process of finding effective values. Methods range from manual adjustment to automated strategies: grid search, random search, Bayesian optimization, Hyperband/Successive Halving, and evolutionary algorithms. More recently, gradient-based or differentiable hyperparameter optimization and AutoML systems automate the process at scale.

Best practices include using a separate validation set to assess performance, keeping track of computational budget, and starting from sensible defaults based on prior experience. It is important to avoid leaking test data into validation during tuning. Hyperparameter choices can have a larger impact on performance than minor architectural changes, especially in deep learning.

Hyperparameter optimization remains a central step in model development and deployment, influencing reproducibility and efficiency. In German-speaking contexts, the term Hyperparametern is commonly used to describe multiple hyperparameters collectively.