nontrainable

Nontrainable describes model parameters, components, or operations that do not participate in gradient-based updates during training. In machine learning, most parameters are trainable, meaning they are adjusted to minimize a loss function. Nontrainable elements remain fixed after initialization or selective freezing, though they may still influence the forward pass and predictions.

In neural networks, it is common to freeze certain layers or submodules to preserve pre-learned representations. This is a standard approach in transfer learning, where a backbone network from a pre-trained model serves as a fixed feature extractor while a new classifier head is trained on the target task. Other examples include fixed embeddings, fixed normalization constants, or constant auxiliary inputs.

Implementation is framework-specific but generally involves preventing gradients from flowing to the nontrainable parameters. In PyTorch, parameters can be marked with requires_grad = False; in TensorFlow, variables can be marked as non-trainable or wrapped in a stop_gradient operation. Nontrainable parameters still participate in the forward computation but do not receive gradient updates.

Benefits of nontrainable components include reduced computational cost and memory usage during training, simpler optimization, and improved stability when incorporating large pre-trained modules. They also enable learning from limited data by leveraging existing representations. Drawbacks include reduced model capacity and less adaptability, requiring careful choice of which parts to freeze and when to fine-tune.

See also: transfer learning, fine-tuning, frozen layers, trainable parameters, gradient flow.