FineTune

Finetune is the process of adapting a pretrained model to a specific task or domain by continuing training on task‑specific data. It leverages general representations learned during pretraining while refining the model to perform well on a narrow objective, such as a particular language task, image classification in a new domain, or speech recognition for a specific corpus.

The typical finetuning workflow starts with selecting a suitable pretrained model and assembling a labeled or evaluation dataset for the target task. Depending on resource constraints and risk of overfitting, practitioners may update all model weights or only a subset. Common approaches include full finetuning, where all parameters are updated, and partial finetuning, where the base layers are frozen and only task-specific heads or a few layers are trained. Training often uses a smaller learning rate than pretraining, with schedules that may include gradual unfreezing and discriminative learning rates across layers. Regularization and data augmentation help mitigate overfitting.

Several parameter‑efficient strategies have emerged, such as adapters, low‑rank adaptation (LoRA), and prefix‑tuning, which add small trainable modules or prompts while keeping most of the pretrained weights fixed. These methods can reduce computational cost and memory usage and are popular in large multilingual models and vision transformers.

Applications span natural language processing, computer vision, and speech tasks. Key considerations include data quality and size, domain shift, evaluation on a held‑out set, and awareness of risks such as overfitting, catastrophic forgetting, and data leakage.