MTFer
MTFer is a term used to describe a family of multitask transformer architectures designed to efficiently learn and perform multiple tasks within a single model. The approach extends standard transformer models by incorporating lightweight, task-specific adapters that modulate the shared representation and by leveraging task tokens or prompts to condition the model on the current task.
The core idea behind MTFer is to maintain a common, powerful backbone while allowing small, task-specific adjustments.
During training, a multitask objective combines losses from all tasks, often with balancing strategies to prevent
Variants of MTFer emphasize different priorities. MTFer-Lite focuses on parameter efficiency with compact adapters; MTFer-Plus adds
Applications span natural language processing, vision-language tasks, and cross-domain adaptation, particularly where a single system handles