multihead

Multihead is a term used to describe systems that incorporate multiple heads or working channels. It is used in various domains to indicate parallel or heterogeneous processing units operating on a common input or task.

In artificial intelligence, multihead attention is central to transformer models. Several attention heads operate in parallel within a layer. Each head uses its own projections for queries, keys, and values, computes scaled dot-product attention, and yields an output. The heads’ outputs are concatenated and projected to form the final representation. This enables the model to attend to information from different subspaces of the input, capturing diverse features such as syntax, semantics, or relations. The number of heads is a hyperparameter; each head’s dimension is typically the overall model dimension divided by the number of heads. Scaling by the square root of the head dimension helps stabilize training.

Beyond AI, “multihead” is used as a generic descriptor for machinery or devices with more than one active head. Examples include multi-head display adapters that drive several monitors, multi-head CNC machines or 3D printers equipped with multiple tooling heads, and other production equipment designed to perform parallel or multi-task operations on a single workflow. In hardware contexts, the term emphasizes modularity and parallelism rather than a single, centralized function.

In summary, multihead denotes multiplicity of functional heads, with specific implementations and implications varying by field, from neural networks to manufacturing and hardware interfaces.