multiplyadd

Multiply-add is an operation that multiplies two numbers and then adds a third, typically executed as a single step in hardware or software. In floating-point arithmetic the fused multiply-add (FMA) computes a*b + c with one rounding, which improves numerical accuracy and can increase throughput. By contrast, performing a*b and then adding c as two separate operations introduces two roundings, which can yield slightly different results and less precision.

The term MAC (multiply-accumulate) is closely related and often used in signal processing and vector processing contexts. A non-fused MAC performs the multiply and add as two distinct steps, while an FMA implements both in a single instruction or pipeline stage. The distinction matters for numerical behavior: FMA provides the most accurate result for a given sequence of operations and can reduce error accumulation in iterative calculations.

Hardware implementations are widespread. Many CPUs and GPUs include FMA instructions (for example, x86 with FMA3/FMA4, various ARM profiles, and numerous GPU architectures expose fmaf or fused mad equivalents). MAC units are also common in digital signal processors and in specialized accelerators for machine learning and scientific computing.

Common applications include dot products, matrix and tensor operations, convolution, and neural network inference, where MACs dominate compute. In software, multiply-add operations are fundamental building blocks for linear algebra routines and optimization algorithms. While FMA offers performance and precision benefits, some software environments must emulate fused behavior when exact IEEE semantics are required.