halfprecision

Half precision, or binary16, is a 16-bit floating-point format defined by the IEEE 754 standard. It uses 1 sign bit, 5 exponent bits, and 10 fraction (mantissa) bits. The exponent has a bias of 15, allowing a range of real values, with normal numbers using an implicit leading 1 in the significand and subnormal numbers (when the exponent is zero) encoding the fraction without that leading bit.

In normal form, values are approximately (-1)^sign × (1.fraction) × 2^(exponent − bias). Subnormal numbers have exponent zero and provide a gradual underflow near zero, with the smallest positive subnormal around 5.96 × 10^−8 and the largest subnormal around 6.10 × 10^−5. The largest finite normal value is about 6.5504 × 10^4, and infinities and NaNs are supported as in other IEEE 754 formats.

Half precision stores two bytes per value, making it attractive for reducing memory usage and bandwidth in applications such as graphics and machine learning. Its reduced precision and dynamic range mean faster arithmetic on some hardware, but with potential loss of accuracy and numerical stability. Operations can be slower on platforms without native FP16 support, and rounding errors or underflow/overflow are more likely compared with higher-precision formats. In practice, half precision is often employed with techniques such as mixed-precision training, where FP16 data are accumulated in higher-precision (for example FP32) to preserve numerical accuracy.

Related formats include brain-friendly alternatives like bfloat16, which preserves the FP32 exponent range but uses fewer mantissa bits.