Float16

Float16, also called binary16 or half precision, is a 16-bit floating-point number format defined by IEEE 754-2008. It uses 1 sign bit, 5 exponent bits, and 10 fraction bits, with an exponent bias of 15. This structure supports normal numbers, subnormals, zeros, infinities, and NaNs.

The normal range is from -65504 to 65504. Subnormal numbers extend the range toward zero, down to about 5.96 × 10^-8. Infinities occur for all-ones exponent with a zero significand, while NaNs occur with all-ones exponent and a nonzero significand. The format provides roughly 3 decimal digits of precision due to its 10-bit significand. Rounding follows the IEEE 754 rules, typically round-to-nearest-even.

Floating-point arithmetic in float16 is faster and uses half the memory of float32, making it attractive for large arrays of values, especially in graphics and machine learning. In practice, many systems use float16 for storage and as inputs to computations, while keeping accumulations or certain critical operations in higher precision (mixed-precision training and inference). Hardware support is common in modern GPUs and AI accelerators; software libraries and languages (such as NumPy, PyTorch, and TensorFlow) provide native or library-level support for float16 conversions and operations.

Limitations include a reduced dynamic range and lower precision compared with float32, which can lead to underflow, overflow, or numerical instability in some algorithms. Converting between formats can incur rounding errors. Float16 is often best suited for neural network weights and activations, where reduced memory usage and bandwidth outweigh precision drawbacks.

See also: bfloat16, IEEE 754 binary16.