hlo

HLO stands for High-Level Optimizer and is an intermediate representation used by Google's XLA compiler to describe computations in a platform-neutral form. It is most commonly associated with accelerating machine learning models in frameworks such as TensorFlow and JAX, enabling optimizations to be performed before lowering to backend-specific code for CPUs, GPUs, and TPUs.

In HLO, a module contains one or more computations, and each computation is a sequence of instructions that operate on tensors with defined shapes and layouts. The IR captures a range of operations, including elementwise arithmetic, reductions, contractions (such as matrix multiplications), convolutions, and control flow. A key feature of HLO is the ability to fuse together multiple operations into a single kernel, which can reduce memory traffic and improve performance.

The primary purpose of HLO is to provide a high-level view of the computation that enables global optimizations. Typical optimizations include operator fusion, algebraic simplifications, layout and memory coherence adjustments, and scheduling decisions that improve cache efficiency and parallelism. By operating on a representation that abstracts away device-specific details, HLO supports portable optimizations across different hardware backends.

HLO originated as part of the XLA project to improve the performance and portability of tensor computations. It serves as a bridge between front-end frameworks and backend code generation, allowing front-ends to express models in a regular, analyzable form while enabling backends to generate optimized executable code. In modern toolchains, HLO is often used alongside MLIR-based components, reflecting evolving strategies for cross-backend optimization and code generation.