aktarmnn

ArmNN is an open-source neural network inference engine developed by Arm Limited. It provides a cross-platform C++ API with optional Python bindings that enable deploying trained neural networks on Arm-based devices with low latency and memory use. ArmNN acts as a runtime layer that maps high-level network graphs to hardware backends, handling graph optimization, memory management, and data layout decisions to maximize performance on Arm CPUs, GPUs, and specialized accelerators.

The engine supports multiple backends, including a CPU backend based on Arm NEON and a GPU backend for Arm Mali GPUs via OpenCL (with broader support depending on platform). It also offers an Ethos-N Neural Processing Unit backend for certain Arm NPU devices. Backends are designed to abstract hardware specifics behind a common graph representation, allowing the same model to run on different Arm hardware with backend-specific optimizations.

ArmNN provides parsers and workflow integration for common model formats such as TensorFlow Lite and ONNX. Models are translated into ArmNN’s internal representation, where graph optimizations and operator mappings are applied before execution on the chosen backend. The project emphasizes compatibility with Arm Compute Library primitives and aims to offer efficient, portable on-device inference across a range of Arm devices.

Licensing and availability: ArmNN is released as open-source software under the Apache 2.0 license and is hosted on Arm’s official GitHub repository. It is used in embedded and edge applications to enable on-device AI inference, often as part of Arm’s broader ML software ecosystem. Related components include Arm Compute Library and Ethos-N tooling.