rdmacm

RDMACM stands for RDMA Connection Manager. It is a component of the RDMA software stack that provides connection-oriented setup and lifecycle management for RDMA-enabled endpoints. Operating between user-space applications and the underlying transport layers, RDMACM coordinates address resolution, route resolution, and the establishment of RDMA connections over transports such as InfiniBand, RoCE, and iWARP.

The RDMA CM handles the process from initial contact to established communication. Applications typically use the librdmacm library to create an event channel and one or more RDMA IDs (endpoints), request address resolution, and resolve routes. When a connection is ready, the RDMA CM negotiates connection parameters and moves the endpoint through states such as address resolved, route resolved, and established. The process is asynchronous, and events indicating status changes (for example, address resolved, route resolved, connection established, rejected, or disconnected) are delivered via the event channel.

RDMACM supports establishing connections for RDMA transports used by remote direct memory access operations. While it does not transfer user data itself, it sets up the queued pair (QP) and resources required for subsequent RDMA verbs operations (reads, writes, and atomic actions). It also supports handling connection requests, rejects, and disconnects as part of the lifecycle management.

Implementation and scope: RDMACM is provided as part of the Linux RDMA stack (kernel components) and user-space libraries (librdmacm) within the rdma-core package. It is widely used in high-performance computing and data-centric applications that rely on RDMA for low-latency, high-throughput communication, including MPI implementations and other distributed systems.