RoCEv2

RoCEv2, short for RDMA over Converged Ethernet version 2, is a protocol that enables remote direct memory access over Ethernet networks and extends RDMA beyond a single Ethernet subnet. It is the routable successor to RoCEv1, which operated at Layer 2 within a single broadcast domain. RoCEv2 is often referred to as Routed RoCE (RRoCE) because it uses IP routing to move RDMA traffic across subnets while preserving the low latency and high throughput characteristics of RDMA.

RoCEv2 builds on the RDMA over Converged Ethernet stack and relies on the RDMA Protocol over Converged Ethernet (RDMAP) to carry RDMA operations and InfiniBand-style verbs. A key requirement for RoCEv2 is a lossless Ethernet fabric, typically achieved with Priority Flow Control (PFC). QoS mechanisms such as Enhanced Transmission Selection (ETS) are commonly used to allocate resources and prevent head-of-line blocking, which can degrade RDMA performance on mixed traffic.

Deployment considerations include the need for RDMA-capable NICs and switches that support RoCEv2, as well as proper configuration of PFC and ETS across the network. RoCEv2 enables routing across IP subnets, but requires careful network design to maintain low latency and high reliability. It is often contrasted with iWARP, which RDMA over Ethernet via TCP/IP, while RoCEv2 maintains InfiniBand-style RDMA semantics over routable Ethernet.

Use cases for RoCEv2 include data-center storage and virtualization, high-performance computing, and other applications demanding low-latency, high-throughput memory transfers with minimal CPU overhead.