Lengthprefixed

Lengthprefixed, or length-prefixed, is a data encoding method in which each data unit is preceded by a length field that specifies the number of bytes in that unit. This framing allows receivers to locate message boundaries in a continuous stream without relying on delimiters within the payload.

The length field can take different forms. A fixed-size prefix uses a set number of bytes (for example 1, 2, 4, or 8) and is often stored in big-endian “network” order. A variable-length prefix, such as a varint, encodes smaller sizes more compactly and is used in systems like Protocol Buffers and some ASN.1 encodings. In both cases the payload is the raw data of the unit, whose internal encoding (for example UTF-8 strings or binary blobs) is independent of the framing.

Common examples include:

- A simple binary message where a 4-byte length field is followed by that many payload bytes.

- Netstrings, where the length is written in ASCII digits, followed by a colon, the data, and a trailing comma, making the format self-delimiting.

- Length-prefixed strings in many programming languages, where strings are stored as a length followed by bytes of the string.

Applications and considerations:

Lengthprefixed framing is widely used in network protocols, inter-process communication, and streaming data to delineate messages in a continuous flow. It simplifies parsing and supports streaming, but requires careful validation to avoid oversized lengths that could cause memory exhaustion or parsing delays. Endianness and encoding choices should be documented and consistently applied.

See also: varint, netstring, delimiter framing, message framing.