strtokr

Strtok_r is a reentrant version of the C string tokenizer strtok, designed for use in POSIX environments. It provides a thread-safe way to tokenize a string by a set of delimiter characters, avoiding the static internal state that strtok uses.

The standard, portable signature is: char strtok_r(char str, const char delim, char saveptr); The first call should pass the input string in str and a delimiter string in delim, and provide a pointer to a char variable as saveptr to hold the tokenizer’s context. On subsequent calls to obtain further tokens from the same string, str should be NULL while the same saveptr is used. The function returns a pointer to the next token, or NULL when there are no more tokens.

Strtok_r modifies the input string in place. It replaces the delimiter that terminates a token with a null character and updates the context stored in saveptr so subsequent calls continue from the correct position. Tokens returned by strtok_r point into the original string; no new memory is allocated.

Key advantages over strtok include thread safety and reentrancy, provided each thread or concurrent tokenizer uses its own saveptr. In contrast, strtok uses an internal static state, making concurrent tokenization unsafe without synchronization.

Portability notes: strtok_r is part of the POSIX standard and is available in most UNIX-like systems and their C libraries (such as glibc, musl, BSD). On Windows, a different function, strtok_s, exists with a distinct signature and behavior.

Common usage involves maintaining a separate saveptr variable for each string to tokenize, ensuring that multiple tokenizations can occur independently. While powerful, strtok_r still alters the original string and does not allocate memory for tokens, so careful string management is required.