stopsequence

A stop sequence is a string or sequence of tokens used to signal a language model to halt generation. It serves as a boundary marker to delimit outputs, prevent unwanted continuation, or separate sections in a multi-part response. Stop sequences can be defined as single strings or a list of acceptable endings, and they may operate at the character level or the token level depending on the system.

In practice, the generation process checks the produced text for any of the specified stop sequences. When a match is found, the model stops emitting further content. Common uses include ending a reply at a natural boundary, preventing the model from continuing into unrelated content, or enforcing structured formats in API responses. For example, an API might stop at a string like "\nUser:" or "\nAssistant:" to keep responses within a defined role or turn structure.

Common frameworks implement stop sequences differently. OpenAI's API uses a stop parameter that accepts one or more strings. HuggingFace transformers provide stopping criteria or custom logic to halt generation when a particular sequence appears in the output. Considerations when choosing stop sequences include the risk of accidental triggering if the sequences appear in legitimate content, and the potential for incomplete results if a sequence is too short or likely to occur naturally.

Limitations include non-deterministic hampering of content and the possibility that a stop signal may not be reached if the model never generates the sequence. Effective use typically involves testing with realistic prompts and adjusting sequences to balance completeness and boundary control.