encodingstandaards

Encoding standards specify the rules for converting text characters into sequences of bytes so they can be stored, transmitted, and later reconstructed. They cover both the repertoire of characters (the character set) and the method for encoding those characters into bytes (the encoding form). A given standard also defines how to handle issues such as endianness, control characters, and compatibility with legacy systems.

The most influential framework is Unicode, which seeks to assign a unique code point to every character used in written text. Unicode defines encoding forms such as UTF-8, UTF-16, and UTF-32, which determine how code points are mapped to bytes. The Unicode standard is coordinated with ISO/IEC 10646, and both describe compatible character repertoires and encoding rules. Standards bodies and organizations involved include the Unicode Consortium, ISO/IEC, and various committees that address data interchange formats, programming interfaces, and web technologies.

Common encodings beyond Unicode include ASCII (a 7-bit subset of Unicode), ISO/IEC 8859 family (Latin-1 and related encodings), Windows-1252, and several locale-specific encodings such as Shift JIS, GB2312/GBK, and Big5. In practice, UTF-8 has become the dominant encoding for the web and many software systems due to its ASCII compatibility and variable length.

Practical considerations include endianness for UTF-16 and UTF-32, the use of byte order marks, and the need for transcoding when interfacing systems with different encodings. Correct handling of encoding is essential for data integrity, interoperability, and correct display of text.