Viseme

Viseme is a term used in linguistics and computer graphics to denote a visual unit of speech corresponding to the mouth shapes and facial movements associated with producing a phoneme. Phonemes are abstract sound categories; visemes describe what the lips, teeth, tongue, and jaw look like during articulation. Because many phonemes produce nearly identical visible articulations, a single viseme often covers multiple phonemes. The exact viseme set varies by language, speaker, and application, but many systems use a compact repertoire of visually distinctive units to represent speech visually.

In visual speech processing, visemes are used to support lip-reading, visual speech recognition, and lip-sync for animated avatars and synthetic speech. In synthesis and animation, mapping from phonemes to visemes enables synthetic voices to display believable lips and facial movement in sync with the audio. Coarticulation—the influence of surrounding sounds on a given articulatory gesture—means that a viseme's appearance can depend on context, complicating the alignment of audio and visual streams.

Challenges include speaker variability, lighting, camera angles, and facial appearance differences that affect visibility of lip movements. The inherent many-to-one mapping from phonemes to visemes can create ambiguity for recognition systems and require contextual modeling. Viseme inventories for English typically comprise a modest set of categories, with larger or language-specific sets used in research and commercial products.