diarization

Diarization is the task of determining "who spoke when" in a multi-speaker audio recording. The goal is to partition the audio into segments attributed to individual speakers and to produce a chronological map of speech activity. Diarization is a common preprocessing step for transcripts, indexing, and analytics in meetings, broadcast media, call centers, and forensics.

A typical diarization pipeline starts with voice activity detection to separate speech from silence, followed by speaker segmentation to locate speaker change points. The segments are then clustered to group those spoken by the same person. In supervised settings, each cluster may be linked to a known identity during the attribution step.

Modern approaches combine neural representations such as x-vectors with probabilistic models (e.g., PLDA) and clustering algorithms, including agglomerative hierarchical clustering or spectral clustering. End-to-end diarization models have also emerged, aiming to jointly segment and attribute speakers in a single network. Both offline (batch) and online (real-time) diarization variants exist, with online systems trading accuracy for immediacy.

Evaluation typically uses diarization error rate (DER), which sums missed speech, false alarms, and speaker misattribution. DER is often reported with adjustments for overlapping speech and with a tolerance window around segment boundaries. Common benchmarks include meeting and broadcast datasets.

Applications include automated meeting transcripts, multimedia search, customer call analytics, and forensic investigations. Diarization remains challenging in the presence of overlapping speech, many speakers, short segments, and diverse recording conditions, and it continues to evolve with advances in speaker representations and scalable clustering methods.