Scorefollowing

Scorefollowing, or score following, is the real-time process of aligning a live musical performance with a pre-defined score or symbolic representation. The goal is to continuously estimate the performer’s current position in the score—measures, beats, and events—as the music unfolds, even when tempo changes, rubato, repeats, or deviations occur. Scorefollowing is central to interactive and adaptive music systems, enabling synchronization between performance and accompaniment, visualization, or stage effects.

Computational approaches combine signal processing and probabilistic modeling. Common methods include hidden Markov models (HMMs) that treat the score as a sequence of states and the performance as observations, dynamic time warping (DTW) variants that align performance events to score events, and neural network models that map audio or performance data directly to score events. Modern systems often integrate onset detection, beat tracking, tempo estimation, and polyphonic transcription to improve robustness and accuracy.

Input modalities typically involve audio recordings or live instrument input, while the score is represented in symbolic form such as MIDI, MusicXML, or a note sequence. Some implementations use MIDI performance data when available for tighter alignment.

Applications of scorefollowing include automatic or adaptive accompaniment, interactive concert systems, educational tools, rehearsal and performance analysis, and assistive music technologies. Challenges remain in handling expressive timing, improvisation, repeats or cuts, noise, and polyphonic textures, as well as achieving low-latency, high-accuracy alignment across diverse repertoires.