FastSpeech

FastSpeech is a neural text-to-speech (TTS) model designed to speed up speech synthesis by removing autoregressive dependencies. Introduced as a non-autoregressive approach, it generates speech in parallel rather than frame by frame, which considerably speeds up inference compared with earlier autoregressive systems.

The core architecture uses a Transformer-based encoder to convert input text, typically represented as phonemes, into a sequence of hidden representations. A duration predictor estimates how many acoustic frames correspond to each phoneme, and a length regulator expands the encoded sequence to match the target mel-spectrogram length. A parallel decoder then produces mel-spectrogram frames in one shot, enabling fast synthesis once the model is trained. Training relies on reconstructing target mel-spectrograms and uses supervision for durations, often derived from forced alignments or phoneme-to-frame mappings.

FastSpeech’s design emphasizes efficiency and robustness, delivering high-quality speech with significantly faster-than-real-time generation on standard hardware when paired with a suitable vocoder to convert mel-spectrograms into waveforms.

Variants and extensions: FastSpeech 2, introduced to improve naturalness and expressiveness, extends the original approach by explicitly modeling prosodic features such as pitch, energy, and duration, while maintaining the non-autoregressive core. This enhances the naturalness and variability of synthesized speech across different speakers and styles.

Applications and impact: FastSpeech has influenced subsequent non-autoregressive TTS research and is used in real-time or near real-time TTS deployments. Its effectiveness depends on a robust vocoder, such as a neural spectrogram-to-waveform model, to generate the final audio.