HiFiGAN

HiFi-GAN is a neural vocoder designed to convert mel-spectrograms into high-fidelity speech waveforms. It was introduced to provide natural-sounding audio with real-time or near real-time generation, improving both quality and efficiency over earlier GAN-based vocoders.

The generator upscales the input mel-spectrogram through a sequence of upsampling layers to produce a raw audio waveform. Training relies on adversarial losses from discriminators plus auxiliary losses such as feature matching and mel-spectrogram or waveform-based reconstruction terms. A distinctive feature is the use of multiple discriminators: multiple-scale discriminators (operating at different audio resolutions) and multiple-period discriminators (evaluating periodic patterns across various pitch intervals). This combination helps the model capture long-range temporal structure and fine-grained waveform details.

HiFi-GAN has evolved through several versions, notably HiFi-GAN v1 and subsequent improvements, which focus on stability, artifact reduction, and efficiency. The model is trained on large speech datasets and can be conditioned on speaker-specific mel-spectrograms, enabling high-quality speech synthesis and voice cloning when paired with capable front-ends. It is widely used in open-source text-to-speech pipelines and commercial-like applications due to its balance of synthesis quality and computational efficiency.

Applications include text-to-speech synthesis, voice conversion, and singing voice synthesis. Limitations can include dependence on representative training data for target voices and languages, potential artifacts with voices or styles not seen during training, and the need for careful pre-processing of mel-spectrograms to achieve optimal results.

Related technologies in the neural vocoder space include WaveNet, MelGAN, Parallel WaveGAN, and other GAN- or flow-based vocoders.