Melspectrogram

A melspectrogram is a time-frequency representation of audio in which the frequency axis is mapped to the mel scale, a perceptual scale that spaces frequencies more densely at lower ranges to reflect human hearing. It is usually obtained by applying a mel-filter bank to the magnitude or power spectrum of the signal, so that each mel-bin aggregates energy from a range of Fourier components. The result is a two-dimensional array with time on one axis and mel bands on the other, representing the evolving energy distribution across perceptual frequency bands.

Compute it typically by first extracting a short-time Fourier transform (STFT) of the audio, optionally applying pre-emphasis and windowing. The magnitude (or power) spectrum from each frame is then multiplied by a bank of triangular filters whose centers lie on the mel scale, producing one energy value per mel-bin per frame. Often a logarithmic or decibel transformation is applied to compress dynamic range, yielding the log-mel spectrogram. Some workflows also compute log-mmel features after a discrete cosine transform to obtain Mel-frequency cepstral coefficients (MFCCs).

Variants include the magnitude or power mel-spectrogram and the log-mel spectrogram, with choices about the number of mel bands, FFT size, hop length, and sampling rate. Mel spectrograms are widely used as inputs to machine learning models for speech recognition, music information retrieval, and audio classification, because they provide a compact, perceptually relevant representation and generally improve learning performance over raw spectrograms.

Limitations include dependence on sampling rate and filter-bank design, potential loss of spectral details not well captured by the mel scale, and assumptions inherent in STFT-based analysis.