ratedistortionoptimering

Rate-distortion optimization, referred to in Scandinavian languages as ratedistortionoptimering, is a set of techniques used in lossy data compression to balance the amount of data (rate) with the fidelity of the reconstructed signal (distortion). Its goal is to choose coding decisions that minimize the overall cost given a target bitrate or quality constraint.

The standard approach uses a Lagrangian cost function J = D + λR, where D measures distortion (for example, mean squared error or a perceptual metric) and R estimates the number of bits required to encode the chosen decision. The parameter λ, the rate–distortion trade-off factor, controls the balance: higher λ favors lower bitrates with potentially more distortion, while lower λ favors higher quality at higher bitrates. RD optimization is applied to a range of candidate decisions, including block partitioning, motion vectors, transform sizes, quantization parameters, and intra versus inter prediction modes.

In video codecs such as H.264/AVC, HEVC, VP9, and AV1, RD optimization is embedded in mode decision, rate control, and quantization. Encoders evaluate multiple candidates, estimate their distortion and bit cost, and select the option with the lowest J. Some implementations adapt λ dynamically to frame type, content complexity, or perceptual considerations, and may incorporate perceptual metrics to better align with human vision.

Practical RD optimization increases encoder complexity due to evaluating many candidates, so fast heuristics and early termination are common. The payoff is improved compression efficiency, yielding lower bitrates for the same quality or better visual quality at a given bitrate. Related concepts include rate-distortion theory and perceptual coding.