RateDistortion
Rate-distortion theory studies the trade-off between the bitrate required to encode a source and the fidelity of its reconstruction when lossy compression is permitted. The rate, R, is the average number of bits per source symbol, and the distortion, D, is a measure of dissimilarity between the original X and its reconstruction X_hat according to a distortion function d(x, x_hat).
Formally, for a source X with distribution p(x) and reconstruction alphabet X_hat, a coding scheme of rate
Common distortion measures include squared error, d(x, x_hat) = (x − x_hat)^2, used for continuous-valued sources, and Hamming
Operationally, R(D) provides the minimal achievable bitrate to ensure average distortion no larger than D, and