Demosaicking

Demosaicking, also called debayering, is the process of reconstructing a full-color image from the incomplete color samples captured by a digital camera’s color filter array (CFA). Most consumer sensors use a Bayer pattern, where each pixel records only one color component (red, green, or blue). The pattern contains twice as many green samples as red or blue to reflect the human eye’s greater sensitivity to luminance detail.

Because each pixel contains only one color, the missing color channels must be estimated from neighboring samples and from correlations between color channels. Simple early methods used straightforward interpolation, such as bilinear or nearest-neighbor, which can blur fine detail and introduce color artifacts. More advanced approaches are edge-aware and gradient-driven, aiming to preserve edges while estimating missing values. Other techniques formulate demosaicking as an optimization problem with priors on smoothness, color consistency, or texture preservation, sometimes incorporating denoising or joint denoising-demosaicking steps.

In addition to traditional algorithms, learning-based methods, including convolutional neural networks, have become common, often producing higher quality results, especially in challenging textures or under low light. Demosaicking also faces challenges such as moiré, zippering along strong edges, and noise amplification; many pipelines perform demosaicking and denoising or other processing in a joint or carefully ordered fashion to mitigate artifacts.

While Bayer remains the dominant CFA, other patterns exist (for example, X-Trans or CYGM), each requiring tailored demosaicking strategies. Quality is evaluated with objective metrics like PSNR or SSIM and, importantly, perceptual image quality in real-world viewing.