NadarayaWatsonin

The Nadaraya–Watson estimator is a nonparametric method for estimating the conditional mean function m(x) = E[Y|X = x] in a regression setting. It is based on kernel smoothing and is used when the relationship between X and Y is complex or unknown.

Given a sample (Xi, Yi), i = 1,…,n, the Nadaraya–Watson estimator of m at a point x is defined as:

m̂(x) = [sum_{i=1}^n K_h(x − Xi) Yi] / [sum_{i=1}^n K_h(x − Xi)]

where K_h(u) = K(u/h)/h, h > 0 is the bandwidth and K is a nonnegative kernel function, typically a symmetric probability density such as a Gaussian or Epanechnikov kernel. The estimator can be interpreted as a locally weighted average of the observed Y-values, with weights determined by the closeness of Xi to x.

Key properties and practical aspects include:

- Bandwidth selection: h controls bias and variance; small h reduces bias but increases variance, while large h smooths more. Bandwidth is commonly chosen by cross-validation or plug-in methods.

- Consistency: under mild regularity conditions, if h → 0 and nh → ∞ as n → ∞, m̂(x) converges to m(x) for interior points.

- Bias and variance: bias depends on the curvature of m, often approximated by (h^2/2) m''(x) μ2(K); variance decreases with larger nh and higher local data density.

- Boundary effects: estimation near the boundaries can be biased; boundary kernels or local linear variants can mitigate this.

Extensions include multivariate inputs with product kernels and connections to local constant and local linear regression. The method was introduced independently by Nadaraya and Watson in the 1960s and remains a foundational tool in nonparametric regression.