kernelprincipalacomponenten

Kernel Principal Component Analysis (KPCA) is a nonlinear extension of principal component analysis (PCA) that uses the kernel trick to operate in a high-dimensional feature space without explicit mapping. By performing linear PCA in this feature space, KPCA can capture nonlinear structure in the data while remaining computable in the input space.

In practice, KPCA forms a centered kernel matrix K with entries k(x_i, x_j) for all training samples. The eigen-decomposition K = V Λ V^T yields eigenvectors v_p that correspond to the principal components in feature space. The projection of a new point x onto the p-th component is given by sum_i v_p[i] k(x_i, x), which avoids direct use of the mapping φ(x).

Common kernel functions include Gaussian (RBF), polynomial, and sigmoid kernels. The choice of kernel determines the implicit feature space and the kinds of nonlinear relationships that can be captured. Parameter selection (for example, the bandwidth of the RBF kernel) is typically guided by cross-validation or domain insight.

KPCA is used for nonlinear dimensionality reduction, data visualization, and as a preprocessing step for clustering or classification when nonlinear structure is present. It can reveal manifolds and patterns that linear PCA cannot identify.

Limitations include computational cost for large datasets, since constructing and decomposing the n x n kernel matrix scales poorly (time roughly cubic in the number of samples; memory quadratic). The preimage problem—recovering a representative input-space point from a kernel-space coordinate—is nontrivial. Extensions address scalability, for example via the Nyström method or random feature approximations.

Kernel PCA was introduced in the late 1990s by Schölkopf, Smola, and Müller as a kernelized generalization of PCA in reproducing kernel Hilbert spaces.