RKHSNorm

RKHSNorm refers to the norm in a Reproducing Kernel Hilbert Space (RKHS). An RKHS is a Hilbert space of functions f on a set X equipped with an inner product <f, g>_H and a reproducing kernel K: X × X → R such that f(x) = <f, K_x>_H for all f in H, where K_x(·) = K(x, ·). The norm is defined by ||f||_H = sqrt(<f, f>_H).

Properties and finite expansions: The RKHS norm is nonnegative, homogeneous, and satisfies the triangle inequality. If f lies in the finite span of kernel evaluations at data points x_1, ..., x_n, i.e., f = sum_i alpha_i K(x_i, ·), then ||f||_H^2 = alpha^T K_{XX} alpha, where K_{XX} is the n×n Gram matrix with entries K(x_i, x_j). The reproducing property also yields the bound |f(x)| ≤ ||f||_H sqrt(K(x, x)) for all x in X.

Role in learning: RKHS norms are commonly used as regularizers in kernel methods. By the representer theorem, minimizers of empirical risk with an RKHS norm penalty lie in the span of training kernels, reducing the optimization to a finite-dimensional problem. For example, minimizing sum_i L(y_i, f(x_i)) + λ ||f||_H^2 over f in H leads to a solution with f(·) = sum_i α_i K(x_i, ·) and a dual problem expressed in terms of the coefficients α.

Notes: The RKHS norm depends on the chosen kernel, encoding different notions of smoothness and complexity. For universal kernels, the RKHS is rich enough to approximate many functions, and the norm serves as a measure of function complexity in kernel-based learning.