Cybenko

Cybenko is a mathematician best known for Cybenko's theorem on neural networks, a foundational result in the theory of function approximation. In 1989 he proved that a feedforward neural network with a single hidden layer and a sigmoidal activation function can approximate any continuous function on a compact subset of R^n to arbitrary accuracy, given a sufficient number of hidden units.

The formal statement of the result describes networks of the form f(x) = sum_{i=1}^N a_i sigma(w_i^T x + b_i), where sigma is a sigmoidal activation function, and the parameters a_i, w_i, b_i are chosen to approximate a target function on a chosen compact domain. The theorem places Cybenko's work in the broader field of universal approximation, showing that even simple architectures possess substantial expressive power.

Cybenko's result had a profound influence on neural network theory and approximation theory, and it paved the way for subsequent generalizations. In particular, later work by Hornik, Stinchcombe, and White extended the universality property to a wider class of activation functions and network architectures, reinforcing the idea that neural networks can serve as universal function approximators under mild conditions.

The theorem remains a central reference in discussions of neural network expressivity and capacity, and it is frequently cited in both mathematical analyses and machine learning literature as one of the early formal confirmations of neural networks’ theoretical capabilities. Cybenko’s contribution is widely recognized as a milestone in the mathematical understanding of neural networks.