twolayer

Twolayer is a term used in machine learning and neural networks to refer to a two-layer neural network, also known as a shallow network. In this context, two layers of trainable connections exist: the first connects the input to a hidden layer, and the second connects the hidden layer to the output. The hidden layer typically applies a nonlinear activation function, while the output layer applies a task-appropriate function (for example linear activation for regression, or softmax for multi-class classification). When the hidden layer is fully connected and the network uses a nonlinear activation, it is commonly called a two-layer perceptron.

Training a twolayer network usually uses backpropagation to compute gradients for both sets of weights, followed by an optimization method such as stochastic gradient descent. Regularization techniques, such as weight decay or dropout, are used to mitigate overfitting. The two-layer structure is of particular interest in learning theory: the universal approximation theorem states that a sufficiently wide two-layer network with a non-polynomial activation can approximate any continuous function on compact input domains, given enough hidden units.

Variants include radial basis function networks, which can be viewed as two-layer architectures with a fixed, nonlearned hidden transform followed by a linear output layer. In practice, two-layer networks serve as simple baselines, educational examples, or components in larger systems; they often require many hidden units to express complex mappings and can be sensitive to initialization.

History and context: foundational results in the 1980s by Cybenko and Hornik demonstrated the expressive power of two-layer networks. See also: multilayer perceptron, feedforward neural network, universal function approximation.