FFNs

FFNs, or feed-forward neural networks, are a class of artificial neural networks in which information moves in only one direction: from input through one or more hidden layers to the output, with no cycles or recurrent connections. Each layer consists of multiple neurons that apply a nonlinear activation to a weighted sum of outputs from the previous layer. In a typical architecture, an input layer accepts feature values, hidden layers extract representations, and an output layer produces predictions.

In fully connected FFNs, every neuron in a given layer connects to every neuron in the preceding layer. Common activation functions include rectified linear units (ReLU), sigmoid, tanh, and, for multiclass classification, softmax at the output. The size and depth of the network determine its capacity, with deeper networks able to capture more complex patterns.

Training is performed by supervised learning using backpropagation and gradient-based optimization. The loss function depends on the task, such as mean squared error for regression or cross-entropy for classification. Optimizers like stochastic gradient descent, Adam, or RMSprop adjust weights to minimize the loss. Regularization methods such as L1/L2 penalties, dropout, and early stopping help prevent overfitting.

Variants and theory: multilayer perceptrons (MLPs) are a common form of FFN. A single hidden layer with enough units can approximate any continuous function under certain conditions (the universal approximation theorem). In practice, deep FFNs with many hidden layers learn hierarchical representations but require large datasets and computational resources.

Applications include tabular data classification and regression, function approximation, and as components within larger architectures. FFNs are less suited to sequence or spatial data unless designed with specialized preprocessing or combined with other layers.