BNN

Bayesian neural networks (BNNs) are neural networks in which the network parameters, typically weights and biases, are treated as random variables with prior distributions. This yields probabilistic predictions and a distribution over functions rather than single point estimates.

In a BNN, the goal is to compute the posterior distribution over weights given data, p(w|D) ∝ p(D|w) p(w). Predictions for a new input x* are obtained by averaging over the posterior: p(y*|x*, D) = ∫ p(y*|x*, w) p(w|D) dw. The integral is generally intractable, so approximate inference is used.

Common approaches include variational inference with tractable approximations, Markov chain Monte Carlo, and Laplace or other second-order methods. A widely used practical technique is Monte Carlo dropout, which approximates Bayesian inference by sampling networks with dropout at prediction time.

BNNs offer calibrated uncertainty and robustness to overfitting, with predictions that express uncertainty. They face challenges such as higher computational cost, dependence on prior choices, and the difficulty of scaling to very deep architectures or large datasets.

Historically, Bayesian neural networks emerged from work in the 1990s and 2000s (notably by Radford Neal) and gained new momentum with deep learning-era variational methods. Applications span robotics, healthcare, finance, active learning, and risk-aware decision making.

BNNs are related to Gaussian processes in the limit of infinite width and are often contrasted with standard neural networks that use point estimates of weights. They remain an active area of research in uncertainty quantification and reliable AI. Note: BNN may be used to refer to other organizations or networks in different contexts; this article discusses Bayesian neural networks.