RBM

RBM stands for Restricted Boltzmann Machine, a generative stochastic neural network that learns a probability distribution over its inputs. It consists of two layers: a visible layer representing observed data and a hidden layer that captures dependencies. The network is a bipartite graph with no connections within a layer, and the units are typically binary stochastic neurons. The joint distribution over visible and hidden units is defined by an energy function E(v,h) = -a^T v - b^T h - v^T W h, where W is the weight matrix and a, b are biases. Because there are no intra-layer connections, the conditional distributions p(h|v) and p(v|h) are tractable, enabling efficient Gibbs sampling.

RBMs are trained to maximize the likelihood of the training data. Exact maximum likelihood is intractable, so algorithms such as contrastive divergence (CD) or persistent CD are used to approximate gradient updates. Variants include Gaussian-BRBMs for real-valued data and multinomial RBMs for categorical data. RBMs can be stacked and pre-trained in a layer-wise fashion to form deep belief networks (DBNs), which historically helped initialize deep neural networks before end-to-end training became standard.

Applications encompass unsupervised feature learning, dimensionality reduction, collaborative filtering, and data denoising. In modern practice, RBMs are less common as stand-alone models, having largely been supplanted by deeper architectures and other generative approaches, but they remain of historical and educational interest and can appear in certain unsupervised representation learning tasks.

Limitations include relatively slow training for large datasets, sensitivity to hyperparameters, and scalability challenges compared with newer models. Nonetheless, the energy-based formulation and training objectives contributed to the development of later unsupervised learning methods.