Madaline

MADALINE is a family of early neural networks built from multiple Adaptive Linear Neuron (ADALINE) units. The term is commonly expanded as Many ADALINEs or Multiple ADALINE, and the architecture originated with researchers at Stanford in the 1960s. MADALINE networks are designed for pattern recognition and associative memory tasks by arranging several ADALINE units in layers whose outputs feed into one or more final decision units.

The characteristic MADALINE structure typically includes a hidden layer (or layers) of ADALINE neurons and an output stage that combines their signals to produce a decision. Each ADALINE unit uses a linear activation and is trained with a delta-rule-like method to minimize squared error. Training MADALINE networks employs specialized algorithms known as MADALINE Rule I and MADALINE Rule II. Rule I emphasizes selective updates to hidden units to correct misclassifications while preserving previously learned mappings. Rule II extends the approach to adjust both hidden and output weights to improve learning in more complex tasks.

Two well-known variants are MADALINE I (M1) and MADALINE II (M2). MADALINE I typically incorporates a single hidden layer of ADALINE units with a straightforward output stage, while MADALINE II introduces refinements intended to improve learning stability and performance on certain pattern-recognition problems. Historically, MADALINE demonstrated capabilities on simple pattern recognition and character recognition tasks and contributed to early discussions of multi-layer networks and rule-based training.

Today, MADALINE is viewed mainly as a historical and educational example, illustrating early ideas about modular networks and supervised learning before backpropagation became the dominant training paradigm.