SumnNN

SumnNN is a term used in machine learning to describe a family of neural network architectures that foreground summation as the primary mechanism for combining inputs. In typical sumnNN designs, neurons aggregate signals by forming a simple or weighted sum of their inputs, often followed by a normalization or nonlinearity. This emphasis on additive aggregation contrasts with conventional networks that rely heavily on multiplicative interactions or dot products.

Variants differ in how the sum is processed: some use a fixed, non-trainable sum with a separate normalization layer; others apply learned scaling or gating to each input before summing. Some proposals incorporate residual-like connections to preserve information across layers and stabilize gradients.

The term is not canonical; sumnNN vocabulary appears in a small set of theoretical discussions and preliminary experiments, without a single agreed definition. It is primarily discussed as an exploration of alternative input-aggregation operators in neural computation.

Advantages cited include potential improvements in training stability and interpretability of input contributions. Limitations include reduced expressivity for certain functions unless augmented with nonlinearities, gating, or normalization, and sensitivity to input scale if not properly regularized.

Related concepts include sum-product networks and various normalization-augmented architectures.