SqueezeandExcitation

Squeeze-and-Excitation (SE) is a neural network architecture component introduced to improve channel‑wise feature recalibration in convolutional neural networks. The method was first presented in the 2017 paper “Squeeze‑and‑Excitation Networks” by Jie Hu, Li Shen and Gang Sun, and quickly became a standard building block for image‑recognition models.

The SE module consists of three steps. In the squeeze phase, global average pooling aggregates spatial information from each feature map into a single scalar, producing a channel descriptor. The excitation phase passes this descriptor through a small bottleneck of two fully connected layers with a non‑linear activation (typically ReLU) followed by a sigmoid, generating per‑channel modulation weights in the range (0,1). Finally, the original feature maps are re‑scaled (excited) by multiplying each channel by its corresponding weight, allowing the network to emphasize informative features and suppress less useful ones.

SE blocks can be inserted into many existing architectures, such as ResNet, Inception, and MobileNet, often yielding noticeable gains in top‑1 accuracy on ImageNet with modest increases in computational cost. Variants of the original design include the SE‑ResNeXt, SE‑DenseNet, and “Concurrent Spatial and Channel Squeeze‑and‑Excitation” (scSE) which extend the idea to spatial attention. More recent work integrates SE concepts with attention mechanisms, transformer‑based models, and lightweight networks for mobile applications.

The success of squeeze‑and‑excitation stems from its simplicity, ease of integration, and ability to improve representational power without substantial redesign of the base network. It has been adopted in computer vision tasks beyond classification, including detection, segmentation, and video understanding, and continues to inspire research on adaptive feature modulation.