NCHW

NCHW is a four-dimensional tensor layout used in computer vision and deep learning. The dimension order is N, C, H, W, where N is the batch size, C the number of channels, H the height, and W the width. It is commonly used to represent a batch of N images with C channels each, each image sized H by W. The counterpart NHWC uses the order N, H, W, C.

Origin and usage: NCHW has been the de facto default in several frameworks and libraries, including Caffe and PyTorch, and is widely supported by cuDNN. TensorFlow typically uses NHWC by default, but can operate with NCHW via data_format options. Some libraries allow memory formats that change contiguity without changing the logical shape.

Memory layout and performance: For a tensor with shape [N, C, H, W], the strides are [C*H*W, H*W, W, 1], so W elements are contiguous. This layout can improve performance of convolution kernels on GPUs where channel-wise operations are optimized, but performance is architecture dependent. Interconverting between NCHW and NHWC can incur cost.

Examples and notes: Use NCHW for 32 images with 3 channels of size 224x224: shape [32, 3, 224, 224]. Always ensure consistency between input data, model weights, and operations across frameworks to avoid errors.