Perbatch

Perbatch is a term used in data processing and machine learning to describe a technique in which data are grouped into fixed-size batches and processed as a unit. The word is a contraction of per and batch, reflecting a design principle that emphasizes batch-oriented computation for efficiency and scalability.

In data processing, perbatch batching enables technologies to operate on collections of records at once rather than individually. This approach can improve throughput by leveraging vectorized operations, reducing scheduling overhead, and improving cache locality. Systems that ingest continuous streams or large datasets often implement perbatch strategies to balance latency and processing efficiency, grouping incoming items into batches of a chosen size before applying transformations or writes.

In machine learning, perbatch typically refers to mini-batch processing during model training. Instead of updating model parameters after every single example (online learning) or only after processing the entire dataset (batch learning), a model is updated after processing a batch of examples. This perbatch update strategy often achieves a practical balance between convergence speed and computational efficiency, enabling hardware accelerators to be used effectively and reducing the noise of single-sample updates.

Variants and considerations include fixed-size batches, dynamic or adaptive batch sizing, and decisions about when to flush a batch in streaming contexts. Key trade-offs involve batch size, memory usage, and desired latency. Larger batches improve throughput but can increase delay, while smaller batches reduce latency but may underutilize computational resources.

See also: batch processing, streaming data, mini-batch gradient descent, vectorization, cache locality.