mikrobatching
MikroBatching is a technique in data processing and machine learning that groups incoming data into small, fixed-size batches for processing as a unit. It sits between true streaming, which processes records as they arrive, and traditional batch processing, which operates on large data sets at longer intervals. In streaming systems, a micro-batch is formed from data collected over a short window and then processed as a single unit. In machine learning workflows, mikro-batching refers to using small batch sizes to feed computation devices (GPUs, TPUs) to balance latency, throughput, and memory usage.
The approach aims to amortize startup and data-transfer overhead, improve utilization of vectorized computation, and provide
Challenges include increased scheduling and windowing complexity, potential latency variability if window sizes are not well
Typical applications are real-time or near-real-time analytics, streaming data pipelines, and online inference systems. In training