samplebatch

SampleBatch is a term used in reinforcement learning to describe a data structure that holds a batch of experience samples collected from interactions between agents and their environment. It serves as a lightweight container for the information needed to train policy and value networks. A typical sample batch includes fields such as observations, actions, rewards, next observations, and done flags, along with optional auxiliary data like action probabilities, infos, or agent identifiers in multi-agent settings.

In practice, a sample batch is implemented as a dictionary-like object or a small class that stores arrays or tensors for each field. It is designed to be easily batched, sliced, concatenated, and converted to formats suitable for machine learning frameworks (e.g., PyTorch or TensorFlow). Sample batches are produced by rollout or replay components and are consumed by training loops to perform gradient updates. They can represent data from a single agent or multiple agents, and may include episode boundaries or sequence lengths for recurrent models.

Design considerations for sample batches emphasize flexibility and efficiency. They must accommodate on-policy and off-policy algorithms, discrete and continuous action spaces, and varying episode lengths. Some implementations separate data per agent or per environment, while others aggregate data in a unified batch. Compatibility with library-specific utilities for batching, shuffling, and tensor conversion is a common goal, enabling smooth integration into training pipelines.