CudaSpace

CudaSpace is a software framework and library that provides a high-level memory management layer for CUDA applications. It abstracts device memory management by offering a unified allocator, memory pools, and data movement facilities between host memory and device memory. The goal is to reduce allocation overhead, fragmentation, and boilerplate code in GPU-accelerated software.

Architecture and API: CudaSpace sits above the CUDA runtime or driver API and exposes a language-agnostic API for requesting memory blocks, freeing them, and obtaining device pointers in host code. It supports typed allocations, alignment guarantees, and per-context pools. It can be configured with policies such as pool size growth, eviction strategy, and caching of recently freed blocks. It integrates with CUDA streams to enable asynchronous copies and to overlap data transfer with computation.

Memory management features: It provides both device-side and host-side allocations, with options for pinned host memory and unified memory emulation. It supports data movement strategies, including asynchronous prefetching and page migration, to optimize data locality. It includes a memory lifetime model to avoid dangling pointers and to simplify resource cleanup.

Performance and portability: By reusing memory pools and reducing system calls, CudaSpace lowers allocation latency and improves throughput for workloads with dynamic memory requirements. It aims to be portable across CUDA-enabled GPUs and supports multi-GPU configurations and peer-to-peer memory where available.

Use cases and adoption: Commonly used in high-performance computing, scientific simulation, machine learning inference and training pipelines, and real-time data processing where GPU memory is a critical resource.