nuScenes

nuScenes is a large-scale autonomous driving dataset and benchmark designed to advance 3D computer vision, perception, and multimodal sensor fusion research. Released in 2019, it provides a curated, multimodal dataset intended to support development and evaluation of perception systems in realistic driving scenarios.

Data collection and content: The dataset features driving scenes collected in multiple cities, notably Boston and Singapore. Each scene is captured with a full ego-vehicle sensor suite, including calibrated cameras arranged for wide field of view, and LiDAR sensors, with additional radar data available in some configurations. The data are accompanied by precise sensor calibration, ego-vehicle poses, and HD semantic maps describing drivable areas, lanes, and related road features. The collection emphasizes urban and suburban environments, varied weather, and diverse traffic conditions.

Annotations and tasks: nuScenes provides rich annotations for perception research, including 3D object bounding boxes with class labels, attributes, and velocity estimates, along with unique object IDs for tracking across frames. Scene-level annotations cover maps and road elements, supporting localization and map-based tasks. The dataset supports multiple tasks such as 3D object detection, tracking, and panoptic segmentation, with data organized into train, validation, and test splits for benchmarking.

Evaluation and benchmarks: Evaluation uses standardized metrics, notably a 3D detection performance measure (mAP) and a composite nuScenes Detection Score (NDS) that combines precision and recall across classes. Additional benchmarks exist for tracking and other perception challenges, enabling researchers to compare methodologies on common baselines.

Impact and availability: nuScenes has become a widely used benchmark in autonomous driving research, providing a comprehensive, publicly available platform for developing and comparing perception algorithms. It is available for research use under specified terms and has influenced a range of subsequent datasets and evaluation protocols.