audioset

AudioSet is a large-scale dataset for audio event detection and tagging released by Google Research in 2017. It aggregates audio from YouTube videos and provides human-labeled annotations for a broad ontology of sound events. The dataset contains over 2 million 10-second audio clips drawn from YouTube and labeled with one or more sound classes from the AudioSet Ontology.

Ontology and labels: The AudioSet Ontology contains 632 classes, of which 527 are leaf nodes, organized in a hierarchical structure. Labels are provided at clip level (weak labeling), indicating which events are present in a clip but not the exact start and end times of each event. The dataset thus enables multi-label tagging and learning across a wide range of sounds.

Data splits and evaluation: AudioSet provides training, validation, and test partitions for evaluation. Models are typically evaluated on clip-level predictions using mean average precision (mAP) or related metrics, reflecting the ability to detect and classify events within clips.

Usage and impact: Since its release, AudioSet has become a widely used benchmark for audio tagging and weakly supervised sound event detection. It has spurred development of deep learning architectures such as convolutional neural networks, recurrent networks, and attention-based models applied to audio. Researchers often use it to pretrain models or as a component in larger systems, sometimes in combination with other datasets.

Limitations: The dataset relies on YouTube sources, and annotations are crowdsourced, which introduces label noise and potential biases. Its clip-level labeling provides limited temporal precision, making fine-grained localization challenging without additional labeling or methods.

---