BABEL is a large-scale dataset with linguistic annotations describing the actions taking place in mocap sequences. BABEL includes action annotations for about 43 hours of mocap sequences from the AMASS dataset. Action annotations have two levels of abstraction – sequence annotations describe the overall action in a sequence, while frame annotations describe all actions for each frame in the sequence. Each frame annotation is precisely aligned with the duration of the corresponding action in the mocap sequence, and multiple actions can overlap.

The dataset contains more than 28,000 sequence annotations and 63,000 frame annotations covering more than 250 unique action categories. BABEL’s annotations can be used for tasks such as action recognition, temporal action localization, and motion synthesis.