The Drive&Act dataset is a state-of-the-art multimodal benchmark for action recognition of drivers in moving vehicles. The dataset includes 3D skeletons and frame-level hierarchical annotations of 9.6 million frames captured from 6 different viewpoints and 3 modalities (RGB).

The dataset has the following characteristics:

• Includes 12 hours of video data, a total of 29 long sequences;

• A calibrated multi-view camera system with 5 views;

• Multimodal video: RGB, IR and depth;

• Markerless Motion Capture: 3D Body and Head Pose

• 83 manually annotated hierarchical activity annotations.
  – Level 1: Long-running tasks (12).
  – Level 2: Semantic Behavior (34)
  – Level 3: Object interaction triad (behavior, object, place) (6|17|14).