MoCapAct is a dataset released by Microsoft Research in 2024 for simulated humanoid robot motion control, designed to provide pre-trained expert policies and their demonstration data for humanoid robot motion control research, reducing the computational cost of training low-level control policies from motion capture data.

The dataset contains pre-trained expert agents capable of tracking over 3 hours of motion capture data in the dm_control physical simulation environment, as well as replay data generated by these expert agents. The replay data includes proprioceptive observations and actions, covering humanoid motor skills such as standing, walking, and running, which can be used for imitation learning, reinforcement learning, and high-level motor behavior learning tasks.

Dataset Composition

• all: Complete expert replay data, available in two scales: large and small. The large data contains 200 replays per segment, while the small data contains 20 replays per segment. The data is provided in multiple .tar.gz files, which contain HDF5 data files after decompression.

• sample: Sample data for quick testing and data reading, containing large.tar.gz and small.tar.gz, corresponding to 200 and 20 replays, respectively.

• videos: Motion capture video data, including full_clip_videos.tar.gz and snippet_videos.tar.gz, which provide full motion capture clips and snippet videos used for training expert agents, respectively.