The DataComp-12M dataset is an enhanced image-text dataset released by Apple in 2024. Its associated paper can be found at 「MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training」. It aims to improve the training efficiency and performance of image-text models through multimodal reinforcement learning strategies.

This dataset consists of UIDs for 12 million samples drawn from the DataComp-1B-BestPool subset and serves as part of the DataCompDR series. DataCompDR enhances raw DataComp data using multimodal dataset reinforcement techniques, optimizing the training process via synthetic caption generation and image augmentation. Experiments demonstrate that image-text models trained on DataCompDR-12M significantly outperform benchmarks such as CC-12M, YFCC-15M, and DataComp-Small/Medium across tasks including zero-shot classification, achieving up to 10x–1,000x improvements in sample efficiency. The underlying images and texts originate from DataComp under the CC-BY-4.0 license, while metadata is published by Apple under the Apple ML Research License (AMLR).

Dataset Composition

The dataset primarily includes a list file containing UIDs used to identify individual samples within DataComp-12M. Specific components include:

• uids.txt: A plain-text file listing all 12,779,520 unique identifiers, one per line.
• uids.npy: A binary array stored in .npy format holding these same IDs; its dtype corresponds to numpy's unsigned integer type (uint64).

These entries map directly onto corresponding shard-level records present both inside the base DataComp archive itself along side their respective counterparts residing elsewhere — ensuring consistency & traceability throughout downstream pipelines alike!

Citation

@InProceedings{mobileclip2024,
  author = {Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vemulapalli, Oncel Tuzel},
  title = {MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2024},
}