MMPR (Multimodal Preference Dataset) is a large-scale multimodal preference dataset jointly released in 2024 by research teams from Shanghai Artificial Intelligence Laboratory, Fudan University, Nanjing University, Chinese University of Hong Kong, Tsinghua University and SenseTime. The related paper results are "Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization". The dataset contains 750,000 samples without clear correct answers and 2.5 million samples with clear correct answers. The samples cover multiple fields such as VQA, science, diagrams, mathematics, OCR, and documents to ensure diversity. When constructing the dataset, the researchers paid special attention to avoiding false positive negative responses due to the limitations of heuristic rules, especially in the fields of general VQA and documents. The dataset is designed to improve the performance of the model in multimodal reasoning tasks while avoiding potential negative effects during training.

Citation

If you find this project useful in your research, please consider citing: “`BibTeX @article{wang2024mpo, title={Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization}, author={Wang, Weiyun and Chen, Zhe and Wang, Wenhai and Cao, Yue and Liu, Yangzhou and Gao, Zhangwei and Zhu, Jinguo and Zhu, Xizhou and Lu, Lewei and Qiao, Yu and Dai, Jifeng} journal={arXiv preprint arXiv:2411.10442}, year={2024} } @article{chen2023internvl, title={InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks}, author={Chen, Zhe and Wu, Jiannan and Wang, Wenhai and Su, Weijie and Chen, Guo and Xing, Sen and Zhong, Muyan and Zhang, Qinglong and Zhu, Xizhou and Lu, Lewei and Li, Bin and Luo, Ping and Lu, Tong and Qiao, Yu and Dai, Jifeng} journal={arXiv preprint arXiv:2312.14238}, year={2023} } @article{chen2024far, title={How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites}, Authors: {Chen, Zhe and Wang, Weiyun and Tian, Hao and Ye, Shenglong and Gao, Zhangwei and Cui, Erfei and Tong, Wenwen and Hu, Kongzhi and Luo, Jiapeng and Ma, Zheng and others} journal={arXiv preprint arXiv:2404.16821}, year={2024} }