EditReward-Bench is a systematic evaluation benchmark for image editing reward models, jointly released by the University of Science and Technology of China, the Institute of Automation of the Chinese Academy of Sciences, and the Beijing Academy of Artificial Intelligence. The related paper is titled "EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward ModelingThe goal is to comprehensively evaluate the discriminative ability of reward models from three core dimensions: instruction compliance, consistency maintenance, and overall quality.

The dataset contains 3,072 expert-annotated preference comparison data points, covering 4 major categories and 13 representative image editing tasks, comprehensively encompassing common and complex real-world application scenarios. All candidate editing results were generated by 11 heterogeneous image editing models, including open-source systems and leading proprietary editors, ensuring the richness and challenge of the data distribution.

Task Category:

• Subject class: Subject addition, subject removal, subject replacement
• Appearance-related modifications: color modification, material change, style transfer, color tone adjustment.
• Scene-related: Background replacement, foreground extraction
• Advanced editing categories: portrait enhancement, text editing, motion manipulation, and mixed editing.