SVG Benchmark is a large model static image generation evaluation dataset released by Rapidata in 2025. It aims to compare the ability of 30 state-of-the-art large language models to generate static SVGs based on text prompts through human evaluation. The data was collected through pairwise comparisons, including 500 English prompts from human-written or publicly available datasets, 188,754 image comparison samples, and 1,355,161 preference responses based on human voting. The evaluation models covered 30 mainstream models, including Claude, GPT, Gemini, DeepSeek, Qwen, and Kimi. The evaluation dimensions were alignment, coherence, and preference, all based on human voting aggregation, without the use of automated metrics.

Dataset composition:

The dataset received a total of 1,355,161 human votes, of which:

• Subjective preferences: 451,452 times
• Match rate: 451,603 times
• Continuity: 452,106 times

Data Fields:

• prompt: String, the original text prompt used to generate the image.
• image1 / image2: Images, model 1 and model 2 generated SVG rasterized PNG files.
• model1 / model2: Strings representing the model identifiers for the corresponding images (model1 is always the model whose name comes first in alphabetical order).
• weighted_results_image*_preference / coherence / alignment: A floating-point number representing the model's aggregate score (0 to 1) across the three dimensions.
• detailed_results_*: String, JSON format, containing detailed records of each manual vote and voter demographic information for this dimension.
• Note: Some samples may only participate in single or two-dimensional evaluation; fields not participating in the evaluation may have null values.

  

  Citation

@dataset{yupp_svg_2025,
title={Yupp SVG Dataset: Exploration of the Reasoning and Coding Abilities of Frontier Models},
author={Yupp AI},
year={2025},
url={https://huggingface.co/datasets/yupp/yupp-svg-20251204}
}