HelpSteer3 is a human preference dataset released by NVIDIA in 2025. The related paper results are "HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages", which aims to improve the model's responsiveness to user prompts through human feedback and reinforcement learning techniques. The dataset contains 40,476 preference samples, each of which includes a domain, language, context, two responses, an overall preference score between the two responses, and personal preference scores from up to three annotators. It includes multilingual data (Chinese, Korean, French, Spanish, Japanese, German, Russian, Portuguese, Italian, Vietnamese, and Dutch).

Citation

@misc{wang2025helpsteer3preferenceopenhumanannotatedpreference,
title={HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Hoo-Chang Shin and Felipe Soares and Alexander Bukharin and Ellie Evans and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2505.11475},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2505.11475},
}
@misc{wang2025helpsteer3humanannotatedfeedbackedit,
title={HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Daniel Egert and Ellie Evans and Hoo-Chang Shin and Felipe Soares and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2503.04378},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2503.04378},
}
@misc{wang2025rlbffbinaryflexiblefeedback,
title={RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards},
author={Zhilin Wang and Jiaqi Zeng and Olivier Delalleau and Ellie Evans and Daniel Egert and Hoo-Chang Shin and Felipe Soares and Yi Dong and Oleksii Kuchaiev},
year={2025},
eprint={2509.21319},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2509.21319},
}