OpenCodeInstruct is a large-scale code instruction-tuning dataset comprising 5 million diverse samples, released by NVIDIA in 2025. The associated research paper is available at OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs. It aims to provide high-quality data for Supervised Fine-Tuning (SFT) to improve open-source code large language models.

The dataset is specifically designed for training language models focused on code generation, employing a hybrid approach of automated and synthetic methods for data collection and annotation.

Dataset Composition

The dataset primarily consists of a training set (train) with 5 million samples. Each sample contains the following fields:

• id: Identifier for each question (string)
• input: The input programming question (string)
• output: The response from the large language model (string)
• domain: Domain type, categorized as either "generic" or "algorithmic" (string)
• generation_algorithm: Generation algorithm, categorized as either "self-instruct" or "evol-instruct" (string)
• llm_judgement: A string representation of a JSON dictionary containing the LLM's evaluation of the response based on multiple criteria
• unit_tests: A string representation of a list of assertion statements
• tests_execution_status: A list of strings indicating "pass" or "fail", representing the execution status of the tests
• average_test_score: The ratio of passed test cases (float)

Citation

@article{ahmad2025opencodeinstruct,
      title={OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs}, 
      author={Wasi Uddin Ahmad and Aleksander Ficek and Mehrzad Samadi and Jocelyn Huang and Vahid Noroozi and Somshubra Majumdar and Boris Ginsburg},
      year={2025},
      eprint={2504.04030},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.04030}, 
}