LCC_python is a Python code dataset released by Microsoft in 2023, derived from the Large Code Corpus (LCC) project. It aims to provide high-quality code corpora for tasks such as code generation, code completion, and code understanding.

The dataset contains 120,000 Python code samples sourced from large open-source code repositories and is divided into training, validation, and test sets. Each sample primarily consists of context code and the corresponding target code snippet, which can be used for supervised learning and pre-training of code-related models.

Dataset Composition

The dataset consists of the following three parts:

• Training set (train): 100,000 samples, approximately 1.76 GB.
• Validation set (validation): 10,000 samples, approximately 146 MB.
• Test set (test): 10,000 samples, approximately 149 MB.

Each sample primarily contains the following fields:

• context: Code context information.
• gt: Ground Truth, i.e., the target code or label.