IFStruct v1.0 is a structured output compliance benchmark dataset released by Liquid AI in 2026. It is designed to systematically evaluate the ability of large language models (LLMs) to generate structured outputs (JSON/YAML) that conform to a specified schema. This dataset contains 2,000 prompts. Each prompt requires the model to generate several instances based on the target pattern. The prompts are presented in various styles, including natural dialogue, explicit path instructions, raw JSON schema, code block requirements, and no additional text, all accompanied by strict underlying schema validation specifications. This test focuses only on output format and structural compliance, and does not assess content quality, semantic correctness, or inference accuracy. The scoring criterion uses a strict binary decision (pass/fail), with a pass only awarded if the model's response fully satisfies all structured constraints. The scoring difficulty has been calibrated to effectively differentiate between low-to-medium capability models and state-of-the-art models.

Data Fields:

• doc_id: Identifier (0–1999).
• entity_type: Schema classification system (e.g., test_recipe).
• prompt: The specific prompt word entered.
• output_format: The required output format (json or yaml).
• top_level_count: The required number of instances, in integer or range format (JSON encoded string).
• top_level_key: Top-level wrapper key name
• require_wrapper_key: A boolean value indicating whether the top layer must be a wrapper object containing an array.
• require_code_block: A boolean value indicating whether the response must be wrapped in a code block.
• require_no_commentary: A boolean value indicating whether to prohibit text outside the payload.
• json_schema: The JSON Schema specification (JSON encoded string) that the model output must meet.

Citation

@article{liquidAI2026IFStruct,
author = {Liquid AI},
title = {IFStruct: Measuring structured-output compliance},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/ifstruct-v1.0}
}