RedlineBench is a legal contract negotiation evaluation benchmark dataset released by Crosby in 2026. It aims to measure the ability of AI agents to make red-line annotations and negotiation decisions in real business transaction scenarios. It is widely used in legal agent evaluation, automated red-line modification of contracts, multi-round negotiation decision modeling, and the ability assessment of large language models in real business workflows. The dataset contains 140 runnable Harbor tasks covering three multi-round negotiation scenarios (two SaaS MSA scenarios and one professional services MSA scenario), with each scenario consisting of four alternating rounds. The three scenarios revolve around AgentCo, a Series A HR technology company, and the negotiation content is a SaaS master service agreement. Each round involves an alternating shift in positions; each scenario shares the same page of business background information, but the specific background information, negotiation manuals (strategy guidelines), initiating documents, negotiating stances, and stakes of each party differ.