One-sentence Summary

Beijing Academy of Artificial Intelligence researchers propose AREX, a family of recursively self-improving agents that alternate evidence gathering and constraint-wise auditing, learn to autonomously compress interaction history into a compact improvement state without external models, and are trained via agentic mid-training and long-horizon reinforcement learning emphasizing key steps where decisive evidence is acquired or erroneous directions are corrected, achieving strong performance on BrowseComp, WideSearch, DeepSearchQA, and Humanity’s Last Exam (HLE) with a dense 4B model and a 122B-A10B Mixture-of-Experts model while remaining competitive with much larger models.

Key Contributions

• Multi-constraint deep research is formulated as a recursively self-improving process that exploits the asymmetry between costly discovery and tractable verification, recursively turning partially verified solutions into better-targeted research problems.
• The AREX agent architecture pairs an inner research loop with an outer self-improvement loop that verifies constraints, flags unresolved claims, and uses a learned context-update tool to autonomously compress interaction history into a compact improvement state.
• A multi-stage training framework emphasizes critical intervals where decisive evidence is acquired or errors are corrected, and dense 4B and 122B-A10B MoE instances of AREX outperform comparable-scale baselines on deep search, wide search, reasoning, and tool-use benchmarks.

Introduction

Deep research tasks require agents to jointly satisfy multiple constraints, making them difficult to solve by simply extending a single search trajectory. Prior systems add more reasoning or tool calls, but this often reintroduces early errors, re-explores exhausted directions, or prematurely accepts partially valid answers. The deeper challenge is not searching longer, but identifying which constraints remain unresolved and using that diagnosis to drive targeted follow-up investigation. The authors observe a discovery–verification asymmetry: verifying a candidate answer can be broken into simpler, constraint-level checks that reveal unsupported or conflicting claims. They leverage this to build AREX, a recursively self-improving agent that treats verification as an active control signal. AREX alternates between an inner research loop that constructs provisional answers and an outer loop that audits those answers constraint by constraint, preserving verified evidence and converting unresolved gaps into refined research problems. An autonomous context-update tool compacts the agent’s history into a structured improvement state, and a multi-stage training pipeline with step-level credit assignment enables long-horizon recursive improvement.

Dataset

The authors construct a specialized training dataset to teach AREX recursive research behaviors. It is built in two stages: first, they synthesize challenging research tasks; second, they collect high‑quality solution trajectories from strong teacher models operating in the same environment.

• Task dataset (D_task)

  • Sources and templates: Human experts define templates for three categories—browse‑intensive, reasoning‑intensive, and scientific‑literature problems. Concrete instances are generated from real‑world web pages, academic papers, structured knowledge bases, and public repositories.
  • Synthesis process: For each task, a hidden answer is paired with multiple verifiable constraints (e.g., temporal, numerical, relational). The constraints are transformed into indirect, multi‑hop descriptions that prevent shallow keyword matching.
  • Filtering: Automatic checks enforce correctness, uniqueness, evidence availability, and difficulty. Tasks that are ambiguous, have inconsistent constraints, or can be solved by simple retrieval / remain unsolved after extensive rollouts are discarded.
  • Result: D_task = {(x, y)} where x is a research query and y is the ground‑truth answer.

• Teacher trajectory dataset (D_traj)

  • Collection: For each synthesized task, a teacher model interacts with the same tools as AREX and samples a full research trajectory (actions, tool calls, retrieved observations, intermediate reasoning, and final answer).
  • Quality control:
    • Keep only trajectories that show iterative investigation, coherent state maintenance, and adaptation to new evidence. Trajectories that guess immediately, ignore observations, or fail to revise wrong hypotheses are removed.
    • Discard trajectories with invalid tool interactions, ignored observations, or unsupported references.
    • Require the final answer to be reconstructable from collected evidence; remove those that answer before sufficient evidence, contain unsupported claims, contradict retrieved information, or rely on unjustified assumptions.
    • Apply a confidence threshold: trajectories with a confidence score below a predefined level are dropped.
  • Result: D_traj = {(x, τ)} where τ is a validated, evidence‑grounded trajectory.

• Usage: The trajectory dataset directly supervises AREX’s recursive skills—iterative evidence acquisition, research‑state maintenance, evidence‑grounded answer construction, and adaptive continuation. No explicit training split or mixture ratio is provided in the text; the construction focuses on quality over quantity, with sizes not numerically reported. The data is used to train the model to operate within the research environment’s double‑loop framework.

Method

The authors propose AREX, a deep research framework that implements a hierarchical, bi-level recursive self-improvement process. This architecture comprises an inner research loop and an outer self-improvement loop to iteratively gather evidence, synthesize answers, and refine the research trajectory.

Given an input query, the system first derives an initial research objective. The inner research loop executes research actions, integrates retrieved evidence, and updates the current answer until the objective is sufficiently addressed. It then outputs a provisional answer along with supporting evidence and an answer-level confidence score. The outer self-improvement loop evaluates this provisional result. If the confidence score exceeds a predefined threshold, the answer is accepted. Otherwise, the loop assesses whether the current research trajectory is recoverable. Useful findings are preserved and converted into a targeted objective for refinement, while noisy or uninformative trajectories trigger a restart from the original problem.

The inner research loop incrementally constructs an answer by analyzing the current research objective, invoking search or browsing tools, and integrating the returned evidence. At each step, the interaction trajectory serves as the model's working research state. The research policy adapts as new evidence is accumulated, guiding the model toward remaining constraints or redirecting the investigation when sources conflict. To manage long-horizon research without losing critical information, the authors introduce an autonomous context updating mechanism. Instead of relying on fixed heuristics, the model autonomously invokes an update context tool to consolidate the accumulated trajectory into a refreshed research state. This state preserves verified findings, current candidates, unresolved constraints, and the next-step plan, while removing redundant observations. Once the objective is sufficiently investigated, the model invokes a structured finish interface to externalize a provisional answer, supporting evidence, and a confidence score reflecting the completeness and consistency of the answer.

The outer self-improvement loop receives the structured result and decides whether to terminate, refine, or restart. If the confidence score is below the threshold, the model evaluates the trajectory's recoverability. For a recoverable trajectory, the outer loop preserves reliable findings, identifies remaining problems, and formulates a targeted objective for the next recursive round, initializing the inner loop from a refreshed state. If the trajectory is deemed too noisy or misleading, it is discarded, and the next round is initialized solely from the original problem. This process repeats until an answer reaches the confidence threshold or a maximum number of rounds is exceeded.

To enable this recursive research capability, the authors construct a specialized training dataset through recursive research task synthesis and teacher trajectory collection. Task synthesis generates verifiable problems requiring multi-source evidence integration and adaptive planning by abstracting constraints into indirect descriptions. Teacher trajectory collection involves strong models interacting with the research environment, followed by rigorous quality control to retain only trajectories with meaningful iterative investigation, valid tool interactions, and evidence-grounded answers.

The training pipeline consists of multi-stage agentic mid-training and step-aware reinforcement learning. The mid-training process progressively establishes fundamental tool-use and evidence acquisition capabilities through browse-intensive tasks, followed by expert-level reasoning tasks to strengthen long-form thinking. To mitigate capability interference, a mixed-capability consolidation stage replays key steps from long-horizon trajectories and incorporates complex academic research tasks. The authors introduce key-step focused supervision, identifying critical research decisions such as discovering answer-relevant evidence or redirecting search. By applying loss only to these high-value decision points while preserving the preceding context, the model better learns the most informative parts of the trajectory.

Building on this, the step-aware reinforcement learning design addresses the heterogeneous nature of long-horizon trajectories. Instead of propagating a sequence-level reward uniformly, the authors adopt a turn-level policy optimization formulation with hierarchical step-balanced normalization. This prevents longer trajectories from dominating the training objective. Additionally, step-level reward shaping is introduced to emphasize informative intermediate decisions. Annotated key steps in successful trajectories receive a bounded auxiliary bonus, providing high-precision supervision for decision-critical steps while retaining final-answer correctness as the primary optimization objective.

Experiment

AREX is evaluated across six benchmarks spanning deep research, agentic task completion, broad retrieval, and high-level reasoning with tools, using a unified agent interface that supports iterative search, context updating, and structured final answers. The framework achieves strong results against larger models and specialized research agents, with autonomous context updating proactively consolidating verified findings and unresolved constraints, and an outer self-improvement loop that refines low-confidence outputs to substantially boost accuracy. Ablation studies confirm that progressive multi-round capability training, supervision concentrated on decision-critical steps, and step-aware reinforcement learning each play a crucial role in establishing the model’s long-horizon research behaviors and overall performance.

AREX-Base achieves competitive performance across deep search and agentic reasoning tasks, surpassing several larger open-source models and rivaling proprietary frontier systems. It outperforms Kimi-K2.6 on GAIA and WideSearch-en, and matches or exceeds specialized agents like MiroThinker-H1 and DeepSeek-V4-Pro on DeepSearchQA and text-only HLE. Frontier models such as Gemini-3.1-Pro and Opus-4.6 still lead on BrowseComp and DeepSearchQA. AREX-Base exceeds Kimi-K2.6 on GAIA and WideSearch-en while using only 10B active parameters. Frontier models like Gemini-3.1-Pro (85.9) and Opus-4.6 (83.7) achieve the highest BrowseComp scores. AREX-Base outperforms the Qwen3.5-397B model despite having substantially fewer parameters. Kimi-K2.6 shows strong open-source results, particularly on xbench-2510 (90.0) and WideSearch-en (80.8).

AREX invokes update_context in 80.3% of BrowseComp cases, with updates occurring at a mean context size of 25,721 tokens, well below the 128K window limit. The leading trigger is revising the search strategy (66.9%), and the updated state nearly always preserves unresolved constraints (95.5%) and a next-step plan (96.4%), while also capturing verified findings (72.1%) and rejected candidates (81.5%). This pattern shows that the agent updates context proactively to consolidate research, rather than waiting until the context is full. Context updates are triggered proactively: the median context size at update is 25,386 tokens, far below the 128K hard limit, and only 0.01% of updates occur at or above that limit. Updated context reliably retains unresolved constraints (95.5%) and a next-step plan (96.4%), and frequently includes verified findings (72.1%) and rejected candidates (81.5%), reflecting a structured research state.

Removing either Autonomous Context Updating (ACU) or the outer self-improvement loop substantially degrades BrowseComp accuracy. Adding ACU alone lifts accuracy from 59.6 to 71.4, while adding the outer loop without ACU lifts it to 69.8; combining both pushes accuracy to 82.5, indicating complementary gains. Without ACU, enabling the outer loop improves accuracy by 10.2 points (59.6 to 69.8). With ACU already active, adding the outer loop yields an additional 11.1-point gain (71.4 to 82.5).

Removing any component of the AREX training recipe degrades BrowseComp accuracy, confirming that progressive multi-round capability training, key-step focused supervision, and step-aware reinforcement learning each provide meaningful gains. The largest drop occurs when key-step supervision is replaced with random-step replay, underscoring the importance of concentrating supervision on difficult decision-critical intermediate actions. Progressive ordering and step-aware RL further contribute, with the full recipe achieving the strongest result. Replacing key-step focused supervision with equal-budget random-step replay causes the largest accuracy decline, demonstrating that routine steps are easily learned while underlearned, decision-critical steps benefit disproportionately from targeted supervision. Progressive multi-round training, which sequences browse-intensive then reasoning-intensive data, outperforms mixed training, suggesting that staged acquisition reduces interference between tool-use and reasoning objectives.

AREX-Base achieves competitive performance across deep search and agentic reasoning tasks, surpassing larger open-source models and rivaling proprietary frontier systems despite using substantially fewer active parameters. Its Autonomous Context Updating mechanism triggers proactively to consolidate research state, while an outer self-improvement loop provides complementary gains. Ablations confirm that the training recipe’s progressive multi-round capability building, key-step focused supervision, and step-aware reinforcement learning each contribute meaningfully, with targeted supervision on decision-critical steps proving especially important.