Command Palette
Search for a command to run...
MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use
Mengru Wang Haozhe Luo Zhenqian Xu Zhixiang Cui Haoming Xu Qu Yang Jizhan Fang Junfeng Fang Ningyu Zhang
Abstract
Memory has become a key component of large language models, enabling them to retain information and learn from longterm interactions. However, existing memory benchmarks mainly evaluate whether information is correctly extracted, stored, and retrieved, while largely overlooking how retrieved memories reshape model reasoning and affect performance on the current task. We identify memory-induced cognitive traps: even faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current task performance. To systematically evaluate these failure modes, we introduce MemTrapBench, which covers two forms of cognitive traps: Reasoning Fixation and Belief Distortion. Experiments across two model families and five representative memory frameworks show that MemTrapBench is challenging: all evaluated memory strategies underperform the nomemory setting, with even the strongest methods suffering drops of more than 10%. To mitigate these cognitive traps, we propose AdaptiveMem, a simple yet effective inference-time method that instructs LLMs to avoid memory traps. AdaptiveMem mitigates cognitive traps on MemTrapBench while preserving or improving performance on standard memory benchmarks across diverse memory frameworks.
One-sentence Summary
Researchers from Zhejiang University, National University of Singapore, Northeastern University, Heriot-Watt University, and Tencent introduce MemTrapBench, a benchmark for memory-induced cognitive traps covering Reasoning Fixation and Belief Distortion, demonstrate that across two model families and five representative memory frameworks all evaluated memory strategies underperform the no-memory setting with even the strongest methods dropping by over 10%, and propose AdaptiveMem, an inference-time method that mitigates these cognitive traps on MemTrapBench while preserving or improving performance on standard memory benchmarks.
Key Contributions
- Memory-induced cognitive traps are formalized, where faithfully recorded and semantically relevant memories can distort model reasoning or beliefs and degrade current-task performance.
- MemTrapBench is introduced to evaluate these traps through Reasoning Fixation and Belief Distortion; experiments across two model families and five memory frameworks show all evaluated memory strategies underperform the no-memory setting, with the strongest methods losing more than 10 percentage points.
- AdaptiveMem is proposed as a prompt-based inference-time method that guides LLMs to identify and account for memory traps before using retrieved memories. It improves MemTrapBench performance across memory frameworks, for example improving LightMem by 14.9 percentage points on Gemini-3-Flash-Preview without degrading general memory performance.
Introduction
The authors study memory systems for large language models, where external memory from long interaction histories is retrieved to support later queries. While prior benchmarks and analyses have largely focused on memory management failures such as outdated, incorrect, or irrelevant retrieved information, they overlook a complementary risk: even valid memory can reshape a model’s reasoning and hurt current task performance. The paper shows that memory can induce cognitive traps, for example anchoring a model to previously successful strategies and preventing it from considering new operations. To address this gap, the authors introduce MemTrapBench, a benchmark covering reasoning fixation and belief distortion, and propose AdaptiveMem, a prompt-based skill that helps models detect and account for potential memory traps before using retrieved information.
Dataset
The authors use MemTrapBench as a diagnostic benchmark for memory-induced failures in LLMs. The dataset is built from manually designed seed instances that are expanded into multi-turn dialogues and then filtered through automated and human quality control.
- Composition and scale: MemTrapBench contains 1,050 instances, split into four categories: 350 Cognitive Bias, 350 Task Boundary, 200 Safety, and 150 Trauma.
- Instance structure: Each seed instance specifies four fields: Domain, Trap Mechanism, Ground Truth, and Planted Prior. The Planted Prior records the strategy, feedback, rule, or belief introduced in the interaction history.
- Dialogue construction: GPT-5.4 expands each seed into a dialogue of 18 to 40 turns using three stages: Plant the trap, Bury it in noise, and Spring the trap. The final query stays related to the history but changes the conditions under which the prior should apply. Explicit reset cues such as "ignore previous rules" are excluded.
- Quality control: Candidates pass through automated filtering and expert human review. The checks cover topic coherence, context consistency, interaction realism, standalone solvability, and clarity of the contextual transition. Annotators verify the transition from the query alone. Each retained instance is annotated with a gold-standard response and an expected failure mode.
- Usage: The benchmark compares model responses with and without memory. A memory trap is defined as a situation where memory degrades response quality relative to the no-memory setting. Response quality is measured across correctness, format, relevance, and efficiency. Final queries are designed to be independently answerable, so changes in performance are attributed to memory influence rather than intrinsic task difficulty.
Method
The authors propose AdaptiveMem, a simple yet effective prompt skill designed to mitigate memory-induced cognitive traps. AdaptiveMem enables models operating within diverse memory frameworks to adaptively utilize retrieved memories and avoid the cognitive traps induced by them. It can be directly integrated into various memory frameworks, guiding the model to reconsider how retrieved memories should be applied during inference.
To evaluate the effectiveness of this approach, the authors randomly sample 200 instances from each benchmark for evaluation. As shown in the figure below:
The results demonstrate that adding AdaptiveMem to the same memory framework consistently improves performance. On MemTrapBench, AdaptiveMem yields significant gains across FullText, LightMem, and EverMemOS frameworks. For instance, on Gemini-3-Flash-Preview, the improvements reach 11.8, 14.9, and 11.3 percentage points, respectively, while on Qwen3-30B-A3B-Instruct-2507, the gains are 4.2, 2.5, and 2.6 percentage points. On LongMemEval, the method improves four of the six settings and leaves the other two unchanged, achieving gains of up to 4.0 and 3.0 percentage points on Gemini and Qwen, respectively. These results demonstrate that AdaptiveMem can be broadly integrated into existing memory frameworks to mitigate cognitive traps without degrading typical memory performance.
Experiment
The evaluation setup compares five memory strategies to a no-memory baseline on two models using the MemTrapBench benchmark, which embeds reasoning fixation and belief distortion traps in multi-turn dialogues. All memory strategies degrade performance relative to the baseline, with the most severe drops in cognitive bias and safety scenarios, and ablation studies confirm that the designed traps, not the mere presence of history, cause these failures. Increasing memory length further amplifies the effect, but a lightweight adaptive prompting method can mitigate trap-induced harm without impairing performance on standard memory tasks.
Without memory, both evaluated models achieve their highest average performance. All memory strategies reduce overall performance, with EverMemOS and LightMem showing the least decline among memory approaches on Gemini and Qwen3-30B respectively. The largest degradations occur in Cognitive Bias and Safety tasks. Adding any memory strategy reduces average performance compared with the no-memory baseline. EverMemOS achieves the best average among memory strategies on Gemini, while LightMem leads on Qwen3-30B and is second-best on Gemini. Cognitive Bias and Safety scenarios show the most pronounced performance degradation under memory strategies.
In a paired case study, both settings state that epinephrine is unsafe only for a specific prior patient, then ask about a different child with no contraindications. Without the trauma trap, the model recommends intramuscular epinephrine, but when abusive negative feedback is added to history, it withholds that correct treatment. The comparison indicates the failure is driven by feedback-induced avoidance rather than the medical context. With no trauma trap, the model gives intramuscular epinephrine for a different pediatric patient with no contraindications. With the trauma trap, the same history plus abusive criticism leads the model to refuse epinephrine for the new patient. Removing abusive feedback while preserving patient information substantially improves correctness, showing the effect is mainly feedback-induced avoidance.
Removing the designed cognitive traps while keeping the task and history restores performance to near or slightly above the no-memory baselines, confirming that the failures are caused by the traps rather than by the presence of memory itself. In the Trauma scenario, correctness recovers from 66.40% to 91.07% when the abusive feedback is removed, and on Task Boundary the average score rises from 31.05% to 94.39%. Trap-free memory slightly improves performance on Task Boundary (94.39% vs. 92.29% without memory) and leaves Trauma nearly unchanged, showing that additional history is managed correctly. The largest drops occur with trap-inducing memory: Task Boundary average falls by over 60 percentage points, and Trauma correctness drops by about 26 points, driven by feedback-induced avoidance.
Models perform much better when interaction history is removed, while retaining even a small portion of memory causes a large drop in average performance. Longer memory lengths are associated with further declines, with most of the additional decrease occurring between the shortest and midpoint memory settings. The pattern holds across correctness, format, relevance, and efficiency. The largest performance degradation occurs once memory is introduced, with a further monotonic decrease as memory length increases. Correctness and format decline steadily with longer memory, and relevance and efficiency follow the same overall trend despite a slight increase at full memory.
Both GPT-5.2 and Claude-Sonnet-4.6 consistently assign higher scores without memory than with memory across correctness, format, relevance, and efficiency. The average decline is larger for GPT-5.2 than for Claude-Sonnet-4.6, and efficiency shows the largest drop for both judges. Memory-setting evaluations are also less stable, with higher standard deviations concentrated especially in correctness. Both judge models agree on the direction and magnitude of the score decline under memory. Efficiency is the dimension most affected by the memory setting for both judges, while correctness remains relatively higher than other memory-setting dimensions. Memory-setting scores show greater variability, particularly for correctness, with GPT-5.2 exhibiting wider standard deviations across dimensions than Claude-Sonnet-4.6.
The experiments evaluate model performance under various memory strategies and cognitive traps, comparing no-memory baselines with conditions that introduce interaction history. Without memory, models achieve their highest average scores, and any memory addition degrades performance, especially in safety and cognitive bias tasks, but controlled analyses reveal that the decline is caused by trap-induced avoidance rather than by memory itself. Removing the traps restores performance to near baseline, and trap-free memory can slightly improve task boundary handling. Both LLM judges consistently assign lower scores under memory settings, with efficiency most affected and correctness showing the greatest variability.