Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

📄 arXiv: 2607.19345v1 📥 PDF

作者: Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

分类: cs.CL, cs.AI

发布日期: 2026-07-21

💡 一句话要点

提出GEAR以解决长上下文推理中的重复复制问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 长上下文推理 奖励机制 证据关注 模型训练 自然语言处理

📋 核心要点

  1. 现有大型语言模型在长上下文推理中存在重复复制问题，导致模型无法有效解决问题。
  2. 论文提出GEAR，通过奖励与关键证据的重叠并惩罚与无关上下文的重叠，来改善模型的推理能力。
  3. 实验结果表明，GEAR在多个基准测试中提高了4.6分，尤其在长上下文中表现更为突出。

📝 摘要（中文）

大型语言模型在复杂任务中通过逐步推理轨迹取得了良好表现，但在长上下文设置中出现了重复复制的严重问题。模型在推理过程中大量复制输入文本，而非有效解决问题。研究表明，这种行为在长上下文模型中普遍存在，并随着上下文长度的增加而加剧。为了解决这一问题，论文提出了GEAR（Grounding Evidence-Aware Reward），通过为与关键证据的重叠提供奖励，并对与无关上下文的重叠施加惩罚，从而增强模型的准确性。实验结果显示，GEAR在多个模型规模和基准测试中均表现出显著提升，尤其在长上下文下，平均提高了4.6分，同时减少了重复复制和思考长度。

🔬 方法详解

问题定义：论文要解决的问题是大型语言模型在长上下文推理中出现的重复复制现象。现有方法未能有效引导模型关注关键证据，导致推理结果不准确。

核心思路：论文的核心思路是通过引入GEAR奖励机制，增强模型对关键证据的关注，减少对无关上下文的依赖，从而提高推理的准确性。

技术框架：整体架构包括数据预处理、奖励机制设计和模型训练三个主要模块。首先，构建带有证据标注的训练数据；然后，设计奖励函数以引导模型关注关键证据；最后，进行模型训练和评估。

关键创新：最重要的技术创新点在于GEAR奖励机制，它通过奖励与关键证据的重叠和惩罚与无关上下文的重叠，显著改善了模型的推理能力。这与现有方法的单一准确性奖励机制形成鲜明对比。

关键设计：在关键设计方面，论文设置了特定的奖励和惩罚参数，定义了损失函数以平衡准确性和证据关注度，并利用自动化管道构建证据标注的训练数据。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，GEAR在多个模型规模和基准测试中均取得了显著提升，平均提高了4.6分，尤其在长上下文情况下，模型的重复复制现象和思考长度均得到了有效减少。这表明GEAR在长上下文推理中的有效性和必要性。

🎯 应用场景

该研究的潜在应用领域包括智能问答系统、对话生成和复杂任务的自动化推理等。通过提高模型在长上下文中的推理能力，能够显著提升这些应用的准确性和用户体验，具有重要的实际价值和未来影响。

📄 摘要（原文）

Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical failure mode in this regime: \emph{repetitive copying}, where models extensively copy text from the input into their reasoning traces rather than productively solving the problem. We show that this behavior is pervasive across frontier long-context LLMs and intensifies with context length. By separating each prompt into task-relevant key evidence and irrelevant distractor context, we further show that the root cause is insufficient grounding: models copy from the prompt indiscriminately, and those that fail to focus on key evidence are far more likely to answer incorrectly. Motivated by this diagnosis, we propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context. To enable GEAR on natural-language data, we develop an automated pipeline that constructs evidence-annotated training data from arbitrary documents. We validate GEAR across multiple model scales and benchmarks, showing consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards, with larger gains at longer contexts, while also reducing repetitive copying and thinking length. Our findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.