When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimation

📄 arXiv: 2610.12061v1 📥 PDF

作者: Yiruo Cheng, Shen Huang, Xiaoshuai Song, Jiejun Tan, Guanting Dong, Pengjun Xie, Ji-Rong Wen, Zhicheng Dou

分类: cs.AI, cs.CL

发布日期: 2026-10-08

💡 一句话要点

提出RACE方法以解决智能体推理效率问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 智能体推理 交叉回合估计 适应性推理 强化学习 计算效率

📋 核心要点

  1. 现有方法在每个交互回合中都进行推理，导致计算成本高且效率低下。
  2. 论文提出RACE方法，通过交叉回合估计判断何时需要新的推理，优化智能体的决策过程。
  3. 实验表明，RACE在四个基准测试中显著降低了推理成本，同时保持或提升了任务性能。

📝 摘要（中文）

基于大型语言模型的智能体在复杂任务中表现出色，但在每个交互步骤中进行推理并非总是必要。本文提出了一种新方法RACE，通过交叉回合估计来适应性地判断何时进行推理，避免了昂贵的生成验证。研究发现，去除额外推理后后续动作的可能性降低，能够有效指示现有推理的支持程度。RACE引入了逐步识别推理回合的LoGiC程序，显著降低推理成本，同时保持或提升任务性能。

🔬 方法详解

问题定义：本文旨在解决智能体在每个交互回合中都进行推理所带来的高计算成本问题。现有方法缺乏有效机制来判断何时推理是必要的，导致效率低下。

核心思路：RACE方法通过交叉回合估计，利用去除额外推理对后续动作可能性的影响，来判断现有推理是否足够，从而适应性地决定何时进行推理。

技术框架：RACE的整体架构包括两个主要模块：Likelihood-Guided Progressive Reasoning Cover Detection (LoGiC)程序和强化学习策略优化。LoGiC程序逐步识别哪些推理回合的去除对当前和后续动作影响有限。

关键创新：RACE的核心创新在于引入了基于可能性的推理适应机制，能够有效减少不必要的推理步骤，与传统方法相比，显著提高了智能体的决策效率。

关键设计：在RACE中，设计了特定的损失函数来优化推理回合的选择，同时在强化学习中融入了去除信号，以指导智能体学习何时推理和何时直接行动。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，RACE在四个代表性基准测试中，推理成本降低了显著比例，同时任务性能保持不变或有所提升，具体性能数据未详述，但提升幅度明显。

🎯 应用场景

该研究的潜在应用领域包括智能客服、自动化决策系统和人机交互等。通过提高智能体的推理效率，RACE能够在实际应用中降低计算资源消耗，提升用户体验，具有广泛的实际价值和未来影响。

📄 摘要（原文）

Large language model (LLM)-based agents have demonstrated strong capabilities on complex tasks. They typically perform reasoning before each action throughout an interaction trajectory. However, reasoning may not be necessary at every turn, as reasoning produced earlier can continue to support subsequent actions. A key challenge is therefore to determine when existing reasoning remains sufficient and when a new reasoning step is needed, without relying on costly generation-based verification. We find that decreases in the likelihood of subsequent reference actions after removing additional reasoning closely track whether those actions remain recoverable given earlier reasoning, providing an effective and lightweight signal for estimating cross-turn action support. Based on this observation, we propose Reasoning Adaptation through Cross-Turn Estimation (RACE), a training approach for adaptive agent reasoning. RACE introduces a Likelihood-Guided Progressive Reasoning Cover Detection (LoGiC) procedure that progressively identifies reasoning turns whose removal has limited impact on the current and subsequent reference actions. The resulting removal signals are incorporated into both supervised fine-tuning and agentic reinforcement learning, enabling the policy to learn when to reason and when to act directly. Extensive experiments on four representative agent benchmarks show that RACE substantially reduces reasoning cost while maintaining or improving task performance.