Quantifying the Stability of Multi-Step Reasoning via Error Amplification

📄 arXiv: 2610.06404v1 📥 PDF

作者: Dongyue Li, Ziniu Zhang, Minxuan Duan, Hongyang R. Zhang

分类: cs.LG, cs.AI, stat.ML

发布日期: 2026-10-05

备注: 37 pages; To appear in NeurIPS 2026

💡 一句话要点

量化多步推理稳定性以解决错误放大问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多步推理 错误放大 推理稳定性 雅可比矩阵 变换器模型 链式思维 量化感知训练

📋 核心要点

  1. 现有的多步推理方法在处理长序列时容易出现错误累积，导致最终结果不稳定。
  2. 论文提出通过量化推理稳定性，利用雅可比矩阵谱范数的乘积来评估错误放大，从而控制推理过程中的稳定性。
  3. 在七项评估中，提出的算法在基线比较上平均提升3.5%，在长输入上提升8.2%，验证了稳定性度量的有效性。

📝 摘要（中文）

本文考虑多步推理过程的稳定性，该过程在语言模型中有广泛应用，包括链式思维和算法推理。尽管更长的推理序列可以提高模型的生成能力，但中间推理步骤的错误可能在自回归生成中累积，并在最后显著增长。我们提出了一个由雅可比矩阵的谱范数乘积决定的推理误差界限，作为推理稳定性的量化指标。通过对训练简单任务的变换器模型进行分析，我们证明了模型收敛到一个稳定性度量衰减的解，从而在任意长步骤上几乎实现零推理损失。最后，我们提出了几种算法性影响，以控制稳定性，并通过在图算法推理任务和符号状态跟踪任务上微调语言模型验证了这些算法的有效性。

🔬 方法详解

问题定义：本文旨在解决多步推理过程中错误累积导致的稳定性问题。现有方法在长序列推理时，容易因为中间步骤的错误而导致最终结果的不准确。

核心思路：我们提出了一个推理误差界限，基于雅可比矩阵的谱范数乘积来量化推理的稳定性。通过控制这一误差放大因子，可以有效提高推理过程的稳定性。

技术框架：整体架构包括两个主要模块：第一，推理稳定性度量的计算；第二，基于该度量的算法设计，包括链式思维长度压缩和量化感知训练。

关键创新：最重要的创新点在于提出了一个新的稳定性度量，并证明了变换器模型在训练过程中能够收敛到一个稳定性度量衰减的解，这与现有方法的稳定性分析有本质区别。

关键设计：在算法设计中，采用了链式思维长度压缩以降低每一步的敏感性，并通过量化感知训练来正则化输入雅可比矩阵的范数。这些设计有效地减少了稳定性度量，验证了其正则化效果。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

实验结果显示，提出的算法在七项评估中平均提升3.5%，在处理长输入时提升8.2%。消融分析表明，稳定性度量减少了3-8倍，验证了对雅可比矩阵谱范数的正则化效果。

🎯 应用场景

该研究的潜在应用领域包括自然语言处理中的推理任务、图算法推理以及符号状态跟踪等。通过提高多步推理的稳定性，能够显著提升模型在复杂任务中的表现，具有重要的实际价值和未来影响。

📄 摘要（原文）

We consider the stability of multi-step reasoning processes, which have extensive applications in language models, including chain-of-thought and algorithmic reasoning. While longer sequences of reasoning can improve a model's generation capability at test time, the errors due to intermediate reasoning steps can accumulate in autoregressive generation, and thus grow substantially at the end. In this paper, we ask: What are the key factors determining the stability of multi-step reasoning? First, we show an inference error bound governed by the product of spectral norms of the Jacobians taken through the input space across generation steps. This product can be viewed as an error amplification factor, which could scale exponentially with the number of reasoning steps, serving as a quantitative measure of reasoning stability. Second, we analyze this measure in transformer models trained to predict simple tasks like linear and quadratic functions. We theoretically prove that the transformer model converges to a solution where the stability measure decays, thus yielding nearly zero inference loss over (arbitrarily) long steps. Finally, the stability analysis leads to several algorithmic implications for controlling the stability, through (i) chain-of-thought length compression that reduces the sensitivity of each step, and (ii) quantization-aware training that regularizes the input Jacobian norms. We validate the proposed algorithms by fine-tuning language models on graph-algorithmic reasoning tasks and symbolic state-tracking tasks. Across seven evaluations, our algorithms improve over baseline comparisons by 3.5% on average, and by 8.2% for longer-length inputs. Ablation analysis validates that the stability measure is drastically reduced by 3-8$\times$, confirming the regularization effect on the spectral norms of the (input space) Jacobians.