Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs

📄 arXiv: 2609.38070v1 📥 PDF

作者: Feiyang Li, Shengjing Liu, Qi Zhan, Sijie Cheng, Weiqing Wang, Hongwen Chen, Yuxuan Yang, Wen Wang, Yile Wang, Hui Huang

分类: cs.AI

发布日期: 2026-09-29

备注: 25 pages, 16 figures, 8 tables. Under peer review

🔗 代码/项目: GITHUB

💡 一句话要点

提出DTC框架以解决大型语言模型推理不确定性量化问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 推理不确定性 信心校准 Divergent Token Confidence Jensen-Shannon散度 黑盒评估 白盒评估

📋 核心要点

  1. 现有方法主要依赖于关键标记的概率来估计大型语言模型的信心，但其机制尚不明确，导致信心校准效果不佳。
  2. 本文提出Divergent Token Confidence (DTC)框架，通过计算模型在推理过程中不一致的标记数量来估计信心，提供了一种新的信号。
  3. 实验结果显示，DTC在白盒和黑盒评估中均显著提高了信心校准，平均期望校准误差降低至13.0%，相比传统方法有显著提升。

📝 摘要（中文）

随着大型语言模型的链式推理能力提升，评估和校准其推理信心变得愈发重要，以量化其答案的不确定性。目前，基于选定关键标记概率的信心估计方法存在机制不明确的问题。本文提出了Divergent Token Confidence (DTC)框架，通过计算两个模型在解码过程中强烈不一致的标记数量来估计信心。DTC利用Jensen-Shannon散度来识别这些分歧标记，发现其数量与答案准确性呈负相关，成为一种简单有效的不确定性量化信号。实验表明，DTC在多种模型和数学基准上均优于基于概率和口头表达的基线方法，提供了改善大型语言模型推理不确定性量化的新思路。

🔬 方法详解

问题定义：本文旨在解决大型语言模型推理信心估计的不足，现有方法主要依赖于概率，缺乏明确的机制，导致校准效果不理想。

核心思路：提出Divergent Token Confidence (DTC)框架，通过计算两个模型在解码过程中强烈不一致的标记数量来估计信心，利用这些分歧标记作为不确定性量化的信号。

技术框架：DTC框架包括两个主要模块：首先，利用Jensen-Shannon散度计算下一个标记的分布；其次，统计在相同推理轨迹下模型之间的分歧标记数量。

关键创新：DTC的核心创新在于通过计数分歧标记来估计信心，这一方法与传统的基于概率的信心估计方法本质上不同，提供了更直观的信号。

关键设计：DTC框架不需要显式训练，且不影响生成过程，支持白盒和黑盒评估，关键参数设置和损失函数设计未在文中详细说明，具体实现可参考公开代码。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，在白盒评估中，DTC的计数估计器平均期望校准误差为13.0%，显著优于标准全序列信心方法的32.7%-42.4%。在黑盒设置中，DTC也改善了原始口头表达分数的校准，期望校准误差从32.1%-40.2%降至13.7%-16.3%。

🎯 应用场景

该研究的潜在应用领域包括自然语言处理、对话系统和智能问答等，能够帮助提升模型在实际应用中的可靠性和信心评估，未来可能影响大型语言模型的开发和应用标准。

📄 摘要（原文）

As the chain-of-thought reasoning capabilities of large language models improve, evaluating and calibrating their reasoning confidence is becoming increasingly important for quantifying the uncertainty of their answers. Current methods for estimating the confidence of large language models are generally based on probabilities of selected key tokens, but the underlying mechanism remains unclear. Our pilot study finds that replacing selected token probabilities with coarse substitutes can also improve calibration, motivating us to further explore effective signals of model confidence. We introduce Divergent Token Confidence (DTC), a framework that estimates confidence by counting tokens at which two models strongly disagree during decoding. DTC identifies these divergent tokens using the Jensen-Shannon divergence between next-token distributions evaluated along the same reasoning trajectory. We find that their count is almost negatively associated with answer accuracy, thereby serving as a simple yet effective signal for uncertainty quantification. DTC supports both white-box and black-box evaluation using auxiliary models, without explicit training and affecting the generation process. Experiments across multiple model families and six mathematical benchmarks demonstrate improved calibration over probability-based and verbalized baselines. Under white-box evaluation, the count-only estimator achieves an average expected calibration error of 13.0%, compared with 32.7%-42.4% for standard full-sequence confidence methods. In black-box settings, it also improves calibration over the original verbalized scores. For example, mean expected calibration error falls from 32.1%-40.2% to 13.7%-16.3% on DeepSeek-V3.2. These findings provide new insights for improving reasoning uncertainty quantification in large language models. The code is released at https://github.com/szu-tera/DTC.git.