Test-time Calibration Learning for Large Language Model Reasoning

📄 arXiv: 2610.02695v1 📥 PDF

作者: Zizhuo Zhang, Xiong Peng, Jingwei Sun, Rong Yao, Shixiong Kai, Mingxuan Yuan, Bo Han

分类: cs.LG, cs.AI

发布日期: 2026-10-02

备注: 34 pages

🔗 代码/项目: GITHUB

💡 一句话要点

提出无标签测试时校准学习以提升大语言模型推理能力

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 校准学习 无标签学习 自监督学习 推理能力 决策支持 领域适应

📋 核心要点

  1. 现有方法依赖于标注数据进行校准，限制了在无标签测试环境中的应用，难以适应新任务。
  2. 本文提出的TTCL框架通过无标签数据自适应推理准确性和置信度，解决了标注数据不足的问题。
  3. 实验结果显示，TTCL在多个基准上实现了平均相对准确性提升40.13%，ECE降低70.80%。

📝 摘要（中文）

可靠的大语言模型（LLMs）不仅需要生成准确的答案，还需表达真实反映其正确概率的置信度。校准对于识别不确定预测和支持可靠决策至关重要。现有方法依赖于标注数据，限制了其在实际测试环境中的适用性。为此，本文提出了测试时校准学习（TTCL），这是一个无标签框架，能够直接在未标记的目标任务数据上联合适应推理准确性和表达的置信度。TTCL通过多个模型生成的响应推导自监督信号，从而在测试时进行校准学习。实验结果表明，TTCL在数学推理和事实问答任务中显著提高了准确性和校准性。

🔬 方法详解

问题定义：本文旨在解决大语言模型在无标签测试环境中的校准问题。现有方法依赖于标注数据进行校准，限制了其在实际应用中的灵活性和适应性。

核心思路：TTCL框架通过自监督学习，利用多个模型生成的响应来推导准确性和置信度的自监督信号，从而在没有标注数据的情况下进行校准。

技术框架：TTCL的整体架构包括数据收集、模型生成响应、信号提取和校准学习四个主要模块。首先收集未标记的目标任务数据，然后生成多个模型响应，接着提取自监督信号，最后进行校准学习。

关键创新：TTCL的主要创新在于其无标签学习能力，能够在测试时动态适应新任务的校准需求，与依赖标注数据的现有方法形成鲜明对比。

关键设计：TTCL设计了特定的损失函数来平衡准确性和置信度的优化，并采用了多模型生成的响应来增强自监督信号的有效性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

TTCL在多个基准测试中表现出色，平均相对准确性提升40.13%，ECE降低70.80%。在数学到事实问答的设置中，TTCL实现了20.35%的准确性提升和53.83%的ECE降低，显示出其在领域转移中的有效性。

🎯 应用场景

该研究的潜在应用领域包括医疗诊断、金融决策和自动化客服等场景，能够提升大语言模型在实际应用中的可靠性和决策支持能力。未来，TTCL可能推动无标签学习在更多领域的应用，促进智能系统的普遍适应性。

📄 摘要（原文）

Reliable large language models (LLMs) must not only produce accurate answers but also express confidence that faithfully reflects their probability of being correct. Such calibration is essential for identifying uncertain predictions and supporting reliable decision-making in real-world deployment. Recent studies incorporate calibration learning into reinforcement learning (RL), jointly optimizing answer correctness and verbalized confidence using ground-truth correctness supervision. However, their reliance on labeled data limits their applicability in practical test-time settings, where ground-truth labels are unavailable and calibration may need to adapt to newly encountered target tasks. To address this challenge, we propose Test-Time Calibration Learning (TTCL), a label-free framework that jointly adapts reasoning accuracy and verbalized confidence directly on unlabeled target-task data. Specifically, TTCL derives self-supervision signals for both correctness and calibration from multiple model-generated responses, enabling calibration learning at test time without ground-truth labels. Theoretical analysis further establishes TTCL as a bounded surrogate for the ideal calibration objective. Extensive experiments on mathematical reasoning and factual question answering demonstrate that TTCL consistently improves both accuracy and calibration across diverse models and tasks. On base models, TTCL achieves an average relative accuracy improvement of +40.13% and an ECE reduction of +70.80% across eight benchmarks. Moreover, TTCL can further improve both accuracy and calibration for already calibrated models under domain shift, particularly when source-domain calibration transfers poorly to target tasks. In the math-to-factQA setting, TTCL achieves an average relative accuracy gain of +20.35% and reduces ECE by +53.83%. The source code is released at https://github.com/tmlr-group/TTCL.