Test-Time Adaptation of Reasoning Strategies with Bayesian Nonparametric Memory
作者: Keshav Ramji, Tahira Naseem, Ramón Fernandez Astudillo
分类: cs.CL
发布日期: 2026-10-05
💡 一句话要点
提出贝叶斯非参数记忆以解决推理策略适应问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 推理策略 贝叶斯方法 在线学习 记忆模块 高斯混合模型 自适应系统 机器学习
📋 核心要点
- 现有大型语言模型在推理时生成成本高，且缺乏有效机制将新发现应用于后续问题。
- 提出贝叶斯备忘单，通过HDP-GMM对行为嵌入建模，实现在线适应和新组件生成。
- 实验结果表明，贝叶斯备忘单在多个推理基准上相较于现有记忆模块有显著性能提升。
📝 摘要（中文）
现代大型语言模型（LLMs）在通过语言化思维链进行推理时，生成成本显著增加，且现有机制难以将新发现应用于后续问题。本文提出了一种结构化的备忘单，利用层次狄利克雷过程高斯混合模型（HDP-GMM）对行为嵌入进行建模，允许跨领域共享组件并使用后验预测检索相关行为。该方法在在线测试时间训练（TTT）环境中实现了廉价适应，显著提升了在多个推理基准上的表现，展示了贝叶斯启发的记忆模块在有效测试时间适应中的价值。
🔬 方法详解
问题定义：本文旨在解决大型语言模型在推理过程中生成成本高和缺乏有效记忆机制的问题。现有方法在处理新输入时难以将新发现的推理策略应用于后续问题，导致推理效率低下。
核心思路：论文提出了一种名为贝叶斯备忘单的机制，通过层次狄利克雷过程高斯混合模型（HDP-GMM）对行为嵌入进行建模，允许跨领域共享组件并根据输入查询动态检索相关行为。这种设计旨在实现在线适应，降低推理成本。
技术框架：整体架构包括行为嵌入的学习、HDP-GMM的构建和后验预测的行为检索。首先，通过观察输入查询生成行为嵌入，然后利用HDP-GMM对这些嵌入进行聚类，最后在推理时根据查询动态检索相关行为。
关键创新：最重要的技术创新在于引入了贝叶斯备忘单，允许在推理过程中根据新样本更新混合模型的充分统计量，并在行为足够新颖时生成新组件。这与现有方法的静态记忆模块形成鲜明对比。
关键设计：在模型设计中，采用了层次狄利克雷过程来实现行为的聚类，设置了适应性混合权重，并通过collapsed Gibbs采样实现行为的重新分配。这些设计使得模型在推理时能够灵活应对新情况。
🖼️ 关键图片
📊 实验亮点
实验结果显示，贝叶斯备忘单在AIME'25、Omni-MATH和PhysReason等推理基准上相较于现有记忆模块有明显的性能提升，尤其在冷启动设置下，表现出更高的适应性和效率。
🎯 应用场景
该研究的潜在应用领域包括智能问答系统、个性化推荐和教育技术等。通过有效的推理策略适应，模型能够在多变的环境中快速调整，提升用户体验和系统效率。未来，该方法可能推动更智能的交互式AI系统的发展。
📄 摘要（原文）
While modern large language models (LLMs) have been trained to reason through verbalized chains-of-thought, the generation cost grows substantially due to suboptimal paths to reach the final answer. Furthermore, as new insights are discovered while observing various input queries (e.g. through self-reflection), limited mechanisms exist for carrying forward these findings to be applied to subsequent problems. One can view the list of such strategies or behaviors as a growing cheatsheet, with elements retrieved from this memory module at inference-time. In this work, we consider structured cheatsheets, with learned clusters of behaviors. We introduce a Hierarchical Dirichlet Process Gaussian Mixture Model (HDP-GMM) over behavior embeddings, which shares components across domains while allowing domain-specific mixing weights, and uses the posterior predictive to retrieve relevant behaviors for a query; we call this a $\textit{Bayesian Cheatsheet}$. This mechanism allows for cheap adaptation in an online test-time training (TTT) setting, softly updating the mixture's sufficient statistics following each sample and enabling the creation of new components when the synthesized behaviors are sufficiently novel. We demonstrate that Bayesian Cheatsheet achieves clear performance gains relative to existing memory modules across reasoning benchmarks such as AIME'25, Omni-MATH, and PhysReason, even in the cold-start setting. We show that the Bayesian Cheatsheet is an adaptively reorganizing memory module, as behaviors can be re-assigned to components through a single step of collapsed Gibbs sampling. Our findings highlight the value of Bayesian-inspired memory modules for effective test-time adaptation and the role of structure in metacognitive reasoning.