PresenceBench: An Exploratory Evaluation of Large Language Model Adaptation to User State
作者: Peña Arroyo Gallego, Mauro Ruiz Moreno, Alejandro Acien, Sergio Romero-Tapiador, Teresa Arroyo-Gallego
分类: cs.HC, cs.AI, cs.CL
发布日期: 2026-10-16
备注: 33 pages, 6 figures, 13 tables
💡 一句话要点
提出PresenceBench以评估大语言模型对用户状态的适应性
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 用户状态适应 人工智能评估 大语言模型 智能助手 响应质量
📋 核心要点
- 现有的人工智能系统在适应用户状态时，往往缺乏有效的评估工具，导致适应性不足。
- 论文提出PresenceBench，通过测量智能层如何利用用户状态，评估其适应性和响应质量。
- 实验结果显示，四个模型在提供合成状态后，均能提升响应质量，且安全性表现良好。
📝 摘要（中文）
本研究提出了一种名为PresenceBench的工具，用于评估人工智能系统在适应用户功能状态（如准备状态、流畅性和稳定性）时的智能表现。PresenceBench关注智能层如何利用提供的状态，而非如何感知该状态，并将能力分解为可测量的属性。研究通过五臂因果设计，评估了四个模型在不同场景下的表现，结果显示，提供合成状态能够提升模型响应的质量，但具体的适应性特征尚未完全解决。安全性方面，政策交付下的失败案例较少，表明该方法在实际应用中具有一定的安全性。
🔬 方法详解
问题定义：本研究旨在解决人工智能系统在适应用户状态时缺乏有效评估工具的问题。现有方法未能充分测量智能层如何利用用户状态，导致适应性不足。
核心思路：PresenceBench的核心思路是通过评估智能层对用户状态的利用情况，来量化其适应性。该方法关注模型在不同状态下的响应质量，而非状态的感知过程。
技术框架：PresenceBench的整体架构包括状态输入、模型响应评估和结果分析三个主要模块。通过五臂因果设计，比较模型在不同状态下的表现。
关键创新：该研究的关键创新在于引入了“接收性”和“特异性”两个可测量属性，以量化模型对用户状态的适应能力。这与传统方法的评估方式有本质区别。
关键设计：在实验中，采用了合成状态的输入，并通过人类配对比较的方式进行评分。设计中还考虑了政策交付的影响，确保了安全性和响应质量的平衡。实验结果显示，政策交付在资源使用上有所优化。
🖼️ 关键图片
📊 实验亮点
实验结果表明，提供合成状态后，四个模型的响应质量均有显著提升，偏好点估计均超过0.5，且三分之二的95%置信区间排除了随机性影响。尽管特异性未能完全解决，但政策交付在资源使用上实现了优化，安全性表现良好。
🎯 应用场景
PresenceBench的研究成果可广泛应用于智能助手、聊天机器人等领域，帮助提升人工智能系统在实际应用中的用户体验。通过更好地适应用户状态，AI系统能够提供更为个性化和高效的服务，未来可能在教育、医疗和客户服务等多个行业产生深远影响。
📄 摘要（原文）
Presence-aware intelligence is the capacity of an AI system's intelligence layer to adapt appropriately to the functional state of the person it serves (here readiness, flow, and stability) while preserving task quality and safety. We introduce PresenceBench, an instrument for measuring it. PresenceBench evaluates how the intelligence layer uses a supplied state, not how that state is sensed, and breaks the capability into measurable properties. Two are tested here. Receptivity: does the model take up the state and give a better response than with no signal? Specificity: does any improvement depend on the correct state, rather than on extra structured context? A third, adaptation across states, is a required endpoint not measured in this release. A five-arm causal design answers both questions, against each model's own baseline and an equally structured mismatched-state control, scored by a correctness-gated human paired comparison. The exploratory evaluation covered four models across eighteen single-turn, six fixed three-turn and six simulated natural-conversation scenarios. Supplying a synthetic state, raw or translated into a response policy, gave human-preference point estimates above 0.5 for all four models (Claude Sonnet 5, DeepSeek V4 Pro, Gemini 3.6 Flash, GPT-5.6 Terra), with three of eight 95% intervals excluding chance. Specificity was not resolved: no per-model interval excluded chance, and the design cannot yet separate the state values from a generic instruction to adapt. Policy delivery reduced priced cost and output tokens against each model's baseline; these are resource-use changes, not efficiency gains, because correctness non-inferiority was not established. Safety-floor failures were rare (13 of 360 cases), with a non-significant excess under policy delivery. We report the instrument, the frozen results, and the controls a confirmatory study needs.