Small Average Error, Wide Individual Error: General-Purpose LLMs Estimating Energy Intake from Meal Photos in a Feeding Trial
作者: Xiaomeng Li
分类: cs.HC, cs.CV, q-bio.QM
发布日期: 2026-10-19
备注: 20 pages, 1 figure, 12 tables. Code, prompts, pre-registration and per-meal outputs: https://github.com/xtea/llm-intake-validation
💡 一句话要点
利用多模态大语言模型估算餐食能量摄入以改善饮食评估
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 饮食评估 能量估算 多模态大语言模型 营养学 机器学习
📋 核心要点
- 现有方法主要依赖餐前照片进行能量估算，未能准确反映实际摄入量，尤其在剩余食物的情况下。
- 论文提出通过比较餐前和餐后照片，或仅使用餐前照片并结合平均摄入比例，来提高能量估算的准确性。
- 实验结果表明，五个模型在使用餐后照片时提高了准确性，但在个体估算上仍存在显著偏差，尤其是对蛋白质的低估。
📝 摘要（中文）
基于照片的食物能量估算能够简化饮食评估，而通用的多模态大语言模型（LLMs）使得这种估算变得更加便捷。大多数评估使用餐前照片并与提供的能量进行比较，而饮食评估需要关注实际摄入的能量。本文评估了来自两个供应商的六个模型在802餐食中的表现，探讨了使用餐前照片、餐前和餐后照片的不同组合对能量估算的影响。实验结果显示，模型在估算摄入量时存在显著偏差，尤其在处理剩余食物时，强调了低平均偏差并不代表个体摄入估算的准确性。最终，研究指出这些估算应作为营养师评估的辅助工具，而非替代品。
🔬 方法详解
问题定义：本文旨在解决基于照片的饮食能量估算中，现有方法未能准确反映实际摄入量的问题，尤其是对剩余食物的处理不足。
核心思路：通过评估餐前和餐后照片的不同组合，结合平均摄入比例，来提高能量估算的准确性，尤其是在处理剩余食物时。
技术框架：整体流程包括数据收集（802餐食）、模型评估（六个不同的LLMs）、不同照片组合的比较（餐前、餐后、两者结合）以及结果分析。
关键创新：本研究的创新点在于通过对比不同照片组合的能量估算，揭示了模型在处理剩余食物时的不足，强调低平均偏差并不代表个体估算的准确性。
关键设计：在实验中，模型的输入包括餐前和餐后照片，使用的损失函数和参数设置根据模型的不同而有所变化，特别是在处理剩余食物时，模型的表现差异显著。
🖼️ 关键图片
📊 实验亮点
实验结果显示，五个模型在使用餐后照片时提高了准确性，尤其是在处理剩余食物方面，平均误差降低了2.2到3.4点。尽管三种模型在90%置信区间内保持了10%的误差范围，但没有模型能够达到预设的30%一致性限制。所有模型均低估了蛋白质摄入，尤其是在蛋白质密集的餐食中。
🎯 应用场景
该研究的潜在应用领域包括营养学、健康管理和饮食监测等。通过提高能量摄入估算的准确性，可以为个体饮食评估提供更可靠的数据支持，帮助营养师制定更有效的饮食计划，进而改善公众健康。未来，随着技术的进步，该方法有望在智能饮食管理应用中得到广泛应用。
📄 摘要（原文）
Photo-based estimation of food energy could ease dietary assessment, and general-purpose multimodal large language models (LLMs) make such estimates easy to obtain. Most evaluations use one photo taken before eating and compare with energy served, while dietary assessment needs energy eaten. We evaluated six models from two vendors (Claude Haiku, Sonnet and Opus 5.5; GPT-6 luna, 6.1 sol and 6 astra) on 802 meals from ACETADA, a controlled feeding trial with weighed intake, with the before photo only, before and after photos in separate calls (subtraction), or both photos in one request. Three results ran against intuition. Subtraction under-estimated intake for five of six models, because the models still report hundreds of kilocalories for a nearly empty tray. Scaling the before-only estimate by the average share eaten (0.87) matched much of the gain from the after photo; beyond that constant, the second photo lowered meal-level error by only 2.2 to 3.4 points, mainly in meals with food left. With one photo, two models had a near-zero intake bias only because they under-estimated the served energy and ignored uneaten food. In ablations, adding the after photo improved accuracy under both prompt formats for five models, while prompt effects varied across models. Over 148 feeding days, three models kept the 90% confidence interval of their mean error within a pre-set 10% margin, but no model reached the pre-set 30% limits of agreement. Protein was under-estimated by every model, most in protein-dense meals; repeated calls changed error by at most 1.4 points; a high reasoning effort helped only the smallest model (4.2 points) and changed the others by less than 1 point. A low average bias is not evidence of accurate individual intake estimation; such estimates should support, not replace, assessment by a dietitian.