NOAH: Learning the Full Patient Journey. A Longitudinal Multimodal Time-Aware Model for Representation and Forecasting

📄 arXiv: 2609.09140v1 📥 PDF

作者: Tobias Susetzky, Raphael Rehms, Dmitrii Seletkov, Özgün Turgut, Michelle Espranita Liman, Lisa Steinhelfer, Rickmer Braren, Daniel Rueckert

分类: cs.LG, cs.AI

发布日期: 2026-09-08

💡 一句话要点

提出NOAH模型以解决多模态患者数据预测问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态数据 生成模型 时间序列预测 临床决策支持 个性化医疗

📋 核心要点

  1. 现有AI模型在处理复杂的多模态患者数据时，难以捕捉时间动态和随机性，限制了其预测能力。
  2. NOAH模型采用双向时间集成和变分潜在空间，能够全面表示和预测患者的多模态旅程，具有任务无关性。
  3. NOAH在临床结果、ICD章节和共病预测等方面表现出色，展示了其在个性化医疗中的潜力。

📝 摘要（中文）

随着医疗数字化的推进，产生了大量的纵向多模态患者记录，但如何充分利用这些数据来表示和预测患者状态轨迹仍然是一个关键挑战。现有的AI模型往往难以捕捉复杂、不规则的时间动态和真实世界多模态患者数据的内在随机性。NOAH是一个时间感知的、任务无关的生成变换器模型，能够表示和预测完整的多模态患者旅程。NOAH通过新颖的双向时间集成和变分潜在空间来捕捉患者状态的连续演变和临床轨迹的随机性，具有强大的预测能力。

🔬 方法详解

问题定义：本论文旨在解决如何有效利用纵向多模态患者记录来表示和预测患者状态轨迹的问题。现有方法多为判别性，难以处理复杂的时间动态和多模态数据的随机性。

核心思路：NOAH模型通过引入双向时间集成和变分潜在空间，能够捕捉患者状态的连续演变，提供更为全面的预测能力。该模型设计为任务无关，适用于多种临床场景。

技术框架：NOAH的整体架构包括数据预处理模块、双向时间集成模块、变分潜在空间模块和生成预测模块。模型能够处理医学图像、时间序列、数值信号、分类事件及结构化和非结构化临床记录。

关键创新：NOAH是首个真正的整体生成模型，支持自回归预测、零样本分类和反事实干预模拟，显著提升了对临床结果的预测能力。

关键设计：NOAH在模型训练中采用了特定的损失函数以优化生成效果，并通过调整潜在空间的维度来平衡模型的复杂性与预测准确性。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

NOAH在临床结果预测、15个ICD章节和29种共病的识别中表现出色，展示了其强大的预测能力。与现有基线相比，NOAH在时间到事件预测任务中提升了预测准确性，证明了其在多模态数据处理中的优势。

🎯 应用场景

NOAH模型在个性化医疗和数字医学领域具有广泛的应用潜力。它能够为临床决策提供智能预测支持，帮助医生更好地理解患者的健康轨迹，从而制定更有效的治疗方案。未来，NOAH可能在公共卫生监测和疾病预防等方面发挥重要作用。

📄 摘要（原文）

The digitization of healthcare has generated vast, longitudinal, and multimodal patient records over a lifetime, yet fully exploiting these data to represent and predict patient state trajectories remains a critical challenge. Current AI models often struggle to capture the complex, irregular temporal dynamics and inherent stochasticity of real-world multimodal patient data. Existing AI approaches for modeling longitudinal patient records are predominantly discriminative, limited to a few modalities, constrained by closed categorical vocabularies, treating time as a monotonic inductive bias, or they are limited in forecasting future patient states. We introduce NOAH, a time-aware, task-agnostic, generative transformer model representing and forecasting the full multimodal patient journey. NOAH features a novel bidirectional time integration and a variational latent space to capture the continuous evolution of patient states and the stochasticity of clinical trajectories. Built from over 559 million clinical events from 431,000 hospital visits of 299,000 patients across the MIMIC dataset family, NOAH natively processes medical images, time-series and numeric signals, categorical events, as well as structured and unstructured clinical records. NOAH is the first truly holistic generative model in its field, enabling autoregressive forecasting with optional time control, zero-shot classification, and counterfactual intervention simulation. It generates highly informative and predictive patient state representations that demonstrate strong performance in probing for clinical outcomes, 15 ICD chapters, and 29 comorbidities, as well as in time-to-event prediction. Seamlessly handling diverse modalities and complex temporal dynamics, NOAH provides a versatile, task-agnostic, scalable foundation for intelligent predictive systems in personalized clinical care and digital medicine.