woma: a real-time foundation model and its fine-tuned models for endoscopy
作者: Thang Tran, Lan Dang
分类: cs.SE, cs.CV, cs.LG
发布日期: 2026-09-14
备注: 28 pages, 10 figures (8 in the main text, 2 supplementary), 13 tables (8 in the main text, 5 supplementary). Preprint. Models and run records are available from the corresponding author
💡 一句话要点
提出woma模型以解决内窥镜实时分析问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 内窥镜分析 实时处理 自监督学习 无标签训练 模型微调 临床应用 深度学习
📋 核心要点
- 现有内窥镜分析方法在实时性和准确性上存在不足，难以满足临床需求。
- woma模型通过无标签训练和自监督学习，构建了一个高效的内窥镜分析框架，支持多种任务的微调。
- 实验结果显示，woma在结肠镜和胃镜检查中均取得了高精度和高速度的性能，显著优于现有方法。
📝 摘要（中文）
woma是一个针对胃肠道内窥镜的实时基础模型，使用约一百万个内窥镜帧进行无标签训练，并在此基础上进行任务模型的微调。我们设计了一个系统化的生产流程，设定了要求和通过标准，筛选了八个候选模型，采用自监督训练并设定停止规则，随后进行微调和部署优化，所有操作均在一个自包含的库numbat中完成。woma模型包括两个微调模型，分别用于结肠镜和胃镜检查，能够识别和标记息肉、命名视野中的结肠段、建议息肉类型，并评估肠道准备情况。所有结果均在未见过的训练数据上进行评估。
🔬 方法详解
问题定义：本论文旨在解决内窥镜实时分析中的准确性和效率问题。现有方法往往依赖于大量标注数据，难以适应临床快速变化的需求。
核心思路：论文提出的woma模型通过无标签训练，利用自监督学习的方式，从大量内窥镜图像中提取特征，进而进行任务模型的微调，以实现高效的实时分析。
技术框架：woma的整体架构包括基础模型的训练、任务模型的微调和优化部署三个主要阶段。首先，使用自监督学习对基础模型进行训练；其次，在此基础上进行针对特定任务的微调；最后，优化模型以适应实际应用场景。
关键创新：woma的核心创新在于其无标签训练的能力和高效的实时处理能力。与传统方法相比，woma能够在未见过的数据上进行有效推理，且在速度和准确性上均有显著提升。
关键设计：在模型设计中，采用了特定的损失函数和网络结构，以确保模型在不同任务上的适应性和准确性。同时，使用了自包含的库numbat，简化了部署过程，减少了对外部工具的依赖。
🖼️ 关键图片
📊 实验亮点
在结肠镜检查中，woma模型在六家医院的PolypGen数据集中以精度≥0.85发现96%的息肉；在胃镜检查中，92%的帧正确命名了地标区域，且在39个保留的肿瘤帧中以特异性0.91标记了37个。整体性能超越了PyTorch、ONNX Runtime和TensorRT等基线，展示了显著的速度优势。
🎯 应用场景
该研究的潜在应用领域包括医院内窥镜检查、远程医疗和自动化诊断系统。woma模型的实时分析能力能够提高内窥镜检查的效率和准确性，帮助医生更快地做出诊断决策，具有重要的临床价值和广泛的应用前景。
📄 摘要（原文）
woma is a real-time foundation model for gastrointestinal endoscopy: a network trained without labels on about a million endoscopy frames, from which task models are fine-tuned. We contribute a systematic design for production. Requirements and pass marks were fixed before any run, eight candidates screened under pre-registered rules, self-supervised training taken to a stopping rule, then fine-tuning and deployment optimisation, all on one self-contained library, numbat. We also contribute woma itself with two fine-tuned models, every outcome reported met or missed. Our colonoscopy model finds and outlines polyps, names which colon segment is in view, suggests polyp type and grades bowel preparation. Our gastroscopy model names a station out of 22 protocol sites, flags and outlines lesions, and names one of seven findings. Every number was read on data never seen in training, and shipped weights were chosen on that record. In colonoscopy, 96% of polyps in a six-hospital PolypGen set are found at precision >=0.85, and 19 of 19 polyps across fifteen full REAL-Colon videos at 1.6 false alarms per procedure. In gastroscopy, landmark region is named correctly on 92% of frames from unseen patients, and 37 of 39 held-out neoplasia frames are flagged at specificity 0.91. On one workstation GPU every task runs over 1080p video at about 100 frames per second, faster than PyTorch, ONNX Runtime and TensorRT in all four precision regimes tested. TensorRT comes closest: one pass of our foundation model takes it 3 to 27% longer than ours, and we deliver 6 to 31% more frames per second from frame to results. A second build links no vendor library at all -- our own kernels over Vulkan -- so a site deploys two files and needs no toolkit, no cuDNN and no framework; in f32 it beats the CUDA build on the same card.