UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement
作者: Fang Wu, Da Xing, Yanjie Huang, Junxi Wang, Ji Wang, Hejia Geng, Guancheng Wan, Bowen Zuo, Xiaomin Li, Shixiang Tang, Xinyu Xiang, Zehong Wang, Shiyi Du, Peng Xia, Shuangjia Zheng, Yining Hong, Li Erran Li, Jure Leskovec, Yejin Choi
分类: cs.AI, cs.CV
发布日期: 2026-09-30
💡 一句话要点
提出UniEvo-VL以实现多模态模型的自我改进
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 自我蒸馏 多模态学习 自我改进 生成模型 反馈机制
📋 核心要点
- 现有多模态模型在自我改进过程中常依赖外部教师，限制了其自我反馈的有效利用。
- UniEvo-VL通过让同一模型在不同上下文中充当教师和学生，利用自我批评进行自我演化。
- 实验显示，UniEvo-VL在GenEval和GenEval2 Soft-TIFA上分别提升了0.061和2.56的性能，验证了其有效性。
📝 摘要（中文）
现代多模态模型将生成与理解整合为一个统一系统，使其能够提供并学习自身反馈。基于此能力，我们提出了UniEvo-VL，一个自我演化框架，旨在利用测试时的自我纠错反馈进行学习。该方法不依赖于外部教师，而是利用模型自身的自我批评作为特权信息，要求同一多模态模型在不同上下文中充当教师和学生。实验结果表明，UniEvo-VL显著提升了多模态模型的图像生成能力，同时保持对额外反思信息的敏感性。
🔬 方法详解
问题定义：本论文旨在解决现有多模态模型在自我改进中对外部教师的依赖问题，限制了模型自我反馈的利用效率。
核心思路：UniEvo-VL的核心思想是利用同一模型的自我批评作为特权信息，使其在不同上下文中充当教师和学生，从而实现自我演化。
技术框架：该框架包含两个主要模块：教师模块和学生模块。学生模块仅接收原始问题，而教师模块则基于自我批评进行条件化，训练过程中最小化两者在去噪扩散分布上的状态差异。
关键创新：UniEvo-VL的创新在于不再依赖外部教师，而是通过自我批评实现自我学习，这一设计使得模型在训练时能够更灵活地利用自身反馈。
关键设计：在损失函数设计上，论文采用了基于去噪扩散的分布差异最小化策略，确保学生和教师在生成过程中的一致性，同时保持对反思信息的敏感性。
🖼️ 关键图片
📊 实验亮点
实验结果表明，UniEvo-VL在GenEval上从0.747提升至0.808，在GenEval2 Soft-TIFA上从32.97提升至35.53，显示出显著的性能提升。此外，使用更强大的外部批评者（如GPT5.6-Luna）时，模型的自我演化潜力进一步增强。
🎯 应用场景
该研究的潜在应用领域包括智能助手、内容生成、图像处理等多模态交互系统。通过提升多模态模型的自我改进能力，用户在使用这些系统时将获得更流畅和个性化的体验，未来可能推动更广泛的人工智能应用发展。
📄 摘要（原文）
Modern multimodal models bring generation and understanding into a single unified system, which enables them to provide and learn from their own feedback. Motivated by this unified capacity, we introduce UniEvo-VL, a self-evolving framework for multimodal models to learn from this constructive self-correction feedback during test-time compute. Instead of relying on a separate, often larger, teacher, we leverage their self-critiques as privileged information and ask a single multimodal model to act as both teacher and student with different contexts. The student only sees the vanilla question, while the teacher conditions on the privileged critique. Then training minimizes the per-state divergence between their denoising diffusion distributions over the student's own sampling trajectories. Experiments demonstrate that UniEvo-VL improves the image generation capabilities of multimodal models, while maintaining their sensitivity to additional reflection information. Specifically, we build on top of the open-source Qwen-image-2512 and observe a significant performance gain from 0.747 to 0.808 on GenEval and from 32.97 to 35.53 on GenEval2 Soft-TIFA. Moreover, attempts with more powerful external critics (e.g., GPT5.6-Luna) show that multimodal models with strong judge capabilities can anticipate a higher self-evolving ceiling. Last but not least, mixed text-rendering outcomes show that our self-improvements may not be uniform across different tasks. Our study aims to shed light on the current hot recursive self-improvement research line to enhance the user experience when using multimodal models without external supervision or guidance.