On-Policy Distillation for Vision-Language Model Adaptation, an Effective Paradigm on Low-Quality Multimodal Data

📄 arXiv: 2609.10321v1 📥 PDF

作者: Hongyuan Zhang, Xianda Guo, Yanlun Peng, Qianlong Yang, Yubin Guo, Pinhan Fu, Mulin Chen, Xiaozhen Qiao, Ping Luo

分类: cs.CL

发布日期: 2026-09-09

💡 一句话要点

提出OnPoKD框架以解决低质量多模态数据下的视觉语言模型适应问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 知识蒸馏 视觉语言模型 多模态数据 动态目标构建 模型适应性 跨数据集转移 深度学习

📋 核心要点

  1. 现有视觉语言蒸馏方法通常使用固定的教师预测作为训练目标，导致在类别和领域转移时的可靠性不足。
  2. 本文提出OnPoKD框架，通过动态构建样本自适应目标，优化蒸馏过程中的目标构建决策。
  3. 在多项基准测试中，OnPoKD在视觉语言蒸馏的性能上显著优于现有强基线，展示了其有效性。

📝 摘要（中文）

知识蒸馏为将任务适应的视觉语言教师模型转移到紧凑的学生模型提供了一种高效的途径。然而，现有的视觉语言蒸馏方法通常采用固定的教师预测作为训练目标，导致在类别和领域转移时的可靠性不足。本文提出OnPoKD框架，将蒸馏目标构建视为动态训练决策，学习样本自适应目标。OnPoKD通过轻量级控制器动态平衡教师监督、零样本先验指导和硬标签锚定，适应样本可靠性和训练阶段的变化。实验结果表明，OnPoKD在基于基准到新颖的泛化和跨数据集转移任务上，显著提升了视觉语言蒸馏的性能。

🔬 方法详解

问题定义：本文旨在解决现有视觉语言蒸馏方法在类别和领域转移时的可靠性不足问题。现有方法通常采用固定的教师预测作为训练目标，缺乏动态适应性。

核心思路：OnPoKD框架的核心思路是将蒸馏目标的构建视为动态训练决策，通过学习样本自适应目标来提高模型的适应性和转移能力。

技术框架：OnPoKD框架包括一个轻量级控制器，该控制器根据教师模型、学生模型和零样本先验的可靠性和不一致性信息，动态构建样本自适应目标。控制器在训练过程中根据验证反馈进行更新，以优化目标构建。

关键创新：OnPoKD是首个将在线策略蒸馏应用于视觉语言模型适应的框架，通过动态平衡教师监督和零样本先验指导，显著提升了蒸馏目标的适应性。

关键设计：控制器的设计允许通过有界策略动作动态调整教师监督、零样本先验和硬标签的权重，确保蒸馏目标能够适应不同样本的可靠性和训练阶段。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在基于基准到新颖的泛化和跨数据集转移任务上，OnPoKD在多个强基线模型中表现出显著提升，具体实验结果显示其在某些任务上性能提升幅度达到XX%，验证了其有效性和优越性。

🎯 应用场景

该研究的潜在应用领域包括自然语言处理与计算机视觉的结合，如图像描述生成、视觉问答等。通过提升视觉语言模型在低质量多模态数据上的适应能力，OnPoKD有望在实际应用中提高模型的泛化能力和鲁棒性，推动相关技术的发展。

📄 摘要（原文）

Knowledge distillation offers an efficient route to transfer a task-adapted vision-language teacher to a compact student. The training target in current vision-language distillation methods is typically constructed from the teacher prediction and applied uniformly to all training samples, making it unreliable under class and domain shifts. In this paper, we argue that distillation target construction should be treated as a dynamic training decision rather than a fixed recipe. To this end, we propose OnPoKD, an on-policy distillation framework for vision-language model adaptation. To the best of our knowledge, OnPoKD is the first framework that applies on-policy distillation to vision-language model adaptation by learning target construction as a policy decision. OnPoKD learns a lightweight controller that constructs sample-wise adaptive targets using reliability and disagreement cues from the teacher model, student model, and zero-shot prior. Instead of relying on a fixed teacher prediction, the controller dynamically balances teacher supervision, zero-shot prior guidance, and hard-label anchoring through bounded policy actions, allowing the distillation target to adapt to varying sample reliability and training stages. The policy controller is updated with validation feedback, encouraging target construction to optimize transferability rather than merely fitting the training distribution. Since the controller is only used during training, OnPoKD can be seamlessly integrated into existing vision-language distillation pipelines while preserving the original inference architecture and test-time cost. Extensive experiments on Base-to-novel generalization and Cross-dataset transfer benchmarks show that OnPoKD consistently improves over strong vision-language distillation baselines.