EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding

📄 arXiv: 2609.36803v1 📥 PDF

作者: Yuwei Miao, Xuesheng Zhang, Wenhao Zou, Jixia Zhang, Jianwei Lv, Bo Yuan, Junfeng Wang, Shiao Xie

分类: cs.CV

发布日期: 2026-09-29

💡 一句话要点

提出EGSD以解决流媒体视频理解中的自蒸馏问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 流媒体视频理解 自蒸馏 事件基础 记忆管理 多样性优化 实时处理

📋 核心要点

  1. 现有的自蒸馏方法在流媒体视频理解中面临学生模型无法端到端优化和有效实体记忆崩溃的挑战。
  2. 本文提出事件基础自蒸馏（EGSD），通过将流媒体记忆视为可验证事件的增量更新来解决上述问题。
  3. EGSD在StreamingBench和OVO-Bench实时轨道上分别取得79.8%和73.4%的性能提升，有效实体召回率提升17.4%。

📝 摘要（中文）

实时视频理解需要增量维护流媒体内容的记忆，并优化此过程需要密集的处理信号。现有的在政策自蒸馏（OPSD）方法存在两个主要问题：一是学生模型无法端到端优化，二是有效实体记忆崩溃。为了解决这些问题，本文提出了事件基础自蒸馏（EGSD），通过将流媒体记忆表征为可验证事件的增量更新，针对这两个问题进行了优化。实验结果表明，EGSD在StreamingBench和OVO-Bench实时轨道上分别达到了79.8%和73.4%的强劲表现，同时有效实体召回率提升了17.4%。

🔬 方法详解

问题定义：本文旨在解决流媒体视频理解中的自蒸馏问题，现有方法在优化过程中存在学生模型无法端到端优化以及有效实体记忆崩溃的痛点。

核心思路：EGSD通过将流媒体记忆表征为可验证事件的增量更新，调整OPSD信号为乘法权重，并引入事件作为特权信息，以解决学生与教师模型之间的偏差问题。

技术框架：EGSD的整体架构包括两个主要模块：一是基于事件的记忆更新模块，二是自蒸馏优化模块，后者结合了事件信息和覆盖奖励。

关键创新：EGSD的核心创新在于通过事件信息重加权教师模型，克服了传统方法中因问题相关性导致的偏见，增强了记忆的多样性。

关键设计：在设计中，采用了乘法权重与结果奖励的结合，增加了实体覆盖奖励，以补偿传统OPSD方法中缺乏的覆盖偏好。具体参数设置和损失函数设计未详述，需进一步研究。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

EGSD在主流基准测试中表现出色，StreamingBench上达到了79.8%的准确率，OVO-Bench实时轨道上为73.4%。此外，有效实体召回率提升了17.4%，仅增加了6.8%的记忆长度，显示出其在性能和效率上的显著优势。

🎯 应用场景

EGSD的研究成果在实时视频理解、智能监控、自动驾驶等领域具有广泛的应用潜力。通过优化流媒体内容的记忆管理，该方法能够提升系统对动态场景的理解能力，进而推动相关技术的实际应用和发展。

📄 摘要（原文）

Real-time video understanding requires incrementally maintaining a memory of streaming content, and optimizing this requires dense process signals. On-Policy Self-Distillation (OPSD), which lets one model serve as both teacher and student with the teacher receiving additional privileged information such as the question and ground-truth (GT) answer, can supply such token-level signals. However, applying it directly to streaming video raises two problems. (1) The student cannot be optimized end-to-end, where memory is written before the question arrives, yet the teacher scores it with the question-and-GT privilege, misaligning their preferences. (2) Effective-entity memory collapses, where the question-and-GT privilege makes the teacher favor only question-relevant entities, and token-mean averaging over a memory renders its signal invariant to how many entities that memory covers, both driving memory against the streaming need for diversity. To address these issues, we propose Event-Grounded Self-Distillation (EGSD), which characterizes streaming memory as an incremental update over verifiable Events (key visual entities, actions, and details) and targets the two problems on this basis. For problem (1), we adapt the OPSD signal into a multiplicative weight combined with the outcome reward; for problem (2), we re-weight the teacher with Events as privileged information to counter its question-relevance bias, and add an entity-coverage reward to supply the coverage preference the token-mean teacher lacks. Extensive experiments on mainstream online and offline benchmarks show EGSD achieves strong performance, reaching 79.8% on StreamingBench and 73.4% on the OVO-Bench Real-Time track, while memory analysis shows effective-entity recall rises 17.4% at only 6.8% more memory length.