An Error Budget for Shuffled Multimodal Windows in the 3rd WEAR Challenge: Stacking, Count Constraints and Seam Matching

📄 arXiv: 2610.32398v1 📥 PDF

作者: Soheil Kooklan

分类: cs.CV

发布日期: 2026-10-23

备注: 12 pages, 10 tables, 1 figure. Technical report on a post-deadline system for the 3rd WEAR Dataset Challenge at HASCA 2026. Code: https://github.com/soheilkooklan/wear-seam-decoder

💡 一句话要点

提出基于堆叠分类器的多模态窗口活动识别方法

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 多模态窗口 活动识别 堆叠分类器 均值场解码器 软计数约束 数据挑战 性能提升

📋 核心要点

  1. 现有方法在处理打乱的多模态窗口时，活动标签的准确性和一致性存在挑战。
  2. 论文提出通过堆叠分类器和均值场解码器结合软计数约束的方法来提高活动识别的准确性。
  3. 实验结果显示，堆叠方法和计数约束分别提升了宏F1分数0.051和0.011，系统整体得分达到0.936。

📝 摘要（中文）

第三届WEAR数据集挑战要求为四名未见参与者的每一秒窗口提供活动标签。每个窗口包含15个VideoMAEv2帧嵌入和一个随机肢体的加速度信号，且窗口顺序被打乱。本文对在挑战截止日期后构建的系统进行了实证分析。通过对每个参与者的图形邻居进行特征平均，堆叠窗口分类器并对候选标签进行评分，同时使用一种均值场风格的解码器对每个参与者的所有窗口进行软计数约束。实验结果显示，堆叠方法在宏F1分数上提升了0.051，而计数约束提升了0.011。最终系统在模拟中得分为0.936，参与者得分范围为0.80到0.97。

🔬 方法详解

问题定义：本文旨在解决在打乱的多模态窗口中进行活动识别的准确性问题。现有方法在处理不同参与者的活动标签时，容易出现标签不一致和识别精度不足的情况。

核心思路：论文的核心思路是通过堆叠分类器和均值场解码器结合软计数约束来提升活动识别的准确性。通过对每个参与者的窗口进行特征平均，增强了模型对活动标签的理解。

技术框架：整体架构包括多个模块：首先对每个参与者的窗口进行特征提取和堆叠分类；然后使用均值场解码器对所有窗口进行解码，并施加软计数约束，确保每个活动标签在每个参与者中只出现一次。

关键创新：最重要的技术创新在于结合了堆叠分类器和均值场解码器的设计，特别是软计数约束的引入，使得模型在处理多模态数据时更加有效。与现有方法相比，该方法在准确性和一致性上有显著提升。

关键设计：在参数设置上，使用了参与者级别的自助法进行模拟，损失函数设计考虑了标签不一致性，网络结构采用了多层堆叠分类器以增强特征学习能力。

📊 实验亮点

实验结果显示，堆叠分类器方法在宏F1分数上提升了0.051，计数约束提升了0.011，最终系统得分达到0.936。参与者得分范围从0.80到0.97，表明该方法在不同个体间具有良好的适应性和准确性。

🎯 应用场景

该研究的潜在应用领域包括智能监控、健康监测和人机交互等。通过提高活动识别的准确性，可以在医疗、运动分析和智能家居等领域实现更智能的应用，提升用户体验和安全性。

📄 摘要（原文）

The 3rd WEAR Dataset Challenge asks for an activity label for every one-second window of four unseen participants. Each window has 15 VideoMAEv2 frame embeddings, the accelerometer signal of one random limb, and the windows are shuffled. This report is an empirical analysis of one system built after the challenge deadline. Window classifiers are stacked with features averaged over per-participant graphs of video neighbours and scored successor candidates, and a mean-field style decoder with a soft count constraint per activity labels all windows of a participant together. The setting is offline and transductive, and the count constraint relies on each activity being performed once. In a participant-wise simulation with participant-level bootstrap intervals, stacking added 0.051 macro F1 over stage-1 fusion and the count constraint 0.011 (95% interval 0.006 to 0.017). A quadratic seam distance ranks the true same-limb successor first among about 720 candidates in 83.9% of cases, but adding it to the link model changed macro F1 by -0.0001 (-0.0006 to 0.0005). With the decoder settings and the mixing weight chosen inside the folds and the mislabelled spans left out of training, the system scores 0.936 in simulation (0.940 for participants who performed each activity once); the submitted configuration scores 0.938 (0.943). Per-participant scores range from 0.80 to 0.97. These results come from a non-nested stacking protocol that uses unlabelled features transductively, so they are not a fully inductive estimate. An oracle that averages the final unary scores over the true segments reaches 0.973, and 79% of the remaining errors involve the null class. On the private leaderboard, which no choice used, the selected Kaggle submission scored 0.9231 (public 0.9254, with bounds partly tuned there), and a later one with bounds chosen in simulation scored 0.9247 (public 0.9253).