TileShare: Training-Free Acceleration of Overlapping Tiled ViT Decoding
作者: Xiangzhong Ye, Zhongshen Li, Shen You, Ziyang Meng, Liang Xu, Junkai Ji, Qiuzhen Lin, Xiangtao Li, Ka-Chun Wong
分类: cs.CV, cs.LG
发布日期: 2026-10-23
备注: 8 pages main paper + supplementary material
💡 一句话要点
提出TileShare以加速重叠切片ViT解码，提升效率
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱三：空间感知与语义 (Perception & Semantics)
关键词: 视觉变换器 重叠切片 解码加速 视频处理 自蒸馏
📋 核心要点
- 现有的ViT解码器在处理重叠切片时存在大量计算重复，导致效率低下。
- TileShare通过共享重叠切片的注意力机制，减少计算冗余，从而加速解码过程。
- 在多个实验中，TileShare实现了显著的加速效果，并在多个性能指标上与传统方法相当或更优。
📝 摘要（中文）
固定窗口的视觉变换器（ViT）解码器在部署时以重叠切片的方式运行，导致每个重叠区域的计算重复。针对这一问题，TileShare提出了一种新方法，通过保留每个切片的注意力并对每个令牌的其他操作进行一次性计算，显著提高了解码速度。实验表明，TileShare在多个视频解码器上实现了1.38-1.51倍的加速，并在PSNR、LPIPS和DISTS指标上与传统解码器相当，且在未见数据上表现出色。
🔬 方法详解
问题定义：论文要解决的问题是固定窗口ViT解码器在重叠切片处理时的计算冗余，尤其是在大型模型中，导致效率低下和资源浪费。现有方法无法有效去除重叠切片的计算重复，影响了性能。
核心思路：TileShare的核心思路是保留每个切片的注意力，同时对每个令牌的其他操作进行一次性计算。通过将重叠区域的结果融合为一个共享状态，减少了计算量，同时保持了模型的性能。
技术框架：TileShare的整体架构包括多个模块，首先是重叠切片的生成，然后是注意力的保留和共享状态的计算，最后是输出结果的融合。该方法在多个解码器上进行了验证，展示了其通用性和有效性。
关键创新：TileShare的主要创新在于其无训练加速机制，通过共享注意力和融合计算结果，显著减少了重复计算。这一设计与传统的切片处理方法本质上不同，后者往往需要多次计算相同的操作。
关键设计：在关键设计上，TileShare保持每个切片的六层注意力，确保在解码过程中不会损失质量。此外，使用自蒸馏技术可以进一步提升模型性能，确保在共享所有层时不损失PSNR。
🖼️ 关键图片
📊 实验亮点
TileShare在MiniMax-H3视频解码器上实现了1.38-1.51倍的加速，使用FP8时可达1.96倍。该方法在PSNR、LPIPS和DISTS指标上与传统解码器相当，并在未见的UVG视频上表现出色，匹配PSNR和DISTS，同时改善LPIPS。
🎯 应用场景
TileShare的研究成果具有广泛的应用潜力，特别是在视频解码、图像处理和实时计算等领域。其高效的解码能力能够为长视频理解、实时视频流处理等场景提供支持，提升用户体验和系统性能。未来，该方法还可以扩展到其他类型的视觉任务，推动相关技术的发展。
📄 摘要（原文）
Fixed-window Vision Transformer (ViT) decoders run at deployment resolution as overlapping tiles: every token in an overlap band is processed by several full decoder instances. In the 2.4B-parameter MiniMax-H3 video decoder, about half of all per-token computation is duplicated. Removing the tiles fails because attention, global tokens and coordinates are bound to them. TileShare keeps each released tile's attention and computes every other operation once per token. The released computation changes at one kind of step only: an overlap token's per-tile results are blended into one shared state with the released weights. Interventions show that this blend costs quality where attention reads the shared states, mainly in the first few layers, which TileShare therefore keeps per tile. Without training, keeping six layers per tile decodes 1.38-1.51x faster than batched and compiled tiling on two GPU types (up to 1.96x with FP8) and is equivalent to the released decoder in PSNR, LPIPS and DISTS on DAVIS videos. On unseen UVG videos with the setting frozen beforehand, it matches PSNR and DISTS and improves LPIPS. TileShare carries over to two further decoders and is again confirmed on unseen data. It is equivalent in PSNR and DISTS on MAGI-1. On the ViTok-v2 image autoencoder at 2K-4K, it matches overlapping tiling in PSNR, improves LPIPS and DISTS and decodes 1.5-1.8x faster. How many layers stay per tile is measured once per model (for decoders, without labels); we report it for 14 decoders and encoders. Optionally, two GPU-hours of self-distillation let MiniMax-H3 share every layer without a loss in PSNR, and a coordinate field trained while the network stays frozen brings fully shared CLIP-based open-vocabulary segmentation to parity with sliding windows on seven benchmarks.