FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

📄 arXiv: 2607.17652v1 📥 PDF

作者: Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao

分类: cs.AI

发布日期: 2026-07-20

💡 一句话要点

提出FlowBlock以解决自校正扩散语言模型的解码效率问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 扩散语言模型 并行解码 自校正 Gated Wavefront Decoding Heterogeneous Wavefront Packing 自然语言处理 模型优化 性能提升

📋 核心要点

  1. 现有的块级扩散语言模型在解码过程中存在串行化的问题，导致效率低下和延迟增加。
  2. 本文提出FlowBlock，通过Gated Wavefront Decoding和Heterogeneous Wavefront Packing机制，实现了块间的并行解码，提升了解码效率。
  3. 实验结果显示，FlowBlock在多个基准测试中显著提高了TPS和准确性，相较于传统方法有显著优势。

📝 摘要（中文）

块级扩散大型语言模型（dLLMs）在块级别上顺序解码，虽然能够有效重用KV缓存，但导致块间解码严格串行。以往工作尝试通过后训练方法解锁块间并行性，但仅实现了适度的加速，且常常降低准确性。本文提出FlowBlock，一种无训练的并行解码框架，通过Gated Wavefront Decoding和Heterogeneous Wavefront Packing机制，显著提高了解码效率和准确性。实验结果表明，FlowBlock在多个基准测试中相较于LLaDA-2.1和LLaDA-2.0，TPS提升最高达2.95倍和4.01倍，同时延迟降低最高达53.6%和77.1%。

🔬 方法详解

问题定义：本文旨在解决块级扩散语言模型在解码过程中存在的串行化问题，导致效率低下和延迟增加。现有方法通过后训练手段尝试实现块间并行，但效果有限，且可能影响模型准确性。

核心思路：FlowBlock的核心思路是利用自校正的特性，通过无训练的方式实现块间并行解码。通过令下游块仅依赖于信息丰富的草稿，而非最终的前驱块，降低了块的最终性依赖，从而实现更高效的调度。

技术框架：FlowBlock整体架构包括两个主要机制：Gated Wavefront Decoding和Heterogeneous Wavefront Packing。前者通过满足准备门控条件来引入块，并通过T2T编辑共同优化活动块；后者则为每个请求分配独立的波前，同时将异步窗口打包成密集的、形状稳定的批量前向传递。

关键创新：FlowBlock的关键创新在于将块的最终性转变为调度资源，利用Gated Wavefront Decoding和Heterogeneous Wavefront Packing实现了高效的并行解码。这一设计与现有方法的本质区别在于不再依赖于最终的块，而是允许在信息草稿的基础上进行并行处理。

关键设计：在设计中，FlowBlock采用了窗口化的块因果掩码，以确保KV缓存的精确重用。此外，关键参数设置和网络结构设计均旨在优化解码效率和准确性，确保在不同基准测试中均能表现出色。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

在多个基准测试中，FlowBlock相较于LLaDA-2.1和LLaDA-2.0，TPS提升最高达2.95倍和4.01倍，延迟降低最高达53.6%和77.1%。此外，FlowBlock还提高了平均准确性1.3分，显示出其在解码效率和准确性上的显著优势。

🎯 应用场景

FlowBlock的研究成果在自然语言处理领域具有广泛的应用潜力，尤其是在需要快速响应的对话系统、实时翻译和大规模文本生成等场景。其高效的解码能力将推动相关技术的进步，提升用户体验和系统性能。

📄 摘要（原文）

Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial. Prior work has attempted to unlock inter-block parallelism through post-training methods, but achieves only modest speedups and often degrades accuracy. We observe that self-correcting dLLMs offer a training-free alternative: token-to-token (T2T) editing can repair tokens drafted with a slightly stale upstream context, so a downstream block requires only an informative draft rather than a finalized predecessor. This turns block finality from a hard dependency into a scheduling resource. We propose \textbf{\flowblock{}}, a training-free parallel decoding framework built on two mechanisms. (i) \emph{Gated Wavefront Decoding} admits blocks into a bounded wavefront only when a readiness gate is satisfied, jointly refines active blocks via T2T editing, and commits blocks in order under a windowed block-causal mask that preserves exact frozen-prefix KV caches reuse. (ii) \emph{Heterogeneous Wavefront Packing} assigns each request an independent wavefront while packing asynchronous windows into dense, shape-stable batched forwards. Across different benchmarks, \flowblock{} improves tokens per second (TPS) over LLaDA-2.1 and LLaDA-2.0, two serial block-wise dLLMs, by up to 2.95$\times$ and 4.01$\times$, while reducing latency by up to 53.6\% and 77.1\%, respectively. It also improves average accuracy by 1.3 points. Compared with D2F, a training-based inter-block-parallel baseline, \flowblock{} achieves higher accuracy and up to 16$\times$ higher batched serving throughput.