SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving

📄 arXiv: 2609.37626v1 📥 PDF

作者: Chuan Liu, Shuoming Zhang, Zhicheng Li, Qianqi Sun, Ruiyuan Xu, Qiuchu Yu, Xiyu Shi, Huimin Cui, Jiacheng Zhao

分类: cs.DC, cs.AI

发布日期: 2026-09-29

备注: 28 pages, 11 figures, 8 tables. Code: https://github.com/ict-agent/SPLASH-sglang

💡 一句话要点

提出SPLASH以解决大语言模型并行注意力布局切换问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 并行注意力 动态负载 去耦所有权并行性 服务系统 性能优化 调度算法

📋 核心要点

  1. 现有的并行化注意力方法在不同负载下表现不佳，固定布局无法适应请求的动态变化。
  2. SPLASH通过在请求运行时切换并行布局，利用现代注意力的去耦特性，实现了高效的布局切换。
  3. 在B200 GPU上，SPLASH的端到端服务吞吐量比固定布局提升了1.3-1.73倍，显示出显著的性能改进。

📝 摘要（中文）

在大语言模型的服务中，没有一种并行化注意力的方法能够在所有负载下表现良好。低并发适合张量并行，而许多独立请求则更适合数据并行注意力，长提示则倾向于上下文并行。由于请求的性质变化，固定的布局选择变得不可行。为此，本文提出了SPLASH，一个能够在请求运行时切换注意力并行布局的服务系统。SPLASH通过重用状态和在后台移动剩余数据，实现了几乎免费的布局切换，且引入了去耦所有权并行性（DOP），显著提高了KV缓存的容量。实验表明，SPLASH在B200 GPU上提升了1.3-1.73倍的端到端服务吞吐量。

🔬 方法详解

问题定义：本文旨在解决大语言模型在服务过程中，因请求性质变化而导致的并行注意力布局固定选择不适应的问题。现有方法在负载变化时无法灵活调整，造成资源浪费和性能下降。

核心思路：SPLASH的核心思路是利用现代注意力机制的去耦特性，在请求运行时动态切换并行布局。通过重用已有状态并在后台移动数据，SPLASH实现了几乎无缝的布局切换。

技术框架：SPLASH的整体架构包括请求管理模块、布局切换模块和调度器。请求管理模块负责接收和处理请求，布局切换模块根据当前负载动态调整并行布局，调度器则优化KV缓存的使用。

关键创新：SPLASH引入了去耦所有权并行性（DOP），该方法在保持每个请求的KV缓存独占的同时，像张量并行一样分片注意力权重。DOP提供了比数据并行注意力更高的KV容量，且不需要进行重复复制。

关键设计：在设计中，SPLASH重用大部分状态，只有在必要时才移动数据，确保切换过程的高效性。调度器根据负载变化选择最佳布局，确保系统在不同请求负载下的性能最优。实验中，SPLASH的切换开销低于0.51%。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

在B200 GPU上，SPLASH的端到端服务吞吐量相比固定布局提升了1.3-1.73倍，展示了其在动态负载下的优越性能。此外，SPLASH的布局切换开销低于0.51%，确保了高效的实时响应能力。

🎯 应用场景

SPLASH的研究成果在大语言模型的服务场景中具有广泛的应用潜力，尤其是在需要处理动态请求负载的情况下。其高效的布局切换机制可以显著提升模型的响应速度和吞吐量，适用于在线服务、智能助手和实时对话系统等领域，未来可能推动更高效的AI服务架构的发展。

📄 摘要（原文）

No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independent requests favor data-parallel attention, and long prompts favor context parallelism. Reasoning, agentic, and RL-rollout workloads make a fixed choice untenable: a batch that begins as many short requests ends as a few very long ones, so the best layout changes while the same requests run. Serving engines nevertheless fix one layout at launch, because changing it has meant draining requests and restarting workers. We present SPLASH, a serving system that switches the parallel layout of attention while requests are running. It builds on one observation: modern attention, with few or no KV heads, decouples where a request's KV cache lives from how attention weights are sharded. This has two consequences. First, layouts differ only in who owns the weights and the cache, and most of that state already sits where the next layout needs it; SPLASH reuses it, moves the rest in the background of ongoing inference, and hands off at a batch boundary, making a switch nearly free: its median overhead is under 0.51% of the step it runs in. Second, the decoupling exposes a layout that existing engines lack: Decoupled Ownership Parallelism (DOP) shards attention weights as tensor parallelism does while keeping each request's cache on a single owner as data-parallel attention does. DOP replicates neither, offers 27-60% more KV capacity than data-parallel attention, and gives the scheduler a choice when KV memory limits admission. A transition-aware scheduler follows the best of the four layouts as load changes. On B200 GPUs serving GLM-5.3, SPLASH improves end-to-end serving throughput by 1.3-1.73x over fixed-layout deployments, and the same layout regimes appear with DeepSeek-V3.2 on H200 and GLM-5.3-Flash on DCU.