EAServe: Encode-Aware Disaggregated Serving for Multimodal Large Language Models
作者: Kunxiong Zhu, Zhihao Shu, Hangyu Zheng, Minghai Qin, Miao Yin, Gagan Agrawal, Wei Niu
分类: cs.DC, cs.LG, cs.PF
发布日期: 2026-09-25
备注: 13 pages, 12 figures, 7 tables. Accepted to PACT 2026
期刊: International Conference on Parallel Architectures and Compilation Techniques (PACT '26), October 19-22, 2026, Chicago, IL, USA
💡 一句话要点
提出EAServe以解决多模态大语言模型服务中的资源分配问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 多模态大语言模型 资源分配 GPU优化 动态微批处理 贝叶斯优化 编码阶段 预填充 解码
📋 核心要点
- 现有的多模态大语言模型服务在资源分配上存在结构性不平衡，导致编码阶段的GPU利用率低下。
- EAServe通过将编码阶段作为控制点，优化了下游请求的流动和GPU共享策略，从而提高了资源利用率。
- 在三种多模态架构的评估中，EAServe的良好吞吐量比NVIDIA Dynamo和vLLM分别高出4.3倍和1.7倍。
📝 摘要（中文）
将预填充和解码两个阶段分离到不同的GPU池已成为文本类大语言模型（LLM）服务的标准优化。然而，多模态大语言模型（MLLM）引入了第三个阶段——编码，这对资源分配提出了新的挑战。现有框架对这一问题的解决方案不够全面，导致编码GPU在高负载下严重低效，影响后续的预填充和解码工作。为此，本文提出EAServe，将编码重新定位为EPD管道的控制点，优化了GPU资源的利用率。通过在两个协同设计的层次上实现动态微批处理和混合自动选择，EAServe在三种MLLM架构上实现了显著的性能提升。
🔬 方法详解
问题定义：本文要解决的是多模态大语言模型（MLLM）服务中编码、预填充和解码三个阶段的资源分配问题。现有方法在编码阶段的GPU利用率低，导致后续处理阶段的性能瓶颈。
核心思路：EAServe的核心思路是将编码阶段重新定位为EPD管道的控制点，通过优化请求流动和GPU共享策略来提高整体资源利用率。
技术框架：EAServe的整体架构包括两个主要层次：运行时管理层和配置层。运行时层负责动态微批处理和部分卸载，而配置层则通过混合自动选择优化GPU分配和编码批量大小。
关键创新：EAServe的关键创新在于将编码阶段作为控制点，结合动态微批处理和基于贝叶斯优化的混合自动选择，显著提高了GPU的利用率和系统的整体性能。
关键设计：在设计中，EAServe采用了负载自适应的微批处理、速率控制的部分卸载策略，以及动态SM分区，以实现可预测的共存。这些设计使得系统能够在高负载下保持平衡的资源分配。
🖼️ 关键图片
📊 实验亮点
在对三种多模态大语言模型架构的评估中，EAServe在相同服务水平目标（SLO）约束下，良好吞吐量比NVIDIA Dynamo高出4.3倍，比vLLM高出1.7倍，且在整个EPD管道中实现了更高的GPU利用率和更快的接近最优配置。
🎯 应用场景
EAServe的研究成果在多模态大语言模型的实际应用中具有广泛的潜在价值，尤其是在图像、视频和音频处理等领域。通过优化资源分配，该方法能够提升多模态模型的响应速度和处理效率，适用于实时交互和大规模数据处理场景。
📄 摘要（原文）
Disaggregating the two stages, Prefill and Decode, onto separate GPU pools is now a standard optimization for (text-only) LLM serving. However, multimodal LLMs (MLLMs), which add a third phase, Encode, pose new challenges for resource allocation. Encode turns images, video, or audio into embeddings that the language model can consume, yielding a three-stage Encode-Prefill-Decode (EPD) pipeline. Existing frameworks offer only partial answers: text-only PD systems lack Encode, while EPD frameworks expose it as a separate service without regulating downstream request flow. The pipeline also carries a structural resource imbalance: every request enters through Encode before downstream work can begin, yet per-request execution leaves the encode GPU severely underutilized even at high loads, starving the downstream Prefill and Decode workers. Addressing this, we reposition Encode as the control point of the EPD pipeline, exposing three tightly coupled dimensions: when work enters downstream, where prefill executes, and how the GPU is shared. We instantiate this in EAServe across two co-designed layers. Its runtime manages load-adaptive micro-batching, rate-controlled partial offload to a co-resident prefill worker, and dynamic SM partitioning for predictable co-location. The configuration layer, Hybrid Auto Selection (HAS), navigates the joint space of GPU allocation, encode batch size, and offload ratio by pruning unbalanced allocations with per-stage capacity profiling and refining the remainder through TPE-based Bayesian optimization. Evaluated on three MLLM architectures spanning image, video, and audio, EAServe delivers up to 4.3x and 1.7x higher goodput than NVIDIA Dynamo and vLLM, respectively, under identical SLO constraints, sustains more balanced and higher GPU utilization across the EPD pipeline, and reaches near-optimal configurations faster than baseline search methods.