PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling

📄 arXiv: 2609.09662v1 📥 PDF

作者: Weisi Yang, Stephen Xia

分类: cs.LG, cs.CL, cs.OS, cs.PF

发布日期: 2026-09-09

备注: Accepted to ACM/IEEE SenSys'26

🔗 代码/项目: GITHUB

💡 一句话要点

提出PELM以解决移动设备上LLM推理的能效问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 动态电压频率调整 能效优化 移动计算 投机解码 可变验证深度 边缘计算

📋 核心要点

  1. 现有的动态电压和频率调整方法在热限制场景下无法有效降低LLM的能耗和热量生成。
  2. PELM通过引入投机解码和可变验证深度，结合传统的DVFS方法，扩展了优化空间以实现更高效的推理。
  3. 实验表明，PELM在性能上优于现有的能量管理方法，速度提升23.1%，能耗降低52.4%。

📝 摘要（中文）

在移动平台上直接部署大型语言模型（LLM）因其隐私性、个性化和低延迟等优点而受到关注。然而，LLM的高计算需求使得资源受限的移动和边缘平台难以满足。现有的动态电压和频率调整（DVFS）方法主要集中于优化硬件参数，未能有效应对热限制场景。为此，本文提出PELM，通过引入投机解码和可变验证深度，扩展了优化空间，实现更高效的LLM推理。实验结果表明，PELM在多个硬件平台和数据集上表现优异，速度提升可达23.1%，能耗降低52.4%，同时保持任务性能。

🔬 方法详解

问题定义：本文旨在解决在移动设备上进行大型语言模型推理时的高能耗和热量生成问题。现有的DVFS方法主要优化硬件参数，未能有效应对热限制场景，导致处理能力下降。

核心思路：PELM的核心思路是识别并利用并非所有token都需要全深度推理的事实，从而通过投机解码和可变验证深度来提高能效。这样的设计使得在保持生成质量的同时，降低计算资源的消耗。

技术框架：PELM的整体架构包括三个主要模块：传统的DVFS频率调节模块、投机解码模块和可变验证深度模块。通过这三个模块的协同工作，PELM能够在不同的计算需求下动态调整推理策略。

关键创新：PELM的关键创新在于引入了投机解码和可变验证深度这两个工作负载特定的调节机制。这与传统方法的单一硬件参数优化形成了鲜明对比，极大地扩展了优化空间。

关键设计：在PELM中，投机解码允许模型在推理过程中对部分token进行快速解码，而可变验证深度则根据token的重要性动态调整推理深度。这些设计使得模型在能耗和性能之间取得了良好的平衡。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

PELM在多个硬件平台和数据集上的实验结果显示，速度提升可达23.1%，能耗降低52.4%。这些结果表明，PELM在能效和性能上均优于现有的能量管理方法，具有显著的实际应用价值。

🎯 应用场景

PELM的研究成果在移动设备、边缘计算和智能终端等领域具有广泛的应用潜力。通过提高LLM推理的能效，PELM能够支持更复杂的应用场景，如实时语音助手、个性化推荐系统和智能客服等，推动智能设备的普及与发展。

📄 摘要（原文）

Deploying Large Language Models (LLMs) directly on mobile platforms at the edge is gaining traction due to a myriad of benefits, such as increased privacy, personalization, and reduced latency. However, LLMs have heavy computational requirements, which are difficult for resource-constrained mobile and edge platforms to fulfill. In addition to limited compute resources, mobile and edge systems often have a compact form factor and lack physical mechanisms to dissipate heat generated from high processor usage rates (e.g., fans) to prevent throttling and reduced processing power, which LLMs can easily cause. To mitigate these effects, prior works have proposed various power governing strategies, such as dynamic voltage and frequency scaling (DVFS), for reducing power and heat generation for heavy computational tasks on mobile platforms. Recently, DVFS methods tailored for mobile LLMs have also been proposed. However, these methods mostly focus on optimizing hardware parameters and processor frequencies, and they fall short under some thermally constrained scenarios. Drawing from recent advances in machine learning, we identify and take advantage of the key insight that not all tokens require full-depth inference to maintain high-quality generation. Motivated by this, we present PELM, a solution that augments traditional DVFS processor frequency tuning with two additional workload-specific knobs: 1) speculative decoding and 2) variable verification depth to expand the optimization space to multiple dimensions for more power efficient on-device LLM inference. In extensive evaluations across hardware platforms and datasets, PELM demonstrates superior performance compared to state-of-the-art power governing methods, with up to 23.1% speedup and 52.4% reduction in energy consumption, while maintaining comparable task performance. The source code is available at https://github.com/imec-nu/PELM.