Green Code or Costly Code? Evaluating the Energy Costs of Iterative LLM-Based Code Optimizations

📄 arXiv: 2610.32891v1 📥 PDF

作者: Sadra Emami, Benjamin Weigell, Mohammad Reza Mousavi, Gunel Jahangirova, Hugo Araujo, Bernhard Bauer

分类: cs.SE, cs.AI

发布日期: 2026-10-23

备注: Accepted at the 49th IEEE/ACM International Conference on Software Engineering (ICSE 2027)

💡 一句话要点

评估LLM代码优化的能耗与性能之间的关系

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 代码优化 能耗评估 性能基准 Mixture-of-Experts 软件开发 可持续性

📋 核心要点

  1. 现有方法假设代码性能优化必然降低能耗，但实际情况可能并非如此。
  2. 通过对11个开源模型在EffiBench基准上的评估，实证研究了LLM自我优化的能效。
  3. 结果显示42个配置的代码能耗高于人类参考，架构选择对能效影响显著。

📝 摘要（中文）

大型语言模型（LLMs）越来越多地用于优化代码性能。本文通过实证研究质疑了性能提升是否必然导致能耗降低的假设。我们测量了LLM自我优化生成的代码相对于人类参考解决方案的能效，并评估了优化过程的能耗是否能通过执行时间节省来抵消。研究涵盖11个开源模型在EffiBench基准上的表现，结果显示大多数优化代码的能耗高于人类编写的代码，且架构选择对能效的影响显著。所有研究成果均可在指定链接获取。

🔬 方法详解

问题定义：本文旨在解决LLM优化代码时能耗与性能之间的关系，现有方法未能充分考虑优化过程的能耗。

核心思路：通过实证测量LLM生成的代码与人类编写代码的能效，挑战了性能提升与能耗降低之间的假设。

技术框架：研究使用EffiBench基准，评估11个开源模型在不同解码配置下的表现，收集硬件级能耗数据。

关键创新：首次系统性评估LLM自我优化代码的能效，揭示了性能优化与能效之间的矛盾。

关键设计：研究中限制输出token数量显著降低能耗，且Mixture-of-Experts架构在能效上优于传统密集模型。具体的能耗数据在0.79到8.68 MJ之间，显示出不同配置的能效差异。

📊 实验亮点

实验结果显示，42个配置的代码能耗均高于人类参考，能耗增加幅度在4%到886%之间。限制输出token数量可减少能耗高达69%。Mixture-of-Experts架构在能效上表现优于同等规模的密集模型，显示出架构选择的重要性。

🎯 应用场景

该研究对软件开发领域具有重要的应用价值，尤其是在追求高性能与低能耗的场景中。通过优化代码的能效，开发者可以在保证性能的同时，降低运行成本和环境影响。未来，LLM在代码优化中的应用可能会推动更可持续的编程实践。

📄 摘要（原文）

Large Language Models (LLMs) are increasingly used to optimize code for performance. This practice often assumes that improvements in execution speed naturally translate into overall lower energy consumption. We challenge this assumption empirically by measuring whether performance-optimized code produced by LLM self-refinement pipelines is also energy-efficient relative to canonical human reference solutions, and whether the energy cost of the optimization process can be offset by subsequent execution-time savings. We evaluate 11 open-source models across four decoding configurations on all 1,000 problems of EffiBench, a state-of-the-art performance benchmark, while collecting per-process, hardware-level energy measurements throughout. Inference energy spans 0.79 to 8.68 MJ across model-variant pairs, which is roughly equivalent to the energy of 250 to 2800 GPT-4.1 queries. Limiting the maximum number of output tokens has the largest impact, reducing energy use by up to 69% compared to the default configuration. On the execution side, 42 of 44 configurations produce code that consumes more energy per problem than the canonical human reference, with overheads ranging from +4% to +886%. The two exceptions achieve only marginal gains, with break-even points on the order of 10^6 to 10^7 executions. Mixture-of-Experts architectures consume substantially less inference energy than equivalently-sized dense models without sacrificing correctness, making architectural choice a more consequential sustainability lever than decoding parameters. Performance optimization and energy efficiency are not always aligned goals: LLM self-refinement consistently produces faster-seeking but energy-heavier code relative to what a human engineer would write. All artifacts are available at https://doi.org/10.5281/zenodo.23598170.