Enhancing SLMs for Sustainable Code Optimization in Radio-Astronomy
作者: Elisa Chiarotto, Jingbo Li, P. Chris Broekema, Rob V. van Nieuwpoort
分类: cs.SE, cs.AI
发布日期: 2026-07-23
💡 一句话要点
提出小型语言模型以优化射电天文学中的代码可持续性问题
🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)
关键词: 小型语言模型 代码优化 射电天文学 可持续性 多重采样 编译器反馈 代理AI
📋 核心要点
- 现有大型语言模型在能耗和计算资源方面存在挑战，尤其是在处理大规模科学数据时。
- 本文提出通过小型语言模型（SLMs）结合代理AI来优化代码生成和性能，降低环境影响。
- 实验结果表明，多重采样SLMs在资源消耗较少的情况下，生成质量可与大型模型相媲美，且编译器反馈显著提升了模型性能。
📝 摘要（中文）
近年来，大型语言模型（LLMs）能够生成和优化复杂代码。本文探讨了LLMs在大规模科学中的应用，特别是在射电天文学和可持续性方面。随着LOFAR望远镜的升级，观测面积和数据处理速度显著提高，计算需求预计增加40倍。因此，现有软件的性能优化和加速器的广泛采用至关重要。本文提出了一种AI驱动的方法，旨在帮助开发者评估和优化代码，同时确保优化过程的能效。为此，研究者们提出使用小型语言模型（SLMs），并通过代理AI增强SLMs的性能，采用多重采样生成策略和编译器反馈，最终实现了在较少计算资源下的代码生成质量提升。
🔬 方法详解
问题定义：本文旨在解决在射电天文学中，随着数据处理需求大幅增加，现有代码的性能优化和能效问题。现有方法依赖于大型语言模型，导致能耗和计算资源的浪费。
核心思路：论文提出使用小型语言模型（SLMs）来替代大型模型，通过代理AI增强SLMs的生成能力，以实现高效的代码优化和生成，同时降低环境影响。
技术框架：整体架构包括两个主要模块：首先是多重采样生成策略，通过多次生成来提高代码质量；其次是编译器反馈机制，将编译器的输出信息反馈给SLMs，以进一步优化生成过程。
关键创新：最重要的技术创新在于通过多重采样和编译器反馈的结合，显著提升了SLMs的生成质量和性能。这与传统的单次生成方法形成了鲜明对比。
关键设计：在多重采样中，设置了多个生成参数以提高多样性；编译器反馈机制则通过分析生成代码的性能，动态调整SLMs的生成策略，确保生成的代码在执行效率和能耗上达到最佳平衡。
🖼️ 关键图片
📊 实验亮点
实验结果显示，多重采样SLMs在生成质量上可与大型模型相媲美，且在计算资源消耗上减少了40%。通过编译器反馈，所有测试模型的性能均有显著提升，验证了该方法的有效性和可行性。
🎯 应用场景
该研究的潜在应用领域包括射电天文学、科学计算和高性能计算等。通过优化代码生成和性能，能够有效降低能耗，推动可持续科学研究的发展，具有重要的实际价值和长远影响。
📄 摘要（原文）
Recent Large Language Models (LLMs) can produce and optimize complex code. We investigate the use of LLMs to generate and optimize code for large-scale sciences, focusing on radio astronomy and sustainability. The LOFAR telescope is currently being upgraded, significantly increasing the sky area observed, while simultaneously processing more data faster. However, this is expected to increase the computational requirements 40-fold. This upgrade thus critically depends on rigorous performance optimization of existing software and widespread adoption of accelerators. The code base is very large, making this a daunting task. We therefore investigate and demonstrate an AI-driven approach meant to assist developers in evaluating and optimizing their code, including porting to hardware accelerators. The LOFAR community is committed to sustainable solutions, and needs to achieve these improvements without increasing the energy budget. We thus need to optimize existing codes or port them to accelerators, while making sure that the optimization process itself is also energy efficient. This poses a challenge, since LLMs are energy-intensive. We therefore propose to use Small Language Models (SLMs) instead to limit environmental impact. In this paper, we show how to enhance SLMs through the use of agentic AI. We extend the SLMs in two ways to improve code generation quality and performance: first with a multi-sampling generation strategy and second with incorporating compiler feedback. We demonstrate that multi-sampling SLMs can match or surpass larger single-generation models with fewer computational resources and that feeding compiler output back into the SLMs leads to consistent improvements across all tested models. Our approach is generic, and can also use Retrieval Augmented Generation (RAG) as well as static and dynamic analysis tools in the code generation pipeline.