Zephyrus-R1: Agentic Reinforcement Learning for Weather Reporting
作者: Fengfei Yu, Marshall Fisher, Isadora White, Sumanth Varambally, Yasaman Jafari, Shivam Yadav, Samuel M. Bartlett, Shawn Roj, Benjamin J. Moore, Michael D. Warner, Luyu Han, Zhirui Xia, Duncan Watson-Parris, Taylor Berg-Kirkpatrick, Yian Ma, Rose Yu
分类: cs.LG
发布日期: 2026-10-19
备注: 27 pages, including references and appendices
💡 一句话要点
提出Zephyrus-R1以解决天气报告中的RL验证难题
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)
关键词: 强化学习 天气预报 自然语言处理 奖励设计 课程学习 模型优化 科学推理
📋 核心要点
- 现有的天气报告生成方法在输出的验证上存在困难，尤其是自然语言输出的多样性使得确定性验证受限。
- 本文提出的Zephyrus-R1框架通过设计结合气象标准的奖励函数，优化了天气报告的生成过程，提升了报告的可读性和准确性。
- 实验结果表明，经过Zephyrus-R1训练的Qwen3.5-4B模型在多个评估指标上显著提升，表现出与前沿开放模型的竞争力。
📝 摘要（中文）
强化学习（RL）已成为训练科学应用AI模型的关键方法，但在输出无法轻易验证的情况下，RL仍然面临挑战。自然语言天气预报报告就是一个例子，它将大气数据转化为天气演变、危险和不确定性的评估。为此，本文开发了Zephyrus-R1，一个用于科学天气预报报告的RL实用框架。我们专注于奖励设计、课程学习和人类专家对齐分析三个关键组件。通过结合气象标准和双向声明蕴涵的奖励函数，我们在模拟工具调用环境中优化该奖励，提升了报告的可读性和事实准确性。经过训练，Qwen3.5-4B模型在六个评估指标上平均提升了26%。
🔬 方法详解
问题定义：本文旨在解决天气报告生成中强化学习的验证难题，现有方法在自然语言输出的多样性和不确定性上难以进行有效的验证。
核心思路：Zephyrus-R1框架通过设计一个结合气象标准和双向声明蕴涵的奖励函数，促进报告的可读性和事实准确性，从而优化天气报告的生成过程。
技术框架：该框架包含三个主要模块：奖励设计、课程学习和人类专家对齐分析。首先，通过奖励设计引导模型生成高质量报告；其次，课程学习帮助模型逐步掌握工具使用和科学推理；最后，通过人类专家的审核确保生成内容的准确性。
关键创新：最重要的创新点在于结合气象标准与双向声明蕴涵的奖励函数设计，使得模型在生成多样化报告时仍能保持高准确性。这一方法与传统的基于确定性验证的RL方法有本质区别。
关键设计：在奖励函数中，采用了气象标准与国家气象局参考报告进行评估，确保生成报告的质量。此外，课程学习策略使得小模型能够在可验证的天气分析问题上先行训练，逐步优化开放式报告的奖励。
🖼️ 关键图片
📊 实验亮点
在实验中，经过Zephyrus-R1训练的Qwen3.5-4B模型在六个评估指标上平均提升了26%，而Qwen3.8-27B模型的提升幅度为13%。这些结果显示出该框架在天气报告生成中的有效性，且与Kimi-K3和GPT-5.6-Luna等前沿模型相比，均表现出显著的竞争力。
🎯 应用场景
Zephyrus-R1框架的潜在应用领域包括气象预报、灾害管理和公共安全等。通过提高天气报告的准确性和可读性，该研究能够为决策者提供更可靠的信息支持，进而提升应对气候变化和极端天气事件的能力。未来，该框架还可扩展至其他科学领域的报告生成任务。
📄 摘要（原文）
Reinforcement learning (RL) has emerged as a key method for training AI models in scientific applications, but RL remains difficult when outputs cannot easily be verified against a known ground-truth answer. Natural-language weather forecast reports are one such domain: they translate atmospheric data into assessments of weather evolution, hazards, and uncertainty for both the general public and key decision-makers. Unlike numerical weather-analysis questions, they admit many valid outputs, which limits deterministic verification. To address this, we develop Zephyrus-R1, a practical framework for RL on scientific weather forecast reporting. We focus on three key components of the agentic RL pipeline: reward design, curriculum learning, and an analysis of human-expert alignment. To encourage both report legibility and factual accuracy, we design a reward function that combines a meteorological rubric with bidirectional claim entailment, both evaluated against National Weather Service reference reports. Our framework optimizes this reward in a simulated tool-calling environment, enabling vision-language agents to gather and analyze forecast evidence across regions and lead times before producing a report. We use annotations from three meteorologists to audit the claim labels produced by our LLM judges. Finally, a curriculum helps small models first develop tool use and scientific reasoning through RL on verifiable weather-analysis questions, then optimize the open-ended report reward. After training Qwen3.5-4B with Zephyrus-R1, we improve the mean across six judge-metric scores by 26% over the base model and achieve results competitive with frontier open-weight models. For Qwen3.8-27B, we improve the same mean by 13% over the base model, by 6.7% over Kimi-K3, and by 2.7% over GPT-5.6-Luna.