ACEA: An Adversarial Co-Evolution Arena for Head-to-Head Red-Team and Blue-Team LLM Testing

📄 arXiv: 2609.08256v1 📥 PDF

作者: Yi Ting Shen, Kentaroh Toyoda, Alex Leung

分类: cs.CR, cs.AI

发布日期: 2026-09-08

备注: Code is available at https://github.com/VulcanLab/ACEA

💡 一句话要点

提出ACEA以解决大型语言模型的红队与蓝队测试问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 对抗性训练 大型语言模型 安全性测试 红队蓝队 模型评估 实时反馈 可视化报告

📋 核心要点

  1. 现有的红队和蓝队测试往往在孤立环境中进行，导致评估结果的可靠性不足。
  2. ACEA通过连接红队和蓝队适配器到共享LLM，提供了一个可插拔的测试平台，支持实时评估和反馈。
  3. 实验表明，ACEA能够有效分解攻击强度和防御效果，为项目改进提供了明确的信号。

📝 摘要（中文）

随着自动化红队攻击和蓝队防御技术的快速发展，现有的攻击者和防御者往往在孤立环境中构建和测试，导致结果难以信任。为此，本文提出了ACEA（对抗共演竞技场），这是一个将可插拔的红队适配器和蓝队适配器连接到共享目标大型语言模型（LLM）的平台，并通过LLM评估其攻击和防御率。ACEA的贡献包括可插拔的模型无关竞技场、针对对抗回合的评估方法、实时游戏风格的可视化以及可选的上下文改进循环，旨在为红队和蓝队项目提供可操作的改进信号。

🔬 方法详解

问题定义：本文旨在解决大型语言模型（LLM）测试中红队与蓝队之间的孤立评估问题，现有方法导致攻击和防御效果难以信任。

核心思路：ACEA通过建立一个共享的对抗竞技场，使红队和蓝队能够在同一环境中进行测试，从而实现实时评估和反馈，提升测试的可靠性。

技术框架：ACEA的整体架构包括四个主要模块：可插拔的竞技场、针对对抗回合的评估方法、实时可视化报告和可选的改进循环。每个模块通过ACEA标准适配器协议（ASAP）进行连接。

关键创新：ACEA的核心创新在于其可插拔的设计和对抗回合的评估方法，能够独立测量攻击的原始强度和防御的有效性，这与现有方法的孤立评估形成了鲜明对比。

关键设计：ACEA采用了最小的HTTP协议进行连接，支持多种编程语言；评估方法通过种子秘密提供可验证的真实数据，并在防御阻止攻击时仍然记录攻击尝试，以便全面评估攻击效果。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，ACEA能够有效分解攻击强度和防御效果，提供清晰的改进信号。与传统方法相比，ACEA在攻击和防御的评估准确性上有显著提升，具体性能数据尚未公开。

🎯 应用场景

ACEA的设计为大型语言模型的安全性测试提供了一个创新的平台，能够广泛应用于人工智能安全、模型评估和对抗性训练等领域。通过实时反馈和可视化报告，研究人员和开发者可以更有效地改进其模型的防御能力，提升整体安全性。

📄 摘要（原文）

Automated red-team attacks and blue-team defenses for large language models (LLMs) are advancing quickly. However, attackers and defenders are built and tested in isolation, and the resulting scores are hard to trust. To tackle this, we present ACEA (Adversarial Co-Evolution Arena), a platform that connects a pluggable red-team adapter and a pluggable blue-team adapter to a shared target LLM and scores their attack and defense rates with an LLM judge. ACEA contributes four components. First, a pluggable, model-agnostic arena. Any red or blue project connects over a minimal HTTP protocol, which we call the ACEA Standard Adapter Protocol (ASAP). It can be written in any language, and a project that exposes nothing but the protocol is a full participant. Second, an evaluation methodology built for adversarial rounds. Seeding the target with canonical secrets gives verifiable ground truth that separates real leakage from hallucination. We also send each attack to the target even when the defense blocks it, which measures the attack's raw potency independently of whether it was stopped. Together these yield a per-round decomposition of attack strength and defense effectiveness. Third, a real-time, game-style visualization with a detailed end-of-battle report that localizes each failure. The evaluation thus becomes an actionable signal for improving a red or blue project. Fourth, an optional in-context improvement loop that turns each round's outcome into advisory hints for the next. An adapter can then adapt across rounds without keeping state, provided it reads the hints. We describe the design of ACEA and the metrics through which red and blue teams are scored head to head.