Beyond Binary Labels: Diagnostic Evaluation of LLM Safety with Prompt-Level Rubrics

📄 arXiv: 2610.29225v1 📥 PDF

作者: Hongyu Li, Xinlong Huang, Zhan Yu, Jing Zhao, Zhe Zhao, Juntao Zhong, Xiong Kuang, Rui Yuan, ZhenXiang Yan, Huilin Xu, Haitao Yu, Lilin Wang, Rui Chen, Blaze Chen, Fengzong Lian, Renfen Hu

分类: cs.CL, cs.AI, cs.CR

发布日期: 2026-10-21

备注: 42 pages, 9 figures, 24 tables. Dataset will be publicly released at https://github.com/Tencent-Hunyuan/DiagSafe

💡 一句话要点

提出DiagSafe框架以解决LLM安全评估的不足问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 安全评估 提示级评估 双语基准 脆弱性分析

📋 核心要点

  1. 现有的LLM安全评估方法主要依赖二元标签，缺乏深度解释，且一致性较低。
  2. 本文提出DiagSafe框架，通过提示级评估标准，提供明确的安全性和有效性评估。
  3. 实验表明，DiagSafe在与人工标注的一致性上显著提升，达到95.34%，并且在不同模型间揭示了独特的脆弱性。

📝 摘要（中文）

当前的大型语言模型（LLM）安全基准主要依赖于二元的安全或不安全标签，缺乏解释深度且在自动评估者之间一致性较低。为此，本文提出了DiagSafe，一个超越二元分类的诊断评估框架，采用了基于提示的评估标准。每个评估标准在安全性和有效性两个维度上提供明确的标准，确保评估的可靠性和现实性。通过构建一个双语基准，实验结果显示DiagSafe显著提高了LLM作为评估者的可靠性，达到了95.34%的与人工标注一致性，远高于基线的69%-87%。

🔬 方法详解

问题定义：当前LLM安全评估主要依赖二元标签，导致评估缺乏深度和一致性，无法有效识别模型的潜在风险和脆弱性。

核心思路：DiagSafe框架通过引入提示级评估标准，超越简单的二元分类，提供更细致的安全性和有效性评估，旨在提高评估的可靠性和可解释性。

技术框架：DiagSafe的整体架构包括两个主要模块：DiagBase和DiagAttack。DiagBase处理直接有害查询和模板化越狱的单轮和多轮提示，而DiagAttack则将有害意图嵌入复杂的指令跟随任务中，形成挑战性的指令攻击。

关键创新：DiagSafe的核心创新在于其基于提示的评估标准，能够在安全性和有效性两个维度上提供明确的评估标准，显著提高了评估的一致性和可靠性。

关键设计：在设计中，使用了84标签的安全分类法，确保评估的全面性，同时采用双语基准（英语-中文）以增强评估的适用性和现实性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，DiagSafe在与人工标注的一致性上达到95.34%，显著高于基线的69%-87%。此外，Cohen's κ值为0.864，表明评估者之间的一致性极高。该框架还揭示了不同模型在安全性方面的显著差异及其独特脆弱性。

🎯 应用场景

DiagSafe框架可广泛应用于大型语言模型的安全性审计和优化，帮助开发者识别和修复模型中的潜在风险。其可解释性和可靠性使其在AI安全政策的制定和实施中具有重要价值，未来可能推动更安全的AI系统的开发与应用。

📄 摘要（原文）

Current LLM safety benchmarks predominantly rely on binary safe or unsafe labels, lacking explanatory depth and suffering from low consistency across automated judges. To address this, we introduce DiagSafe, a diagnostic evaluation framework that moves beyond binary classification by employing prompt-level rubrics. Grounded in globally applicable AI safety policies, each rubric provides explicit criteria along two dimensions: safety, which targets concrete harms with specific exemptions to suppress false positives, and usefulness, which independently evaluates instruction fulfillment. To ensure a robust and realistic evaluation, we construct a bilingual (English-Chinese) benchmark grounded in a comprehensive 84-label safety taxonomy, consisting of two subsets: DiagBase, which covers single- and multi-turn prompts with direct harmful queries and template-based jailbreaks, and DiagAttack, which embeds harmful intents into complex instruction-following tasks to form challenging instruction attacks. Extensive experiments demonstrate that DiagSafe significantly enhances the reliability of LLM-as-a-judge evaluation, achieving 95.34% agreement with human annotations -- substantially higher than the 69%-87% achieved by the baselines -- and high inter-judge consistency (Cohen's $κ$ = 0.864). Furthermore, applying our framework reveals that leading LLMs differ markedly in their unsafe rates, and each model also exhibits unique vulnerabilities along different dimensions. In addition, we observe that models are particularly susceptible in Chinese and multi-turn contexts. Ultimately, DiagSafe provides a reliable and explainable tool for auditing and optimizing LLM alignment.