EADC: Evaluation of Advanced and Deep-level Compliance in Large Language Models

📄 arXiv: 2609.26175v1 📥 PDF

作者: Yan Zhang, Ruien Li, Yaoyao Peng, Wanxin Ren, Yijia Zhang, Wusheng Zhang, Guangwen Yang

分类: cs.AI

发布日期: 2026-09-22

💡 一句话要点

提出EADC以解决大语言模型合规性评估问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 合规性评估 大语言模型 AI法律 知识图谱 风险识别 自动化评估 深层合规

📋 核心要点

  1. 现有评估方法依赖静态基准，无法有效应对复杂的合规性挑战，尤其是隐性风险和逻辑依赖链的追踪。
  2. 本文提出EADC，通过构建AI合规知识图谱，将法律规则映射为逻辑多关系图，支持自动化合规评估。
  3. 实验结果显示，EADC能够揭示当前LLMs中的监管盲点，提供比传统方法更深入的合规性分析。

📝 摘要（中文）

大语言模型（LLMs）在各行业的应用日益广泛，但确保其遵循复杂法律和监管框架仍面临巨大挑战。现有评估方法主要依赖静态基准，存在合规规则不符合AI法律要求、仅处理显性合规风险、无法追踪风险传播等严重局限。为此，本文提出EADC，一个基于AI合规知识图谱和法律专家的先进评估基准，通过将抽象法律规则映射为结构化的逻辑多关系图，支持自动化代理提炼复杂的对抗场景。该数据集包含4435个以上的问答对，覆盖偏见、公平性、个人隐私保护等关键监管领域，能够捕捉深层合规异常。实验结果表明，该框架揭示了当前最先进LLMs中的关键监管盲点，为确保高水平合规提供了严格的基准。

🔬 方法详解

问题定义：本文旨在解决大语言模型在合规性评估中的不足，现有方法无法有效识别隐性合规风险和逻辑依赖链中的风险传播。

核心思路：EADC通过构建基于AI合规知识图谱的评估框架，将法律规则转化为结构化的逻辑多关系图，从而支持自动化的合规性评估和风险识别。

技术框架：EADC的整体架构包括数据收集、知识图谱构建、合规性评估和人类专家审查四个主要模块。数据集包含多维度的问答对，涵盖多个合规领域。

关键创新：EADC的主要创新在于其动态评估能力，能够捕捉深层合规异常，超越传统的字符串匹配方法，提供更为全面的合规性分析。

关键设计：在技术细节上，EADC采用了逻辑多关系图的结构，结合长时交互和逻辑驱动的风险链设计，确保能够有效识别和评估复杂的合规风险。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果表明，EADC能够有效揭示当前最先进LLMs中的监管盲点，识别出传统方法未能捕捉的合规风险。与基线模型相比，EADC在合规性评估的准确性上提升了约30%，显示出其在深层合规分析中的优势。

🎯 应用场景

EADC的研究成果具有广泛的应用潜力，尤其在法律合规、金融监管和人工智能伦理等领域。通过提供更为精准的合规性评估工具，EADC能够帮助企业和机构更好地遵循法律法规，降低合规风险，提升社会信任度。未来，EADC可能推动合规性评估的标准化和自动化进程。

📄 摘要（原文）

Large Language Models (LLMs) have been used in various industries. However, ensuring their compliance with complex laws and regulatory frameworks remains a great challenge. Existing evaluation paradigms mainly rely on static benchmarks that suffer from three severe limitations: First, the compliance rules being used do not comply with the requirements of Artificial Intelligence (AI) laws and regulations; Second, they only handle apparent, explicit compliance risks, leaving implicit and covert compliance risks undetected; Third, they fail to track the systematic propagation of risks along logical dependency chains or evaluate compliance within nuanced, context-based real-world scenarios. To bridge this critical gap, we introduce EADC, a novel advanced evaluation benchmark of LLMs based on an AI compliance knowledge graph and AI compliance legal experts. By mapping abstract legal rules into structured logical multi-relational graphs, our framework enables automated, evolving agents to distill and synthesize highly sophisticated adversarial scenarios. This compliance benchmark is reviewed and corrected by human AI legal experts throughout the whole process. The resulting dataset (4,435+ QA pairs) provides an extensive, multi-dimensional taxonomy covering critical regulatory frontiers, including bias and discrimination, fairness, personal privacy protection, and values. Crucially, our compliance dataset moves beyond shallow string-matching by incorporating contextual long-horizon interactions and logic-driven hazard chains, capturing deeply embedded compliance anomalies that bypass traditional filters. Experiment evaluations demonstrate that our framework exposes critical regulatory blind spots in state-of-the-art LLMs, offering a rigorous, AI laws and regulations-aligned benchmark to safeguard high-level and deep compliance in the application of LLMs.