DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

📄 arXiv: 2607.06326v1 📥 PDF

作者: He Liu, Changtao Miao, Xinjie Yang, Tianle Song, Yin Wu, Junchi Chen, Bintao He, Xinyuan Zhang, Bo Zhang, Shi Yan, Wei Lu, Wei Wang, Danyang Xu, Jiansheng Cai, Zhe Li

分类: cs.AI

发布日期: 2026-07-07

💡 一句话要点

提出DT-Guard以解决大语言模型安全性问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 安全防护 推理训练 低延迟推断 内容审核 风险管理 机器学习

📋 核心要点

  1. 现有安全防护措施在处理隐含意图和模糊语义时存在不足，且推理模型引入了额外的延迟。
  2. DT-Guard通过推理主动训练和推理无关推断的方式，在训练中使用推理监督，推断时生成结构化安全标签。
  3. 实验结果显示DT-Guard在安全基准测试中平均F1分数达到0.878，优于8B基线，验证了推理监督的有效性。

📝 摘要（中文）

在开放世界应用中部署的大语言模型需要既能应对复杂风险又能实现低延迟运行的安全防护措施。现有的防护措施在轻量级分类模型与基于推理的模型之间存在权衡：前者高效但难以处理隐含意图和模糊语义，后者提高判断质量但增加延迟。本文提出DT-Guard，一种基于推理主动训练和推理无关推断的内容安全防护模型。DT-Guard在训练时使用推理监督，而在推断时仅生成结构化安全标签。实验结果表明，DT-Guard在安全基准测试中表现优异，平均F1分数达到0.878，超越了强大的8B基线。

🔬 方法详解

问题定义：本文旨在解决大语言模型在开放应用中的安全性问题，现有方法在处理复杂风险时存在效率与判断质量之间的权衡。

核心思路：DT-Guard的核心思想是通过推理监督进行训练，但在推断时仅生成结构化的安全标签，从而提高效率并降低延迟。

技术框架：DT-Guard的整体架构包括三个主要阶段：意图判断、类别识别和安全标签生成。训练过程中构建意图驱动的数据集，包含意图标签、风险类别和安全标签。

关键创新：DT-Guard的关键创新在于引入了推理主动训练（Reasoning-Active Training）和推理无关推断（Reasoning-Free Inference），使得推理监督能够有效内化到低延迟的安全判断中。

关键设计：在设计中，DT-Guard采用了多回合一致性（Rollout-Guided Progressive Hard-Case Optimization）来优化难案例，并通过针对性的监督和偏好优化来提升模型的鲁棒性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

DT-Guard在安全基准测试中取得了平均F1分数0.886和0.870，使用4B模型时双侧平均F1达到0.878，显著优于强大的8B基线，展示了推理监督在低延迟安全判别中的有效性。

🎯 应用场景

DT-Guard的研究成果可广泛应用于需要高安全性的开放世界应用，如社交媒体内容审核、在线客服系统和自动化内容生成等领域。其高效的安全防护机制将为用户提供更安全的使用体验，同时降低运营成本。

📄 摘要（原文）

Large language models deployed in open-world applications require safety guardrails that are both robust to complex risks and efficient enough for low-latency runtime moderation. Existing guardrails face a practical trade-off between lightweight classification-based models, which are efficient but often struggle with concealed intent, ambiguous semantics, and borderline safety decisions, and reasoning-based guards, which improve judgment quality but introduce additional token generation and inference latency. We present DT-Guard, a content safety guardrail model based on a Reasoning-Active Training, Reasoning-Free Inference paradigm. The key idea is to use reasoning supervision during training while emitting only structured safety labels at inference time. DT-Guard formulates safety judgment as a progressive decision process, Intent - Category - Safety, and constructs an intent-driven dataset with intent labels, risk categories, safety labels, and structured reasoning trajectories. To further improve hard-case robustness, we propose Rollout-Guided Progressive Hard-Case Optimization (RG-PHO), which uses multi-rollout consistency to identify stably mastered, persistently failed, and preference-unstable samples, and applies targeted supervised and preference optimization accordingly. At inference time, DT-Guard directly generates structured labels without explicit reasoning traces, preserving deployment efficiency. Experiments on prompt-side and response-side safety benchmarks show that DT-Guard achieves average F1 scores of 0.886 and 0.870, respectively. With only a 4B backbone, it reaches a dual-side average F1 of 0.878, outperforming strong 8B guardrail baselines. These results demonstrate that reasoning supervision can be effectively internalized into low-latency safety discrimination.