From GAN-Based Models to GPT-5: A Taxonomy-Guided Study of Human Performance in Detecting Deepfake Face Images

📄 arXiv: 2610.23144v1 📥 PDF

作者: Zhiqing Guo, Lixin Jia, Luning Sun, Xiaowei Chen, Guofeng Yu, Liejun Wang, Gaobo Yang, David Stillwell, Thorsten Holz, Shujun Li

分类: cs.CV, cs.CR, cs.HC

发布日期: 2026-10-16

💡 一句话要点

提出深度伪造面孔伪影分类法以提升检测效果

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 深度伪造 检测技术 生成对抗网络 模型特定训练 人类表现评估 视觉伪影 多模态学习

📋 核心要点

  1. 现有研究未能跟上生成模型的快速演变，导致人类在检测深度伪造面孔图像方面的表现不明确。
  2. 提出深度伪造面孔伪影分类法（DFAT），构建了基于模型的特定训练材料，以提升检测效果。
  3. 实验结果显示，模型特定训练相较于通用训练在大多数情况下能显著提高检测准确性。

📝 摘要（中文）

生成模型在深度伪造技术中迅速演变，但人类在检测深度伪造面孔图像的能力尚未得到充分研究。本文提出了深度伪造面孔伪影分类法（DFAT），并通过在线实验评估了人类在不同生成模型下的检测表现。结果表明，检测准确性与生成模型密切相关，模型特定训练相较于通用训练能显著提升检测效果。五个月后跟踪测试显示，参与者的检测准确性没有显著下降，且感知压力没有显著增加。这些发现为设计有效的深度伪造检测培训方案提供了可行的见解。

🔬 方法详解

问题定义：本文旨在解决人类在检测不同生成模型生成的深度伪造面孔图像时的表现差异，现有方法未能有效评估这一问题的复杂性。

核心思路：通过构建深度伪造面孔伪影分类法（DFAT），为不同生成模型提供特定的训练材料，以提高人类的检测能力。

技术框架：研究采用在线实验的方式，参与者在不同生成模型下进行深度伪造图像的检测，评估其准确性和训练效果。实验涵盖三种基于GAN的模型、一种基于扩散的模型和一种多模态大型语言模型（GPT-5）。

关键创新：DFAT的提出为深度伪造图像的检测提供了新的视角，强调了模型特定训练的重要性，突破了传统通用训练的局限性。

关键设计：实验设计包括524名参与者的在线测试，评估了不同模型下的检测准确性和训练效果，后续五个月的跟踪测试也确保了结果的可靠性。参与者在模型特定训练后表现出显著的准确性提升。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

实验结果显示，参与者在模型特定训练后，检测准确性显著提高，尤其是在使用最新生成模型时。与通用训练相比，模型特定训练的数值提升更为显著，后续跟踪测试中准确性保持稳定，未见显著下降。

🎯 应用场景

该研究为深度伪造检测提供了新的培训方法，具有广泛的应用潜力，尤其在社交媒体、新闻传播和安全监控等领域。通过有效的培训方案，可以提升公众对深度伪造内容的识别能力，从而减少潜在的社会危害。

📄 摘要（原文）

Generative models for deepfakes have rapidly evolved from GAN-based architectures to diffusion models and multimodal large language models (MLLMs). However, empirical research on human performance in detecting deepfake face images has not kept pace. It remains unclear if (and how) detection performance varies across generative models and whether model-specific training improves detection more effectively than general-purpose training. In this paper, we propose the \emph{Deepfake Face Artifact Taxonomy} (DFAT), a cross-model taxonomy of human-recognizable visual artifacts in deepfake face images, based on which we build both general-purpose and model-specific training materials. Through an online experiment ($n = 524$), we examined human performance in detecting deepfake face images and evaluated training efficacy across five representative generative models: three GAN-based, one diffusion-based, and one MLLM (GPT-5). Our results show that participants' detection accuracy was strongly model-dependent and that images generated by more recent and advanced models were not necessarily harder to detect. After training, participants achieved significant performance gains across most conditions, and their improvement varied by model. Notably, model-specific training generally yielded greater numerical gains than general-purpose training. In a five-month follow-up test ($n = 342$), we observed no significant decline in detection accuracy and no significant increase in perceived stress. Overall, our findings provide actionable insights for designing effective, sustainable, and targeted training programs to counter the growing threat posed by deepfakes.