Self-Adaptive Anomaly Detection with Reinforcement Learning and Human Feedback in Connected Vehicles
作者: Matthias Weiß, Athreya Hosahalli Prakash, Maurice Artelt, Falk Dettinger, Nasser Jazdi, Michael Weyrich
分类: cs.LG, cs.AI
发布日期: 2026-07-09
备注: Accepted at the 30th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA 2026), Special Session SS10: Evaluation Methods for Autonomous Cyber-Physical Systems' Behavior. 8 pages, 3 figures, 3 tables
💡 一句话要点
提出自适应异常检测框架以解决联网车辆监控问题
🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)
关键词: 异常检测 深度学习 人机反馈 联网车辆 自动驾驶 统计漂移 强化学习
📋 核心要点
- 现有异常检测方法无法有效应对联网车辆系统的动态变化，导致监控性能下降。
- 论文提出的框架集成了深度Q网络、自适应漂移检测和人机反馈机制，实现了异常检测的自适应性。
- 在实验中，该框架的F1分数达到0.69，显著高于单一检测器的0.11，展示了其优越的性能。
📝 摘要（中文）
联网车辆是自主的网络物理系统，其行为在运行过程中必须持续监控，以便在故障传播之前检测到正常操作的偏差。现有方法通常单独处理自动模型适应或操作员集成，未能形成协调的监督循环。本文提出了一种在线异常检测框架，结合了深度Q网络、自适应统计漂移检测和人机反馈机制，能够有效应对系统动态变化带来的挑战。实验结果表明，该框架在自动代客泊车应用中表现出色，能够在软件更新后保持适应性。
🔬 方法详解
问题定义：本文旨在解决联网车辆在运行过程中异常检测的挑战，现有方法无法适应系统动态变化，导致监控效果逐渐下降。
核心思路：提出的框架通过整合深度Q网络与人机反馈机制，动态选择最合适的异常检测器，并结合统计漂移检测，确保高精度的异常识别。
技术框架：框架主要包括三个模块：1) 基于自注意力机制的深度Q网络，用于选择检测器；2) 三个统计漂移检测器，监控输入分布；3) 人机反馈机制，允许操作员在系统学习的基础上进行再训练。
关键创新：最重要的创新在于将自动模型适应与人机反馈结合，形成一个协调的监督循环，克服了传统方法的局限性。
关键设计：使用了60/40的优先重放策略和待转移缓冲区，以有效整合专家知识，同时保持系统对先前数据分布的响应能力。
🖼️ 关键图片
📊 实验亮点
实验结果显示，框架在自动代客泊车应用中的F1分数达到0.69，而任何单一检测器的最高F1分数仅为0.11。在经历软件更新后，F1分数降至0.52，但经过操作员触发的再训练后，性能恢复至0.65，展示了系统的持续适应能力。
🎯 应用场景
该研究的潜在应用领域包括自动驾驶、智能交通系统和车联网等场景。通过提高异常检测的准确性和适应性，可以有效减少故障发生的风险，提升车辆的安全性和可靠性，具有重要的实际价值和社会影响。
📄 摘要（原文）
Connected vehicles are autonomous cyber-physical systems whose behavior must be continuously monitored during operation to detect deviations from normal operation before they propagate into failures. Such evaluation is challenging because the systems themselves evolve: over-the-air updates, configuration changes, and shifting workloads alter the definition of normal behavior, causing static diagnostic methods to degrade silently over time. Existing approaches typically address either automated model adaptation or operator integration in isolation, rather than as a single coordinated supervisory loop. This paper presents an online anomaly detection framework for autonomous CPS that integrates three coordinated mechanisms. A factorized deep Q-network with self-attention selects the most suitable detector from a candidate pool for each monitored service, exploiting inter-service dependencies in the microservice topology. An ensemble of three statistical drift detectors monitors the input distribution and raises an alarm only when all three concur, prioritizing precision over recall. A human-in-the-loop retraining mechanism, built around a pending transition buffer and a 60/40 prioritized replay strategy, allows the operator to incorporate expert knowledge while preserving the system's learned response to prior data distributions. The framework is evaluated on a connected-vehicle testbed running an automated valet parking application across seven backend microservices. The attention-augmented agent achieves an F1 score of 0.69, compared to at most 0.11 for any single detector applied uniformly. Following a real software update that induces measurable concept drift, F1 drops to 0.52; after operator-triggered retraining, performance recovers to 0.65 on the new distribution while remaining at 0.69 on the prior one, demonstrating sustained adaptation without catastrophic forgetting.