Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models

📄 arXiv: 2310.01074v2 📥 PDF

作者: Chenhan Yuan, Qianqian Xie, Jimin Huang, Sophia Ananiadou

分类: cs.CL, cs.AI

发布日期: 2023-10-02 (更新: 2023-10-08)

备注: 14 pages, 5 figures, code and dataset: https://github.com/chenhan97/TimeLlama

💡 一句话要点

提出可解释的时间推理任务以解决复杂事件预测问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 时间推理 可解释性 事件预测 大型语言模型 知识图谱 自然语言处理 多步骤推理

📋 核心要点

  1. 现有方法主要集中在简单的时间表达和关系提取，缺乏对复杂事件预测的研究。
  2. 本文提出可解释的时间推理任务，要求模型在多事件推理的基础上预测未来事件，并提供推理解释。
  3. 实验表明，所提方法在时间预测和解释方面超越了多种大型语言模型，达到了最先进的性能。

📝 摘要（中文）

时间推理是自然语言处理中的重要任务，能够深入理解文本数据中的时间敏感上下文。尽管大型语言模型在时间推理方面取得了一定进展，但现有研究主要集中在时间表达和时间关系提取等简单任务上，复杂的事件预测任务仍存在显著不足。本文首次提出可解释的时间推理任务，旨在基于上下文预测事件在未来时间戳的发生，并提供清晰的推理解释。为支持该任务，作者构建了第一个多源指令调优数据集ExpTime，并提出了开源的时间推理模型TimeLlaMA，实验结果显示该方法在时间预测和解释方面达到了最先进的性能。

🔬 方法详解

问题定义：本文旨在解决复杂事件预测中的可解释时间推理问题。现有方法在处理多步骤推理和未来时间戳预测时存在局限，无法提供推理过程的解释。

核心思路：提出可解释的时间推理任务，要求模型在多事件推理的基础上进行未来事件的预测，并提供清晰的推理解释，以增强模型的可解释性。

技术框架：整体架构包括数据集构建、模型设计和评估三个主要模块。首先，构建ExpTime数据集，接着基于LlaMA2设计TimeLlaMA模型，最后通过多种基线模型进行性能评估。

关键创新：首次引入可解释的时间推理任务，强调多步骤推理和未来事件预测的结合，显著提升了模型的推理能力和可解释性。

关键设计：在模型设计中，采用知识图谱指导生成策略，设置了特定的损失函数以优化时间推理的准确性，确保模型能够有效地进行指令跟随。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，所提出的TimeLlaMA模型在时间预测和解释任务上达到了最先进的性能，相较于多种基线模型，性能提升幅度显著，具体数据表明在准确率和可解释性方面均有明显改善。

🎯 应用场景

该研究在多个领域具有潜在应用价值，包括智能助手、事件预测系统和自动化决策支持等。通过提高模型的可解释性，用户能够更好地理解模型的推理过程，从而增强对AI系统的信任和接受度。未来，该方法可能推动更复杂的时间推理任务的发展，促进自然语言处理技术的进步。

📄 摘要（原文）

Temporal reasoning is a crucial NLP task, providing a nuanced understanding of time-sensitive contexts within textual data. Although recent advancements in LLMs have demonstrated their potential in temporal reasoning, the predominant focus has been on tasks such as temporal expression and temporal relation extraction. These tasks are primarily designed for the extraction of direct and past temporal cues and to engage in simple reasoning processes. A significant gap remains when considering complex reasoning tasks such as event forecasting, which requires multi-step temporal reasoning on events and prediction on the future timestamp. Another notable limitation of existing methods is their incapability to provide an illustration of their reasoning process, hindering explainability. In this paper, we introduce the first task of explainable temporal reasoning, to predict an event's occurrence at a future timestamp based on context which requires multiple reasoning over multiple events, and subsequently provide a clear explanation for their prediction. Our task offers a comprehensive evaluation of both the LLMs' complex temporal reasoning ability, the future event prediction ability, and explainability-a critical attribute for AI applications. To support this task, we present the first multi-source instruction-tuning dataset of explainable temporal reasoning (ExpTime) with 26k derived from the temporal knowledge graph datasets and their temporal reasoning paths, using a novel knowledge-graph-instructed-generation strategy. Based on the dataset, we propose the first open-source LLM series TimeLlaMA based on the foundation LlaMA2, with the ability of instruction following for explainable temporal reasoning. We compare the performance of our method and a variety of LLMs, where our method achieves the state-of-the-art performance of temporal prediction and explanation.