Autonomous Tree-search Ability of Large Language Models

📄 arXiv: 2310.10686v1 📥 PDF

作者: Zheyu Zhang, Zhuorui Ye, Yikang Shen, Chuang Gan

分类: cs.CL, cs.AI

发布日期: 2023-10-14

备注: Due to the limitation "The abstract field cannot be longer than 1,920 characters", the abstract here is shorter than that in the PDF file

💡 一句话要点

提出自主树搜索能力以提升大语言模型的推理效率

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 自主树搜索 大语言模型 推理能力 API调用 微调 性能提升 智能决策

📋 核心要点

  1. 现有方法依赖外部程序进行被动树搜索，导致效率低下且缺乏灵活性，无法适应多样化任务。
  2. 本文提出自主树搜索能力（ATS），使大型语言模型能够在无外部程序支持下自动生成搜索轨迹，提升推理能力。
  3. 实验结果表明，ATS-BFS方法在准确性上平均提升33%，且在API调用成本上显著低于现有方法，展现出更高的效率。

📝 摘要（中文）

大型语言模型在推理能力方面表现出色，但在探索、战略前瞻和顺序决策任务上存在不足。现有方法依赖外部程序进行被动树搜索，效率低且灵活性差。本文提出自主树搜索能力（ATS），使得LLM能够在无外部程序的情况下自动生成包含搜索轨迹的响应。通过在四个谜题游戏上的实验，ATS-BFS方法在准确性上平均提升了33%，并且在成本上显著低于现有方法。进一步的微调实验表明，基于ATS提示的方法在性能上优于基于链式思维（CoT）微调的LLaMA模型。

🔬 方法详解

问题定义：本文旨在解决大型语言模型在推理任务中缺乏自主探索能力的问题。现有的被动树搜索方法效率低下，且需要针对特定任务设计程序，限制了其灵活性。

核心思路：提出自主树搜索能力（ATS），通过固定的系统提示使LLM能够直接进行树搜索，自动生成包含搜索轨迹的响应，从而提高推理效率。

技术框架：整体架构包括LLM API的调用和固定系统提示的设计。通过这种设计，模型能够在无外部程序的情况下，直接进行自主树搜索。

关键创新：最重要的创新在于自主树搜索能力的提出，使得LLM能够在不依赖外部程序的情况下，灵活应对多样化的推理任务，显著提升了推理效率。

关键设计：在参数设置上，使用了固定的系统提示来引导模型进行搜索，确保生成的响应中包含清晰的搜索轨迹。同时，微调过程中使用了基于ATS提示的数据集，以进一步提升模型性能。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，ATS-BFS方法在准确性上平均提升33%，并且在API调用成本上相比于现有的链式思维方法减少了65.6%至47.7%。此外，基于ATS提示微调的LLaMA模型在性能上优于基于链式思维微调的模型，分别提升了40.6%和38.5%。

🎯 应用场景

该研究的潜在应用领域包括智能问答系统、游戏AI、自动化决策支持等。通过提升大型语言模型的推理能力，能够更好地应对复杂任务，具有广泛的实际价值和未来影响。

📄 摘要（原文）

Large Language Models have excelled in remarkable reasoning capabilities with advanced prompting techniques, but they fall short on tasks that require exploration, strategic foresight, and sequential decision-making. Recent works propose to utilize external programs to define search logic, such that LLMs can perform passive tree search to solve more challenging reasoning tasks. Though impressive results have been achieved, there are several fundamental limitations of these approaches. First, passive tree searches are not efficient as they usually require multiple rounds of LLM API calls to solve one single problem. Moreover, passive search methods are not flexible since they need task-specific program designs. Then a natural question arises: can we maintain the tree-search capability of LLMs without the aid of external programs, and can still generate responses that clearly demonstrate the process of a tree-structure search? To this end, we propose a new concept called autonomous tree-search ability of LLM, which can automatically generate a response containing search trajectories for the correct answer. Concretely, we perform search trajectories using capable LLM API via a fixed system prompt, allowing them to perform autonomous tree-search (ATS) right out of the box. Experiments on 4 puzzle games demonstrate our method can achieve huge improvements. The ATS-BFS method outperforms the Chain of Thought approach by achieving an average accuracy improvement of 33%. Compared to Tree of Thoughts, it requires 65.6% or 47.7% less GPT-api cost to attain a comparable level of accuracy. Moreover, we have collected data using the ATS prompt method and fine-tuned LLaMA. This approach yield a greater improvement compared to the ones fine-tuned on CoT data. Specifically, it outperforms CoT-tuned LLaMAs by an average of 40.6% and 38.5% for LLaMA2-7B and LLaMA2-13B, respectively.