TensorRL-QAS: Reinforcement learning with tensor networks for improved quantum architecture search

📄 arXiv: 2505.09371v2 📥 PDF

作者: Akash Kundu, Stefano Mangini

分类: quant-ph, cs.AI, cs.ET, cs.LG

发布日期: 2025-05-14 (更新: 2025-09-30)

备注: Accepted at NeurIPS 2025. Code is at: https://github.com/Aqasch/TensorRL-QAS

💡 一句话要点

提出TensorRL-QAS以解决量子电路设计的可扩展性问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture)

关键词: 量子电路设计 强化学习 张量网络 量子架构搜索 量子化学 可扩展性 优化算法

📋 核心要点

  1. 现有的RL基础量子架构搜索方法在处理量子比特数量、电路深度和硬件噪声时面临显著的可扩展性问题。
  2. TensorRL-QAS通过结合张量网络方法与强化学习，利用矩阵乘积态的近似来缩小搜索空间，从而加速电路设计过程。
  3. 在多项量子化学问题上，TensorRL-QAS实现了CNOT门数量和电路深度的显著减少，同时提高了成功率，表现出优越的鲁棒性。

📝 摘要（中文）

变分量子算法有望在噪声中间规模量子硬件上解决重要的量子问题。然而，设计既能解决目标问题又符合设备限制的量子电路面临挑战。量子架构搜索（QAS）自动化电路设计过程，强化学习（RL）成为一种有前景的方法，但现有RL基础的QAS方法在可扩展性上存在显著问题。为此，本文提出了TensorRL-QAS框架，将张量网络方法与RL结合，通过用矩阵乘积态近似目标解来热启动QAS，有效缩小搜索空间，加速收敛。实验结果表明，TensorRL-QAS在多个量子化学问题上实现了CNOT门数量和电路深度的最高10倍减少，同时保持或超越化学精度。它还将经典优化器函数评估减少至100倍，加速训练过程达98%。

🔬 方法详解

问题定义：本文旨在解决量子电路设计中的可扩展性问题，现有的RL基础QAS方法在处理大规模量子系统时计算和训练成本迅速增加，导致效率低下。

核心思路：TensorRL-QAS的核心思路是将张量网络方法与强化学习相结合，通过热启动过程使用矩阵乘积态近似目标解，从而有效缩小搜索空间，加速收敛。

技术框架：TensorRL-QAS框架包括两个主要模块：首先是张量网络模块，用于生成目标解的近似；其次是强化学习模块，负责在缩小的搜索空间内优化电路设计。

关键创新：TensorRL-QAS的关键创新在于通过张量网络的近似来引导强化学习过程，显著提高了搜索效率和成功率，这与传统的RL方法形成了鲜明对比。

关键设计：在设计中，TensorRL-QAS采用了特定的损失函数来评估电路的性能，并优化了网络结构以适应量子电路的特性，确保在不同噪声条件下的鲁棒性。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

TensorRL-QAS在多个量子化学问题上表现出色，CNOT门数量和电路深度最高减少10倍，成功率在10量子比特系统中达到50%，远超基线的1%以下。此外，经典优化器函数评估减少至100倍，训练过程加速98%。

🎯 应用场景

TensorRL-QAS的研究成果具有广泛的应用潜力，尤其是在量子化学、量子计算和量子通信等领域。通过提高量子电路设计的效率和准确性，该框架能够推动量子技术的实际应用，促进量子硬件的开发与优化。

📄 摘要（原文）

Variational quantum algorithms hold the promise to address meaningful quantum problems already on noisy intermediate-scale quantum hardware. In spite of the promise, they face the challenge of designing quantum circuits that both solve the target problem and comply with device limitations. Quantum architecture search (QAS) automates the design process of quantum circuits, with reinforcement learning (RL) emerging as a promising approach. Yet, RL-based QAS methods encounter significant scalability issues, as computational and training costs grow rapidly with the number of qubits, circuit depth, and hardware noise. To address these challenges, we introduce $\textit{TensorRL-QAS}$, an improved framework that combines tensor network methods with RL for QAS. By warm-starting the QAS with a matrix product state approximation of the target solution, TensorRL-QAS effectively narrows the search space to physically meaningful circuits and accelerates the convergence to the desired solution. Tested on several quantum chemistry problems of up to 12-qubit, TensorRL-QAS achieves up to a 10-fold reduction in CNOT count and circuit depth compared to baseline methods, while maintaining or surpassing chemical accuracy. It reduces classical optimizer function evaluation by up to 100-fold, accelerates training episodes by up to 98$\%$, and can achieve 50$\%$ success probability for 10-qubit systems, far exceeding the $<$1$\%$ rates of baseline. Robustness and versatility are demonstrated both in the noiseless and noisy scenarios, where we report a simulation of an 8-qubit system. Furthermore, TensorRL-QAS demonstrates effectiveness on systems on 20-qubit quantum systems, positioning it as a state-of-the-art quantum circuit discovery framework for near-term hardware and beyond.