Curriculum Learning Teaches LLMs Long-Horizon Implicit Reasoning, Up To A Limit

📄 arXiv: 2610.25612v1 📥 PDF

作者: Jimson Huang, Maxwell Nickell, Ruowang Zhang, Abulhair Saparov

分类: cs.LG, cs.CL

发布日期: 2026-10-19

备注: 31 pages, 13 figures

💡 一句话要点

通过课程学习提升LLM的长远隐式推理能力

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 隐式推理 课程学习 自然语言处理 推理能力提升

📋 核心要点

  1. 现有的LLMs在长远任务上表现不佳，尤其是在隐式推理深度上受限，无法有效利用其推理能力。
  2. 本文提出通过课程学习来逐步增加训练样本的复杂性，以提升模型的隐式推理能力和长远推理深度。
  3. 实验结果显示，课程训练的模型在前瞻步数上取得了显著提升，Qwen3-0.6B在256步前瞻的准确率达到98%，而未经过课程训练的模型仅为45%。

📝 摘要（中文）

大型语言模型（LLMs）通过将中间步骤外化为链式思维（CoT）中的标记进行推理，但在长远任务上表现出明显的性能下降。隐式推理在CoT生成过程中非常有用，因为模型可以“向前搜索”以决定正确的下一步，而无需显式搜索。本文探讨了课程学习是否能够通过逐步增加词汇量和前瞻步数来扩展隐式推理的深度。实验结果表明，经过课程训练的模型在前瞻步数上显著优于未经过课程训练的模型，且在自然语言推理任务中表现出色。

🔬 方法详解

问题定义：本文旨在解决大型语言模型在长远任务中隐式推理能力不足的问题，现有方法在深度推理时性能急剧下降。

核心思路：通过课程学习的方式，逐步增加训练样本的词汇量和前瞻步数，以此提升模型的隐式推理能力。这样的设计使模型能够在更复杂的任务中有效推理。

技术框架：整体流程包括：首先，设计课程学习的训练策略，其次，逐步增加样本的复杂性，最后，通过评估模型在不同前瞻步数下的表现来验证效果。

关键创新：最重要的创新在于通过课程学习显著提高了模型的隐式推理深度，尤其是在长远任务中，模型能够直接识别正确的推理策略。

关键设计：在训练过程中，设置了合适的课程步长，避免过大的步长导致训练停滞，同时采用了适当的损失函数以优化模型的推理能力。

🖼️ 关键图片

fig_0
img_1
img_2

📊 实验亮点

实验结果显示，经过课程训练的Qwen3-0.6B模型在前瞻步数为256的任务中达到了98%的准确率，而未经过课程训练的模型仅为45%。此外，课程训练的模型在11个自然语言推理基准测试中也表现优于未经过课程训练的模型。

🎯 应用场景

该研究的潜在应用场景包括自然语言处理、智能问答系统和复杂决策支持系统等领域。通过提升LLM的隐式推理能力，可以更好地处理复杂的推理任务，进而提高系统的智能水平和用户体验。

📄 摘要（原文）

LLMs "reason" by externalizing intermediate steps as tokens through chain-of-thought (CoT), yet even frontier reasoning models hit abrupt performance cliffs on long-horizon tasks. Implicit reasoning (reasoning within a single forward pass) is useful during CoT generation, as the model can "search ahead" to decide the correct next move without explicit search. Models with weak implicit reasoning are more likely to commit to wrong paths and must attempt to recover via backtracking, while models with strong implicit reasoning identify the correct proof strategy directly, shortening the CoT. However, prior work shows LLMs struggle to reason implicitly beyond shallow depths, with no relief from scale. We investigate whether curriculum learning can extend this depth by fine-tuning on examples with progressively increasing vocabulary size and lookahead (the number of steps a model must search ahead to identify the correct next move). At matched compute, curriculum-trained models achieve lookaheads where no-curriculum training fails. Qwen3-0.6B reaches 98% accuracy on problems with lookahead up to 256 with curriculum, compared to 45% without. However, progress slows as fine-tuning continues, following a saturating exponential that suggests a maximum intrinsic lookahead to which models can learn to reason implicitly. Larger models reach higher fitted ceilings, fine-tuning pretrained models far outpaces training from scratch, and curriculum step sizes that are too large stall progression. Finally, the capability acquired on synthetic data transfers to natural language reasoning, with curriculum-trained models outperforming models trained without curriculum at matched compute on 11 natural language reasoning benchmarks.