Breaking the Tie: A Cluster-Aware Routing Framework for Large Language Models

📄 arXiv: 2610.05982v1 📥 PDF

作者: Yao Lu, Zhaiyuan Ji, Yaxin Gao, Zeyu Wang, Zhe Tang, Jiaheng Wei, Zhaowei Zhu, Shanqing Yu, Qi Xuan

分类: cs.CL

发布日期: 2026-10-05

备注: 12 pages, 8 figures

💡 一句话要点

提出Cluster-Aware Soft-Labeling Routing框架以解决LLM路由噪声问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 路由框架 聚类意识 soft-labeling 模型选择 人工智能 机器学习

📋 核心要点

  1. 现有LLM路由方法简化为标准分类任务，导致在多个候选模型正确回答同一查询时出现路由噪声，影响模型的泛化能力。
  2. 本文提出的CASLR框架通过引入掩蔽softmax机制，替代传统的one-hot向量，提升了路由的准确性和效率。
  3. CASLR在多个基准测试中表现优异，整体平均性能比Llama-3.3-70B-Instruct提升了7.80%，且路由推理延迟仅为1.13秒。

📝 摘要（中文）

随着人工智能的快速发展，各种大型语言模型（LLMs）的出现形成了丰富的模型生态。然而，如何为特定用户查询选择最佳模型成为一大挑战。现有的LLM路由框架往往将此过程简化为标准分类任务，导致在多个候选模型正确回答同一查询时出现路由噪声，最终导致路由崩溃。为了解决这一问题，本文提出了一种新颖的Cluster-Aware Soft-Labeling Routing（CASLR）框架，通过引入掩蔽softmax机制替代传统的one-hot向量，提升了模型的准确性和响应质量。实验结果表明，CASLR在多个基准测试中表现优异，整体平均性能比Llama-3.3-70B-Instruct提升了7.80%。

🔬 方法详解

问题定义：本文旨在解决现有LLM路由框架在多个候选模型正确回答同一查询时产生的路由噪声问题，这种噪声会导致路由崩溃，影响模型的泛化能力。

核心思路：CASLR框架通过引入掩蔽softmax机制，替代传统的one-hot向量，转变评估范式，从单一查询的成功转向宏观领域共识，提升了路由的准确性。

技术框架：CASLR框架主要包括两个模块：一是对错误回答的专家进行惩罚，将其目标概率设为零；二是根据全球聚类效用分数计算剩余候选模型的细粒度soft标签，进而监督轻量级路由器。

关键创新：CASLR的核心创新在于引入了掩蔽softmax机制和细粒度soft标签，这一设计与传统的one-hot向量方法本质上不同，能够有效减少路由噪声。

关键设计：在损失函数设计上，CASLR对错误回答的专家进行惩罚，确保其对路由决策没有影响，同时通过聚类效用分数计算soft标签，提升了模型的整体性能。具体参数设置和网络结构细节在实验部分进行了详细描述。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

CASLR框架在多个基准测试中表现优异，整体平均性能比Llama-3.3-70B-Instruct提升了7.80%。此外，其路由推理延迟仅为1.13秒，显示出几乎零额外开销的高效系统调度能力。

🎯 应用场景

该研究的潜在应用领域包括智能客服、个性化推荐系统以及多模态交互等场景。通过提高LLM的路由效率和准确性，CASLR框架能够显著提升用户体验，满足不同用户的需求，具有广泛的实际价值和未来影响。

📄 摘要（原文）

With the rapid development of artificial intelligence, the emergence of various Large Language Models (LLMs) has created a rich model ecosystem. However, this also brings a key challenge: how to select the optimal model for a specific user query. LLM routing addresses this need by dynamically assigning queries to the most suitable expert in the pool of candidate models. However, existing routing frameworks often simplify this process to a standard classification task; thus, a critical vulnerability is exposed when multiple candidate models correctly answer the same query. We formalize this capability overlap as routing noise, which misleads the router with arbitrarily correct candidate models, ultimately leading to routing collapse (a severe decline in generalization ability on unseen tasks). To address this problem, we propose a novel Cluster-Aware Soft-Labeling Routing (CASLR) framework. CASLR shifts the evaluation paradigm from the success of a single query to macro-domain consensus by replacing traditional one-hot vectors with a masked softmax mechanism. Specifically, for experts who answer incorrectly, we penalize their target probability to zero; for the remaining candidates, we directly compute continuous fine-grained soft labels based on their global clustering utility scores. We then use these refined soft labels to supervise a lightweight router. Specifically, the framework not only demonstrates superior accuracy on multiple benchmarks, but also outperforms Llama-3.3-70B-Instruct by 7.80% in overall average performance. Furthermore, the extremely low routing inference latency of only 1.13s further confirms that CASLR can achieve efficient system scheduling with almost zero additional overhead, while ensuring high response quality.