Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

📄 arXiv: 2610.02353v1 📥 PDF

作者: Songyuan Sui, Srikanth Malla, Chiho Choi, Joon Hee Choi

分类: cs.LG, cs.CL

发布日期: 2026-10-01

备注: 10 pages main content, 36 pages total including appendix, 7 figures

💡 一句话要点

提出LINEUP以解决个性化大语言模型的用户适应性问题

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 个性化模型 低秩因子 用户适应 共享能力 机器学习

📋 核心要点

  1. 现有个性化大语言模型需要为每个用户适应大量参数，导致扩展性差。
  2. LINEUP通过学习可重用的低秩个性化因子，减少了每个用户的训练参数需求。
  3. 在六个任务中，LINEUP在12个指标上均表现优异，相比基线减少了11.4%的RMSE。

📝 摘要（中文）

个性化的大语言模型通常需要为每个用户提供完整的适应状态，但这种方法在用户数量增加时扩展性较差。本文通过个性化能力分配的视角重新审视这一设计，探讨用户间共享适应能力的程度及其组成方式。研究发现，独立用户适配器中存在大量可跨用户重用的结构，且可重用方向的效用与用户相关性及查询变化有关。基于这些发现，提出了LINEUP方法，通过学习可重用的低秩个性化因子，结合用户条件回忆和查询依赖校准，限制目标用户适应于共享校正空间中的少量用户代码。实验表明，LINEUP在多个任务上均优于现有方法，展示了丰富的个性化能力主要依赖于可重用的共享能力。

🔬 方法详解

问题定义：本文旨在解决个性化大语言模型在用户数量增加时的适应性问题。现有方法需要为每个用户提供完整的适应状态，导致参数量庞大且扩展性差。

核心思路：LINEUP的核心思路是通过共享可重用的低秩个性化因子，减少每个用户所需的适应参数。通过用户条件回忆和查询依赖校准，限制用户适应于共享校正空间中的少量用户代码。

技术框架：LINEUP的整体架构包括三个主要模块：可重用低秩因子的学习、用户条件回忆机制和查询依赖校准。用户仅需优化少量参数，而共享组件保持固定。

关键创新：LINEUP的关键创新在于将个性化能力与每个用户的可训练状态解耦，使得用户只需优化八个标量参数，而不再需要庞大的用户特定参数。

关键设计：在技术细节上，LINEUP采用了低秩矩阵分解技术，设计了特定的损失函数以优化共享因子的学习，同时确保用户历史信息能够有效传递信号以进行个体校正。

🖼️ 关键图片

img_0
img_1
img_2

📊 实验亮点

LINEUP在六个任务中均表现优异，12个指标的平均结果显示，相比最强基线，RMSE减少了11.4%。即使在历史信息有限的情况下，LINEUP依然保持了其优势，展示了其在个性化任务中的强大能力。

🎯 应用场景

该研究的潜在应用领域包括个性化推荐系统、智能客服和个性化教育等。通过减少每个用户所需的参数量，LINEUP能够在用户数量大幅增加的情况下，依然保持良好的个性化效果，具有重要的实际价值和未来影响。

📄 摘要（原文）

Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how the shared capacity should be composed, and how much must remain user-specific. We answer them through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.