Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models

📄 arXiv: 2610.01821v1 📥 PDF

作者: Tido Specht, Elias Benedict Krey, Nils Neukirch, Nils Strodthoff

分类: cs.LG, cs.CL

发布日期: 2026-10-01

备注: 24 pages, 13 figures. Code: https://anonymous.4open.science/r/NLMCD-NLP-C5E7

💡 一句话要点

提出非线性概念发现与对齐方法以提升大语言模型理解能力

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱六：视频提取与匹配 (Video Extraction) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大语言模型 非线性概念 几何对齐 多维概念发现 机械可解释性 跨语言共享 模型对齐

📋 核心要点

  1. 现有的机械可解释性方法在提取概念时受到强线性假设的限制，无法有效捕捉非线性特征流形。
  2. 本文提出了一种新的方法，通过将非线性多维概念发现技术应用于LLM的激活，建模概念为低维流形。
  3. 实验结果表明，CBA评分在敏感性和对齐结构方面优于传统的PCA和CKA方法，揭示了多层次的概念对齐特征。

📝 摘要（中文）

理解大语言模型（LLMs）中的信息处理需要剖析其内部标记表示的几何组织。现有的机械可解释性方法受限于强线性假设，而证据表明存在非线性特征流形。本文通过将非线性多维概念发现（NLMCD）方法适应于LLM的标记级激活，超越线性概念，将概念建模为低维流形。我们引入了一种概念基础对齐（CBA）评分，作为一种广义Rand指数，测量几何接近性而不需要显式特征匹配。分析结果显示了六个关键发现，包括CBA在邻层检查中比线性基线更敏感等。

🔬 方法详解

问题定义：本文旨在解决现有方法在大语言模型中对概念提取的线性假设限制，无法有效捕捉非线性特征流形的问题。

核心思路：通过将非线性多维概念发现（NLMCD）方法引入到LLM的标记级激活中，模型概念为低维流形，从而超越传统的线性概念提取。

技术框架：整体架构包括概念发现模块、对齐评分计算模块和多层次分析模块，逐层比较不同模型的概念流形。

关键创新：引入概念基础对齐（CBA）评分，作为一种新的几何接近性度量，能够在没有显式特征匹配的情况下进行概念对齐，显著提高了对概念流形的理解。

关键设计：在实验中，CBA评分的计算基于几何结构，采用了改进的损失函数和参数设置，以确保对不同层次和模型的概念进行有效对齐。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，CBA评分在邻层检查中比传统的PCA和CKA方法更敏感，揭示了中间层和后期层的块结构。此外，跨模型对齐显示同家族模型之间的强对应关系，而不同家族模型之间的对齐较弱，提供了新的见解。

🎯 应用场景

该研究的潜在应用领域包括自然语言处理、机器翻译和多语言模型的开发。通过更好地理解大语言模型中的概念结构，研究者可以优化模型训练，提高模型在复杂任务中的表现，推动人工智能的进一步发展。

📄 摘要（原文）

Understanding information processing in large language models (LLMs) requires dissecting the geometric organization of their internal token representations. While existing mechanistic interpretability (MI) methods seek to extract concepts, they are constrained by a strong linearity assumption challenged by evidence of non-linear feature manifolds. We move beyond linear concepts by adapting Non-Linear Multi-Dimensional Concept Discovery (NLMCD) from computer vision to token-level LLM activations, modeling concepts as low-dimensional manifolds. To compare concept manifolds across layers and models, we introduce a concept-based alignment (CBA) score, a generalized Rand index that measures geometric proximity without explicit feature matching. Our analysis yields six key findings: (i) a neighboring-layer sanity check shows CBA is more sensitive than PCA- or CKA-based linear baselines; (ii) layer-by-layer alignment matrices reveal two block structures in intermediate and late layers, consistent across models and obscured by linear metrics; (iii) concept composition remains syntax-dominated through most of the network before giving way to increasingly mixed syntactic-semantic concepts in later layers, with increasing output-orientation toward the final layers; (iv) multilingual concept sharing between English and Mandarin is training-dependent rather than universal, strongest in Qwen, weaker in Llama, and absent in GPT-2; (v) inter-model alignment mirrors this structure, with strong correspondence between same-family Qwen models of different scale but weak alignment across model families; and (vi) across Tulu-3 training stages, alignment is highest between adjacent stages, with the largest shift between the base model and SFT, while subsequent preference-alignment stages (DPO, RLVR) leave early layers largely unchanged and RLVR mostly preserves DPO's concepts in late layers.