RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

📄 arXiv: 2610.08539v1 📥 PDF

作者: Dongchen Si, Di Wang, Mingzhen Xu, Jing Zhang, Bo Du, Liangpei Zhang

分类: cs.CV

发布日期: 2026-10-06

🔗 代码/项目: GITHUB

💡 一句话要点

提出RSJEV以解决遥感场景分类中的多模态决策问题

🎯 匹配领域: 支柱二：RL算法与架构 (RL & Architecture) 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 遥感分类 多模态决策 视觉-语言交互 深度学习 高效推理

📋 核心要点

  1. 现有遥感场景分类方法存在依赖预定义标签空间和静态图像-文本对齐的不足，限制了分类性能。
  2. RSJEV通过将场景分类重新表述为候选条件的多模态判别决策过程，提出了一种高效的分类框架。
  3. 在UC Merced、AID和NWPU-RESISC45等基准测试中，RSJEV在分类性能上显著优于多种现有方法，并降低了推理成本。

📝 摘要（中文）

遥感场景分类是地球观测和地理空间分析中的基础任务。现有方法主要遵循三种范式：任务特定的视觉分类、视觉-语言相似性匹配和自回归多模态生成。然而，视觉分类器依赖于预定义标签空间，CLIP方法通过静态图像-文本对齐进行识别，而多模态大语言模型（MLLMs）在分类任务中引入了不必要的令牌级生成。为了解决这些局限性，我们提出了RSJEV，这是一种一遍式多模态决策框架，重新将场景分类表述为候选条件的多模态判别决策过程。通过引入OnePass Decider，RSJEV提取多模态决策状态并直接估计类别概率，消除了自回归解码，同时保留了视觉-语言交互。实验结果表明，RSJEV在多个基准测试中表现优异，且显著降低推理成本。

🔬 方法详解

问题定义：本论文旨在解决遥感场景分类中的多模态决策问题。现有方法如视觉分类器和CLIP方法存在依赖于静态标签和图像-文本对齐的局限性，导致分类性能受限。

核心思路：RSJEV的核心思路是将场景分类重新表述为候选条件的多模态判别决策过程，避免了传统自回归文本生成的复杂性，同时实现了视觉与语言的有效交互。

技术框架：RSJEV的整体架构包括OnePass Decider模块，该模块提取多模态决策状态，并在候选类别空间内直接估计类别概率，简化了决策过程。

关键创新：RSJEV的主要创新在于其一遍式决策框架，消除了自回归解码的需求，直接在候选类别上进行判别，显著提高了分类效率和准确性。

关键设计：在设计上，RSJEV采用了紧凑的0.8B参数模型，优化了推理成本，并通过多模态状态建模来增强视觉与语言的交互效果。损失函数和网络结构的具体细节在实验中进行了优化，以确保最佳性能。

🖼️ 关键图片

fig_0
fig_1

📊 实验亮点

RSJEV在UC Merced、AID和NWPU-RESISC45等基准测试中表现出色，分类性能显著优于CNN、Transformer、CLIP和其他多模态大语言模型方法，且推理成本大幅降低，展示了更优的准确性与效率平衡。

🎯 应用场景

该研究在遥感图像分析、地理空间数据处理和环境监测等领域具有广泛的应用潜力。通过提高遥感场景分类的效率和准确性，RSJEV可为智能城市、农业监测和灾害管理等实际应用提供强有力的支持，推动相关领域的发展。

📄 摘要（原文）

Remote sensing scene classification is a fundamental task in Earth observation and geospatial analysis. Existing approaches mainly follow three paradigms: task-specific visual classification, vision-language similarity matching, and autoregressive multimodal generation. However, visual classifiers rely on predefined label spaces, CLIP-based methods perform recognition through static image-text alignment, and multimodal large language models (MLLMs) introduce unnecessary token-level generation for classification tasks with explicit candidate categories. To address these limitations, we propose RSJEV, a one-pass multimodal decision framework for remote sensing scene classification. Unlike conventional MLLMs that formulate classification as autoregressive text generation, RSJEV reformulates scene classification as a candidate-conditioned multimodal discriminative decision process, where visual representations, task instructions, and candidate category semantics are jointly modeled. Specifically, we introduce a OnePass Decider that extracts multimodal decision states and directly estimates category probabilities within the candidate category space, eliminating autoregressive decoding while preserving vision-language interactions. Extensive experiments on three widely used remote sensing scene classification benchmarks, including UC Merced, AID, and NWPU-RESISC45, demonstrate that RSJEV achieves superior classification performance compared with representative CNN-, Transformer-, Mamba-, CLIP-, and MLLM-based methods. Moreover, RSJEV significantly reduces inference costs and achieves a better accuracy-efficiency trade-off with only a compact 0.8B-parameter model. These results demonstrate the effectiveness of state-conditioned multimodal decision making for efficient remote sensing image understanding. The code will be available at https://github.com/Dongtcs/RSJEV.