GPT-4 as an Agronomist Assistant? Answering Agriculture Exams Using Large Language Models

📄 arXiv: 2310.06225v2 📥 PDF

作者: Bruno Silva, Leonardo Nunes, Roberto Estevão, Vijay Aski, Ranveer Chandra

分类: cs.AI, cs.LG

发布日期: 2023-10-10 (更新: 2023-10-12)

💡 一句话要点

利用GPT-4提升农业考试答题能力

🎯 匹配领域: 支柱九：具身大模型 (Embodied Foundation Models)

关键词: 大型语言模型 农业教育 信息检索 生成模型 RAG ER技术 农业管理 考试评估

📋 核心要点

  1. 现有的农业教育和评估方法在准确性和效率上存在不足，传统考试难以全面评估学生的实际能力。
  2. 论文提出通过结合RAG和ER技术，提升大型语言模型在农业领域的答题能力，利用信息检索和生成能力的结合。
  3. 实验结果显示，GPT-4在农业考试中取得93%的正确率，超越了其他模型，并在某些实验中表现优于人类考生。

📝 摘要（中文）

大型语言模型（LLMs）在自然语言理解方面展现出卓越的能力，尤其在医疗和金融等领域。本文对流行的LLMs，如Llama 2和GPT，进行全面评估，重点考察其回答农业相关问题的能力。通过结合信息检索、生成能力和提示策略的RAG和ER技术，GPT-4在农业考试中取得93%的正确率，超越了早期模型的88%准确率，显示出其在农业教育和管理实践中的潜在贡献。

🔬 方法详解

问题定义：本文旨在评估大型语言模型在农业考试中的表现，现有方法未能有效利用LLMs的潜力，导致农业教育评估的局限性。

核心思路：通过引入RAG和ER技术，结合信息检索与生成能力，提升LLMs在农业相关问题上的回答准确性和效率。

技术框架：整体架构包括数据收集、模型训练、信息检索模块、生成模块和评估模块，确保模型能够从多个来源获取信息并生成准确答案。

关键创新：本研究的创新在于将RAG和ER技术结合应用于农业领域，显著提升了模型的答题能力，与传统方法相比，提供了更高的准确性和实用性。

关键设计：在模型训练中，采用了特定的损失函数和参数设置，以优化信息检索和生成的协同效果，确保模型能够有效处理农业领域的复杂问题。

🖼️ 关键图片

fig_0
fig_1
fig_2

📊 实验亮点

实验结果显示，GPT-4在农业考试中取得93%的正确率，显著高于早期模型的88%准确率，并在某些实验中超越了人类考生，表明其在农业教育和评估中的巨大潜力。

🎯 应用场景

该研究的潜在应用领域包括农业教育、农作物管理和农业政策制定。通过提供准确的农业知识和管理建议，GPT-4可以帮助农民和农业专业人士做出更明智的决策，提升农业生产效率和可持续性。

📄 摘要（原文）

Large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding across various domains, including healthcare and finance. For some tasks, LLMs achieve similar or better performance than trained human beings, therefore it is reasonable to employ human exams (e.g., certification tests) to assess the performance of LLMs. We present a comprehensive evaluation of popular LLMs, such as Llama 2 and GPT, on their ability to answer agriculture-related questions. In our evaluation, we also employ RAG (Retrieval-Augmented Generation) and ER (Ensemble Refinement) techniques, which combine information retrieval, generation capabilities, and prompting strategies to improve the LLMs' performance. To demonstrate the capabilities of LLMs, we selected agriculture exams and benchmark datasets from three of the largest agriculture producer countries: Brazil, India, and the USA. Our analysis highlights GPT-4's ability to achieve a passing score on exams to earn credits for renewing agronomist certifications, answering 93% of the questions correctly and outperforming earlier general-purpose models, which achieved 88% accuracy. On one of our experiments, GPT-4 obtained the highest performance when compared to human subjects. This performance suggests that GPT-4 could potentially pass on major graduate education admission tests or even earn credits for renewing agronomy certificates. We also explore the models' capacity to address general agriculture-related questions and generate crop management guidelines for Brazilian and Indian farmers, utilizing robust datasets from the Brazilian Agency of Agriculture (Embrapa) and graduate program exams from India. The results suggest that GPT-4, ER, and RAG can contribute meaningfully to agricultural education, assessment, and crop management practice, offering valuable insights to farmers and agricultural professionals.