| 48 | 0 | 328 |
| 下载次数 | 被引频次 | 阅读次数 |
在面向学科知识问答应用中，基础教育场景对学科准确性要求高，需要消除通用大语言模型所存在的事实性错误与“知识幻觉”。同时，普通学校或教育机构难以承受高昂的大语言模型运行成本。在消费级GPU环境下，基于自建的初中生物高质量问答数据集，采用QLoRA方法对Chat GLM3-6B量化模型进行参数高效微调，注入领域知识，优化模型。实验结果表明，该方法使模型训练过程中的GPU显存占用降低超过50%，微调后的模型在初中生物学知识问答中的事实准确性与专业性显著提升。
Abstract：Among the subject knowledge Q&A applications,basic education has high demands on disciplinary accuracy,requiring the elimination of factual errors and “knowledge hallucinations” that are prevalent in general large language models( LLMs). Furthermore,ordinary schools or educational institutions often struggle to afford the high operational costs associated with running large language models. Under the constraints of consumer-grade GPU environments,this study employs the QLoRA method to perform parameter-efficient fine-tuning on the quantized Chat GLM3-6B model based on a self-constructed high-quality Q&A dataset on junior high school biology. This process infuses domain-specific knowledge and optimizes the model. Experimental results demonstrate that the proposed approach reduces more than 50% of GPU memory consumption during the model training while significantly enhancing the factual accuracy and professionalism of the fine-tuned model in answering junior high school biology questions.
[1]刘明，吴忠明，杨箫，等．教育大语言模型的内涵?构建和挑战[J]．现代远程教育研究，2024,36(05):50~60.
[2]Ji Z,Lee N,Frieske R,et al. Survey of hallucination in natural language generation[J]. ACM Computing Surveys,2022,55:1~38.
[3]KASNECI E,SEΒLER K,KÜCHEMANN S,et al. Chat GPT for good? On opportunities and challenges of large language models for education[J]. Learning and Individual Differences,2023,103:102274. https://doi. org/10. 1016/j. lindif.2023. 102274.
[4]DETTMERS T,PAGNONI A,HOLTZMAN A,et al. QLoRA:Efficient Finetuning of Quantized LLMs[EB/OL]. ar Xiv,2023[2025-12-22]. https://arxiv. org/abs/2305. 14314.
[5]ZENG A,LIU X,DU Z,et al. Glm-130b:An open bilingual pre-trained model[C]. ICLR,2023. https://openreview. net/forum? id=-Aw0rrrPUF.
[6]DU Z,QIAN Y,LIU X,et al. GLM:General Language Model Pretraining with Autoregressive Blank Infilling[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022,1:320~335. https://aclanthology. org/2022. acl-long. 24/．
[7]ZHANG R,HAN J,ZHOU A,et al. Llama-adapter:Efficient fine-tuning of language models with zero-init attention[C]//Proceedings of the International Conference on Learning Representations. 2024. https://openreview. net/forum?id=Pqv MR9Yk0r.
[8]HU E J,SHEN Y,WALLIS P,et al. LoRA:Low-rank adaptation of large language models[C]//Proceedings of the International Conference on Learning Representations. 2022. https://openreview. net/forum? id=n Ze VKee FYf9.
[9]VASWANI A,SHAZEER N,PARMAR N,et al. Attention is All you Need[C]//Advances in Neural Information Processing Systems 30. 2017:5998~6008. https://proceedings. neurips. cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa5-Abstract. html.
[10]程乐超．视觉大模型参数高效微调技术应用与展望[J]．人工智能，2024,(01):54~65. DOI:10. 16453/j.2096-5036. 202407.
[11]刘旭凯，刘洋，黄浩桢．EC-MIIP:基于高效微调的知识产权小参数大语言模型[J/OL]．计算机科学，1~15[2025-09-12]. https://link. cnki. net/urlid/50. 1075. TP. 20250902. 1423. 017.
[12]祝欣宇，窦迅，牛鹏艺，等．基于改进Vi T模型的电网关键线路智能预测方法[J/OL]．电力自动化设备:1~15.[2025-09-12]. https://doi. org/10. 16081/j. epae. 202508025.
[13]徐小放，杨春德，舒坤贤，等．基于BERT与Text-CNN的抗菌肽识别方法[J]．生物工程学报，2023,39(04):1815~1824. DOI:10. 13345/j. cjb. 220878.
[14]LIN C Y. ROUGE:A package for automatic evaluation of summaries[C]//Proceedings of the ACL 2004 Workshop on Text Summarization Branches Out. 2004:74~81. https://aclanthology. org/W04-1013/．
[15]张淦，袁堂晓，汪惠芬，等．基于BERT和Text CNN的智能制造成熟度评估方法[J]．计算机集成制造系统，2024,30(03):852~863. DOI:10. 13196/j. cims. 2023. 0639.
[16]LIN S,HILTON J,EVANS O. Truthful QA:Measuring How Models Mimic Human Falsehoods[C]//Proceedings of the60th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers). 2022:3214~3252.https://aclanthology. org/2022. acl-long. 222/．
[17]车万翔，窦志成，冯岩松，等．大模型时代的自然语言处理:挑战、机遇与发展[J]．中国科学:信息科学，2023,53(09):1645~1687.
基本信息:
中图分类号:TP18;G633.91
引用信息:
[1]熊旭辉,万桃,覃守垣,等.面向初中生物学科问答的大模型低资源适配方法研究[J].湖北师范大学学报(自然科学版),2026,46(01):15-23.
基金信息:
省级产学研合作协同育人项目(2024IT120)
2025-09-15
2025
2025-11-19
2025
2025-10-23
1
2026-03-25
2026-03-25