中药药理与临床

目的:探索基于人工智能的多模态大模型舌诊，推动智能化医疗在中医舌诊中的临床研究，以期为中医舌诊的标准化、智能化提供新的思路。方法:于2023年3月-2024年3月，随机选取就诊于我院中医、中西医结合科门诊350例患者舌象图像，同时对舌象图像及医生对舌象的初步描述(舌苔、舌质、舌形)进行预处理、特征地提取以及模式识别，依据幽门螺旋杆菌(HP)检测报告的结果，分为训练集和测试集，其中训练集中HP(-)患者200例，HP(+)患者50例，测试集中HP(-)患者50例，HP(+)患者50例。应用阿里达摩院开源的通义千问VL模型，通过人工智能技术分析验证多模态模型在中医舌诊辨识与疾病诊断(HP感染)中的效能。结果:通过该模型有效提高了舌诊结果的客观性和准确性，训练组在决策阈值为0.5时，感染样本的召回率为94.0%,对正常样本的误检率为5.0%。测试集在决策阈值为0.5时，感染样本的召回率为86.0%,对正常样本的误检率为8.0%。结论:本研究开发并测试了一种结合舌象分析与医生初步评估的多模态大型模型，用于识别幽门螺杆菌感染，实验结果显示出该方法有良好的泛化能力和一定的诊断精确度。该模型具有快速学习的优势，能通过分析舌象特征来检测幽门螺杆菌感染，无创且高效。未来可以进一步深入探索智能化医疗在中医舌诊中的应用，以提高中医舌诊的准确性、客观性和效率。同时，未来多模态大模型有望整合更多维度的数据，如患者的脸部、耳朵、手部图像以及脉象等生物特征，连同生活习惯和环境影响等非生物因素，进一步推进中医标准化及精准医疗服务的发展。

Objective:To explore the application of artificial intelligence-based multimodal large models in tongue diagnosis, and to promote the clinical research of intelligent healthcare in traditional Chinese medicine(TCM) tongue diagnosis, aiming to provide new ideas for the standardization and intelligence of TCM tongue diagnosis. Methods:From March 2023 to March 2024,tongue images were randomly selected from 350 patients admitted to Department of Integrative Chinese and Western Medicine in this hospital. The tongue images, along with preliminary descriptions by doctors(including tongue coating, tongue color, and tongue shape),were preprocessed, features were extracted, and pattern recognition was applied. Based on Helicobacter pylori(HP) test results, the data were divided into training and testing sets. The training set included 200 HP-negative and 50 HP-positive patients, while the testing set included 50 HP-negative and 50 HP-positive patients. The Qwen-VL model, an open-source model from Alibaba DAMO Academy, was used to analyze and validate the efficacy of the multimodal model in TCM tongue diagnosis and disease diagnosis(HP infection) through AI technology. Results:The model effectively improved the objectivity and accuracy of tongue diagnosis results. In the training set, at a decision threshold of 0.5,the recall rate for infected samples was 94.0%,with a false detection rate for normal samples of 5.0%. In the testing set, at the same decision threshold, the recall rate for infected samples was 86.0%,with a false detection rate for normal samples of 8.0%. Conclusion:This study developed and tested a multimodal large model that combines tongue image analysis with preliminary doctor assessments for identifying HP infection. The experimental results show that this method has good generalization ability and certain diagnostic accuracy. The model has the advantage of rapid learning, capable of detecting HP infection through analysis of tongue image features, in a non-invasive and efficient manner. Future research can further explore the application of intelligent healthcare in TCM tongue diagnosis to enhance its accuracy, objectivity, and efficiency. Additionally, future multimodal large models may integrate data from more dimensions, such as facial, ear, and hand images, along with pulse diagnosis and non-biological factors like lifestyle habits and environmental influences, to further promote the development of TCM standardization and precision healthcare services.