隨著全球健康意識的提升，病人的就診體驗被視為醫療保健品質的重要指標 (Oben, 2020)。在這樣的背景下，臨床決策支援系統 (Clinical Decision Support System, CDSS) 是提升病人醫療品質的一種有效手段，CDSS能夠利用數據分析和知識管理來輔助醫護人員進行臨床決策 (Lu et al., 2021)，設計良好的CDSS可以支援整個護理過程中的各種管理和醫療決策來整合臨床工作流程 (Piri, 2020)。近年來，人工智慧 (Artificial Intelligence, AI)的迅速發展為CDSS的優化提供了新契機，在諸多的AI技術中由OpenAI開發的聊天生成預訓練轉換器 (Chat Generative Pre-trained Transformer, ChatGPT)，成為了醫療領域應用的熱門工具 (Alzubaidi et al., 2023)。
為改良與優化目前CDSS在臨床應用中的不足，本研究旨在結合CDSS與ChatGPT來設計一個新的智慧型CDSS系統來輔助醫師在診療過程中進行決策，透過語言模型 (Language Model, LM) 訓練時所使用的提示模版工程 (Prompt Engineering) 將ChatGPT與CDSS有效整合。本研究將病人於就診前量測的生理參數、就診時醫師登打的病症以及病人就診科別的相關診斷作為參數，傳入設計之智慧型CDSS，再依照提示模板工程將三項參數組合並透過API將參數輸入給ChatGPT以取得建議診斷。研究結果顯示本研究設計之智慧型CDSS在臨床的實驗中可以達到71%的正確率 (Accuracy)、69%的精準率 (Precision)、100%的召回率 (Recall) 以及0.84的F1-Score，這表示本研究設計的智慧型CDSS可以在一定程度上提供醫師有價值的診斷建議。

Healthcare institutions seek innovative methods to enhance patient safety and care quality in order to improve the efficiency of healthcare services (Punnakitikashem & Hallinger, 2019). CDSS (Clinical Decision Support System), which integrates medical knowledge with clinical data, is a common tool that provides healthcare professionals with real-time recommendations during the diagnostic process. Recently, ChatGPT plays a role in various industries to offer real-time responses to customer inquiries and providing accurate solutions, thereby enhancing customer experience and reducing operational costs (Chui et al., 2022). It is possible to integrate CDSS with ChatGPT to construct an intelligent CDSS to improve the effectiveness of CSS.
The purpose of this research is to demonstrate the feasibility of a framework of an intelligent CDSS which leverage the power of ChatGPT through clinical experiments. Prompt engineering is applied during the training of the Language Model (LM) and the physiological parameters measured before consultation, the symptoms recorded by the physician during the consultation, and the relevant diagnoses related to the patient's medical department are used as input parameters. These input parameters are then used by prompt engineering and plugged into ChatGPT via API to obtain diagnostic suggestions. Based on clinical experiments, approved by IRB, the results indicate that the proposed intelligent CDSS designed achieved accuracy of 71%, precision of 69%, recall of 100%, and F1-Score of 0.84, which demonstrate the feasibility of applying ChatGPT with CDSS to provide physicians with valuable diagnostic recommendations.

Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A. S., Al-dabbagh, B. S. N., ... & Gu, Y. (2023). A survey on deep learning tools dealing with data scarcity: definitions, challenges, solutions, tips, and applications. Journal of Big Data, 10(1), 46.
Ahsani-Estahbanati, E., Sergeevich Gordeev, V., & Doshmangir, L. (2022). Interventions to reduce the incidence of medical error and its financial burden in health care systems: A systematic review of systematic reviews. Frontiers in Medicine, 9, 875426.
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S., Karthikesalingam, A., King, D., ... & Darzi, A. (2021). Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ digital medicine, 4(1), 65.
Biswas, S., Dobaria, D., & Cohen, H. L. (2023). Focus: Big Data: ChatGPT and the Future of Journal Reviews: A Feasibility Study. The Yale Journal of Biology and Medicine, 96(3), 415.
Brünn, R., Lemke, D., Basten, J., Kellermann-Mühlhoff, P., Köberlein-Neu, J., Muth, C., ... & AdAM Study Group. (2022). Use of an Electronic Medication Management Support System in Patients with Polypharmacy in General Practice: A Quantitative Process Evaluation of the AdAM Trial. Pharmaceuticals, 15(6), 759.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
Chinchor, N. (1992). MUC-4 Evaluation Metrics. Proceedings of the 4th Conference on Message Understanding (pp. 22-29).
Caruccio, L., Cirillo, S., Polese, G., Solimando, G., Sundaramurthy, S., & Tortora, G. (2024). Can ChatGPT provide intelligent diagnoses? A comparative study between predictive models and ChatGPT to define a new medical diagnostic bot. Expert Systems with Applications, 235, 121186.
Choi, D. J., Park, J. J., Ali, T., & Lee, S. (2020). Artificial intelligence for the diagnosis of heart failure. NPJ digital medicine, 3(1), 54.
Chui, M., Roberts, R., & Yee, L. (2022). Generative AI is here: How tools like ChatGPT could change your business. Quantum Black AI by McKinsey.
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative adversarial networks: An overview. IEEE signal processing magazine, 35(1), 53-65.
Cao, Y., Zhang, H., & Liu, Q. (2023). Multilingual Prompt Engineering: Adapting AI Systems for Cultural and Linguistic Diversity. Journal of Artificial Intelligence Research, 67(2), 145-163.
Davis, J., & Goadrich, M. (2006, June). The relationship between Precision-Recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning (pp. 233-240).
Eyo, E. U., Abbey, S. J., Lawrence, T. T., & Tetteh, F. K. (2022). Improved prediction of clay soil expansion using machine learning algorithms and meta-heuristic dichotomous ensemble classifiers. Geoscience Frontiers, 13(1), 101296.
Gusak, J., Cherniuk, D., Shilova, A., Katrutsa, A., Bershatsky, D., Zhao, X., ... & Beaumont, O. (2022). Survey on large scale neural network training. arXiv preprint arXiv:2202.10435.
Goldstein, J. E., Guo, X., Swenor, B. K., Boland, M. V., & Smith, K. (2022). Using Electronic Clinical Decision Support to Examine Vision Rehabilitation Referrals and Practice Guidelines in Ophthalmology. Translational Vision Science & Technology, 11(10), 8-8.
Gao, H., Lu, S., & Kou, X. (2022). Research on the identification of medical service quality factors: based on a data-driven method. Internet Research, 32(5), 1617-1645.
He, J., Baxter, S. L., Xu, J., Xu, J., Zhou, X., & Zhang, K. (2019). The practical implementation of artificial intelligence technologies in medicine. Nature medicine, 25(1), 30-36.
Hartley, K., Hayak, M., & Ko, U. H. (2024). Artificial Intelligence Supporting Independent Student Learning: An Evaluative Case Study of ChatGPT and Learning to Code. Education Sciences, 14(2), 120.
Hyder, Z., Siau, K., & Nah, F. (2019). Artificial intelligence, machine learning, and autonomous technologies in mining industry. Journal of Database Management (JDM), 30(2), 67-79.
Ji, M., Genchev, G. Z., Huang, H., Xu, T., Lu, H., & Yu, G. (2021). Evaluation Framework for Successful Artificial Intelligence–Enabled Clinical Decision Support Systems: Mixed Methods Study. Journal of Medical Internet Research, 23(6), e25929.
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., ... & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS digital health, 2(2), e0000198.
Kaplan, A., & Haenlein, M. (2019). Siri, Siri, in my hand: Who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Business horizons, 62(1), 15-25.
Lee, P., Bubeck, S., & Petro, J. (2023). Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. New England Journal of Medicine, 388(13), 1233-1239.
Lu, Y., Chen, S., Miao, Z., Delen, D., & Gin, A. (2021). Clustering temporal disease networks to assist clinical decision support systems in visual analytics of comorbidity progression. Decision Support Systems, 148, 113583.
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9), 1-35.
McCallum, S. (2023). ChatGPT banned in Italy over privacy concerns. BBC News.
Morita, P. P., Abhari, S., Kaur, J., Lotto, M., Miranda, P. A. D. S. E. S., & Oetomo, A. (2023). Applying ChatGPT in public health: a SWOT and PESTLE analysis. Frontiers in Public Health, 11, 1225861.
Moulaei, K., & Bahaadinbeigy, K. (2022). Diagnosing, Managing, and Controlling COVID-19 using Clinical Decision Support Systems: A Study to Introduce CDSS Applications. Journal of Biomedical Physics & Engineering, 12(2), 213.
Mlakar, I., Smrke, U., Flis, V., Bergauer, A., Kobilica, N., Kampič, T., ... & Plohl, N. (2022). A randomized controlled trial for evaluating the impact of integrating a computerized clinical decision support system and a socially assistive humanoid robot into grand rounds during pre/post-operative care. Digital Health, 8, 20552076221129068.
Oben, P. (2020). Understanding the patient experience: a conceptual framework. Journal of patient experience, 7(6), 906-910.
Ozalp, H., Ozcan, P., Dinckol, D., Zachariadis, M., & Gawer, A. (2022). “Digital colonization” of highly regulated industries: an analysis of big tech platforms’ entry into health care and education. California management review, 64(4), 78-107.
Powers, D. M. W. (2011). Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. Journal of Machine Learning Technologies, 2(1), 37-63.
Piri, S. (2020). Missing care: A framework to address the issue of frequent missing values; The case of a clinical decision support system for Parkinson's disease. Decision Support Systems, 136, 113339.
Pereira, J., Antunes, N., Rosa, J., Ferreira, J. C., Mogo, S., & Pereira, M. (2023). Intelligent clinical decision support system for managing COPD patients. Journal of Personalized Medicine, 13(9), 1359.
Punnakitikashem, P., & Hallinger, P. (2019). Bibliometric review of the knowledge base on healthcare management for sustainability, 1994–2018. Sustainability, 12(1), 205.
Papadopoulos, P., Soflano, M., Chaudy, Y., Adejo, W., & Connolly, T. M. (2022). A systematic review of technologies and standards used in the development of rule-based clinical decision support systems. Health and Technology, 12(4), 713-727.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021, July). Learning transferable visual models from natural language supervision. In International conference on machine learning (pp. 8748-8763). PMLR.
Reynolds, L., & McDonell, K. (2021). Prompt Programming for Large Language Models: An Empirical Study. arXiv preprint arXiv:2102.07350.
Robert, L., Quindroit, P., Henry, H., Perez, M., Rousselière, C., Lemaitre, M., ... & Beuscart, J. B. (2023). Implementation of a clinical decision support system for the optimization of antidiabetic drug orders by pharmacists. British Journal of Clinical Pharmacology.
Sokolova, M., & Lapalme, G. (2009). A systematic analysis of performance measures for classification tasks. Information processing & management, 45(4), 427-437.
Samineni, S., Patel, H., & Rao, A. (2023). Machine Learning Algorithms for Cardiovascular Risk Prediction: A Comparative Study. IEEE Access, 11, 12345-12358.
Shi, Y., Shang, M., & Qi, Z. (2023). Intelligent layout generation based on deep generative models: A comprehensive survey. Information Fusion, 101940.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention Is All You Need.(Nips), 2017. arXiv preprint arXiv:1706.03762, 10, S0140525X16001837.
Wu, J., Gan, W., Chen, Z., Wan, S., & Lin, H. (2023). Ai-generated content (aigc): A survey. arXiv preprint arXiv:2304.06632.
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., ... & Yang, M. H. (2022). Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys.
Zhou, X., Li, Y., & Wang, Z. (2022). The Impact of Prompt Design on Large Language Models: A Comprehensive Study. arXiv preprint arXiv:2203.09755.
Zhai, Y., & Zhang, Y. (2023). How Nurses Develop Standardized Care Plans Under a Clinical Decision Support System: A Mixed-Methods Study. CIN: Computers, Informatics, Nursing, 10-1097.