藥物不良反應是指病人使用某種藥物之後產生有害且不可預期或過度的反應，不同於藥物副作用輕微且可預期，藥物不良反應一般都對病人的治療不利，輕度影響可能造成不適但可自行恢復，重度影響則會造成永久性傷害，甚至危及生命。除了對病人造成的傷害之外，世界衛生組織發表的藥物治療培訓資料中也提到，藥物不良反應造成的醫療成本極高，在全世界一直是個值得重視的議題。因此本研究致力提出一個藥物不良反應的判別模型，透過病程紀錄資料，辨別病人於住院過程中是否發生藥物不良反應，期望為醫療產業及藥物不良反應等相關領域盡一份心力。
本研究擷取由SOAP格式撰寫的臨床電子病歷進行實驗，前處理階段將資料集透過醫療字典組合進行過濾，藉此刪除較無意義的單詞，並進行特徵擷取以提高模型的準確度。透過支援向量資料描述法的模型，使模型分類是否為藥物不良反應，因資料時間長度不一，故再結合強化學習以DQN框架搭配LSTM，使模型得以盡可能的早期預測該病人是否有藥物不良的反應。全時間段的實驗中，本研究的方法準確度可達92%，以時間段區分的準確度也有75%以上，成效優於其他機器學習的演算法。結果呈現中，除預測出是否有藥物不良反應外，本研究提供專家一份藥物不良反應比例評估表，呈現每位病人藥物不良反應以及沒有藥物不良反應的機率，使專家在用藥評估上更有依據，並可以藉此評估表更有效的對應患者進行診治。

Adverse Drug Reaction (ADR) refers to the harmful, serious, and unintended results caused by taking medicines. Different from "side effect" that is predictable, ADR is generally deleterious to patients' treatment. Mild ADR effects can result in recoverable discomfort, while severe effects may cause permanent injury or even dangerous to life.
Therefore, our study aims at proposing a discriminant model for ADRs that can identify through progress notes whether a patient has ADR during hospitalization
We applied SVDD model to classify ADR and combined it with the Reinforcement Learning framework using DQN and LSTM to deal with the varying data length. This enables the model to predict as early as possible whether a patient has ADR.
In the experiments of the entire period, the accuracy of our method is 92%, and the accuracy of distinguishing by period is more than 75%, which is better than other algorithms.
In addition to predicting whether a patient will have adverse drug reactions, our study also provides an assessment table that shows the probability of having ADR for each patient, so that experts can have more basis for drug evaluation.

Beijer, H. J. M., & de Blaey, C. J. (2002). Hospitalisations caused by adverse drug reactions (ADR): a meta-analysis of observational studies. Pharmacy World and Science, 24(2), 46-54. doi:10.1023/A:1015570104121
Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157-166. doi:10.1109/72.279181
Blei, D. M., & Lafferty, J. D. (2006). Dynamic topic models. International conference on Machine learning, Association for Computing Machinery, New York, US, 113–120. doi:10.1145/1143844.1143859.
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022.
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297. doi:10.1007/BF00994018
El-allaly, E., Sarrouti, M., En-Nahnahi, N., & Alaoui, S. O. E. (2020). A LSTM-Based Method with Attention Mechanism for Adverse Drug Reaction Sentences Detection. Advances in Intelligent Systems and Computing, Springer, Cham, 17-26. doi:10.1007/978-3-030-36664-3_3.
Gao, X. (2018). Deep reinforcement learning for time series: playing idealized trading games. arXiv preprint arXiv:1803.03916.
Guo, Y., Barnes, S. J., & Jia, Q. (2017). Mining meaning from online ratings and reviews: Tourist satisfaction analysis using latent dirichlet allocation. Tourism management, 59, 467-483. doi:10.1016/j.tourman.2016.09.009
Hajjar, E. R., Cafiero, A. C., & Hanlon, J. T. (2007). Polypharmacy in elderly patients. The American journal of geriatric pharmacotherapy, 5(4), 345-351. doi:10.1016/j.amjopharm.2007.12.002
Heinrich, G. (2005). Parameter estimation for text analysis. Technical report
Hitchen, L. (2006). Adverse drug reactions result in 250 000 UK admissions a year. BMJ: British Medical Journal, 332, 1109. doi:10.1136/bmj.332.7550.1109
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780. doi:10.1162/neco.1997.9.8.1735
Hoffman, M., Bach, F., & Blei, D. (2010). Online learning for latent dirichlet allocation. Advances in neural information processing systems, 23.
Huynh, T., He, Y., Willis, A., & Rüger, S. (2016). Adverse drug reaction classification with deep neural networks. COLING, International Committee on Computational Linguistics, Osaka ,Japan, 877-887.
Jeon, E., Kim, Y., Park, H., Park, R. W., Shin, H., & Park, H.-A. (2020). Analysis of adverse drug reactions identified in nursing notes using reinforcement learning. Healthcare Informatics Research, 26(2), 104-111. doi:10.4258/hir.2020.26.2.104
Jia, S., Zhang, X., Wang, X., & Liu, Y. (2018). Fake reviews detection based on LDA. International Conference on Information Management (ICIM), IEEE, 280-283. doi:10.1109/INFOMAN.2018.8392850.
Jiang, Z., Xu, D., & Liang, J. (2017). A deep reinforcement learning framework for the financial portfolio management problem. arXiv preprint arXiv:1706.10059.
Jo, Y., Lee, L., & Palaskar, S. (2017). Combining LSTM and latent topic modeling for mortality prediction. arXiv preprint arXiv:1709.02842.
Kamaruddin, S., & Ravi, V. (2016). Credit card fraud detection using big data analytics: use of PSOAANN based one-class classification. International Conference on Informatics and Analytics, Association for Computing Machinery, New York, US, 1-8. doi:10.1145/2980258.2980319.
Khan, S. S., & Madden, M. G. (2009). A survey of recent trends in one class classification. Irish conference on artificial intelligence and cognitive science, Springer, Berlin, 188-197. doi:10.1007/978-3-642-17080-5_21.
Kraemer, H. C. (2014). Kappa coefficient. Wiley StatsRef: statistics reference online, 1-4. doi:10.1002/9781118445112.stat00365.pub2
Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.
Li, Y., Ni, P., & Chang, V. (2020). Application of deep reinforcement learning in stock trading strategies and stock forecasting. Computing, 102(6), 1305-1322. doi:10.1007/s00607-019-00773-w
Lin, L. J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3), 293-321. doi:10.1007/BF00992699
Lin, Y. W., Zhou, Y., Faghri, F., Shaw, M. J., & Campbell, R. H. (2019). Analysis and prediction of unplanned intensive care unit readmission using recurrent neural networks with long short-term memory. PloS one, 14(7), e0218942. doi:10.1371/journal.pone.0218942
Martinez, C., Perrin, G., Ramasso, E., & Rombaut, M. (2018). A deep reinforcement learning approach for early classification of time series. European Signal Processing Conference (EUSIPCO), IEEE, 2030-2034. doi:10.23919/EUSIPCO.2018.8553544.
Mikolov, T. (2012). Statistical language models based on neural networks. Presentation at
Google, Mountain View, 2nd April, 80, 26.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie , C., Sadik , A., Antonoglou, I., King, H., Kumaran, D., Wierstra , D., Legg, S., & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533. doi:10.1038/nature14236
Ruff, L., Zemlyanskiy, Y., Vandermeulen, R., Schnake, T., & Kloft, M. (2019). Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, ACL, Florence, Italy, 4061-4071. doi:10.18653/v1/P19-1398.
Rumshisky, A., Ghassemi, M., Naumann, T., Szolovits, P., Castro, V. M., McCoy, T. H., & Perlis, R. H. (2016). Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. Translational Psychiatry, 6(10), e921-e921. doi:10.1038/tp.2015.182
Schölkopf, B., Williamson, R. C., Smola, A., Shawe-Taylor, J., & Platt, J. (1999). Support vector method for novelty detection. Advances in neural information processing systems, 12(3), 582-588.
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7), 1443-1471. doi:10.1162/089976601750264965
Shepherd, G., Mohorn, P., Yacoub, K., & May, D. W. (2012). Adverse drug reaction deaths reported in United States vital statistics, 1999-2006. Annals of Pharmacotherapy, 46(2), 169-175. doi:10.1345/aph.1P592
Sultana, J., Cutroneo, P., & Trifirò, G. (2013). Clinical and economic burden of adverse drug reactions. Journal of pharmacology & pharmacotherapeutics, 4(Suppl 1), S73-S77. doi:10.4103/0976-500X.120957
Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. IEEE Transactions on Machine learning, 3, 9-44. doi:10.1007/BF00115009
Tax, D. M. J., & Duin, R. P. W. (2004). Support vector data description. Machine learning, 54(1), 45-66. doi:10.1023/B:MACH.0000008084.60811.49
Wang, Q., Lopes, L. S., & Tax, D. M. J. (2004). Visual object recognition through one-class learning. International Conference Image Analysis and Recognition, Springer, Berlin, Heidelberg, 463-470. doi:10.1007/978-3-540-30125-7_58.
Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine learning, 8(3), 279-292. doi:10.1007/BF00992698
Xin, J., Zhao, H., Liu, D., & Li, M. (2017). Application of deep reinforcement learning in mobile robot path planning. Chinese Automation Congress (CAC), IEEE, Jinan, China, 7112-7116. doi:10.1109/CAC.2017.8244061.
Xing, Z., Pei, J., & Philip, S. Y. (2009). Early prediction on time series: A nearest neighbor approach. International Joint Conference on Artificial Intelligence, Citeseer, Pasadena, US, 1297-1302.
Yang, C. C., Yang, H., Jiang, L., & Zhang, M. (2012). Social media mining for drug safety signal detection. International workshop on smart health and wellbeing, Association for Computing Machinery, New York, US, 33-40. doi:10.1145/2389707.2389714.
Zhang, M., & Geng, G. (2019). Adverse drug event detection using a weakly supervised convolutional neural network and recurrent neural network model. Information, 10(9), 276. doi:10.3390/info10090276