簡易檢索 / 詳目顯示
| 研究生： |
林欣蓉 Lin, Xin-Rong |
|---|---|
| 論文名稱： |
設計及實現硬體友善之卷積類神經網路/YOLOv4於特定心律不整及結構性心臟疾病分類/脈搏振幅變異度偵測系統 Design and Implementation of a Hardware-Friendly CNN/YOLOv4 for Specific Arrhythmia and Structural Heart Disease Classification/Pulse Amplitude Variability Prediction |
| 指導教授： |
林哲偉
Lin, Che-Wei |
| 學位類別： |
碩士 Master |
| 系所名稱： |
工學院 - 生物醫學工程學系 Department of BioMedical Engineering |
| 論文出版年： | 2021 |
| 畢業學年度： | 109 |
| 語文別： | 英文 |
| 論文頁數： | 83 |
| 中文關鍵詞： | 長期心電圖 、心律不整 、結構性心臟疾病 、脈搏短絀 、連續小波轉換 、卷積神經網絡 、硬體友善 、深度學習優化 、脈搏振幅變異度 、YOLOv4 、ARM 、STM32 、Jetson TX2 、心房顫動 |
| 外文關鍵詞： | long-term ECG, arrhythmia, convolutional neural networks, time-frequency transform, CWT, pulse amplitude variability, hardware friendly, YOLOv4, STM32, Jetson TX2, structural heart disease, atrial fibrillation |
| 相關次數： | 點閱：125 下載：1 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
為了通過實現心血管疾病發病率和死亡率的全球預防，以社區為基礎的方式有效管理和監測居民的健康，基於人工智慧的硬體實現通常被認為是最具成本效益的遠程監測和預防解決方案之一。此外，利用人工智慧建立特殊心臟疾病的快篩系統除了可以協助醫生尋找潛在罹患心臟疾病卻不自知的病人，同時可以即時性的監測罹患特定心臟疾病病人當下的健康狀態，其好處在於藉由落實分級醫療，讓醫生可以花費更多時間在治療需要更進一步詳細檢查或是迫切需要醫療資源的人。然而以既有文獻來看，自2012年起深度學習網絡的蓬勃發展並沒有因此帶動其在硬體上的實現。究其原因，可能與深度學習本身架構的層數與深度複雜度有關。因此本論文提出了一種硬體相容之卷積類神經網路的硬體實現系統，用於a) 快速篩檢特定心律不整和心臟瓣膜疾病，b)脈搏振幅變異度的評分量表作為心房顫動病人嚴重程度的評估指標。使用的嵌入式AI計算設備為STM32F746ZG Nucleo-144 (STM32)和Jetson TX2。
在特定心律不整和心臟瓣膜疾病部分，比較了基於連續小波轉換的SEmbedNet、simplified AlexNet和simplified GoogLeNet以及具有56/112圖像輸入尺寸的組合，以選擇在硬體中實現最佳和最有效的方案以建立這個快篩系統。從結果來看，在MIT-BIH心律不整的ECG數據庫中，使用輸入圖像為56像素的5層CNN（SEmbedNet）比8層CNN（simplified AlexNet）具備更好的性能，在量測單一心跳的準確率可達到99.89%。此外，SEmbedNet與輸入圖像大小為56像素和STM32的組合可以在MIT-BIH心律不整數據庫中以每單位1.3秒和1.1W獲得最佳效益。另外，針對使用通過IEC60601-1安全規範所量測出來的10秒脈搏波資料顯示，SEmbedNet對於分類竇性心律、心律不整和結構性心臟病可以達到95.26%的最佳準確率，而且在分類竇性心律和結構性心臟病可以達到99.83%的辨識率，並且只需花4秒左右的時間。
在預測脈搏振幅變異度的部分，利用Jetson TX2上實現的基於YOLOv4的QRS複合波的偵測，透過10秒心電圖及手腕橈動脈脈搏波訊號提取其中的峰值振幅和峰對峰值的間隔，並利用脈搏波峰值強度定義四個指標，分別為標準，75%、50% 和25% 的脈搏振幅變異度。結果顯示在14例心房顫動患者的初步評估中，QRS複合波的偵測可達到平均mAP (mean Average Precisions)為98.63%和F1-score 0.96。此外，針對QRS複合波的偵測和振幅變異度評分量表，其總計算時間為8.9秒、功耗為7.23瓦特。
綜合上述，在本論文中實現深度學習與硬體的結合並且達到了高準確度與高效率的成果，提升了發展成輔助醫生判斷的快篩系統的可行性，利用分級醫療的概念協助遠程監測檢測的社區醫療的可能性。
To effectively manage and monitor residents’ health in a community-based way by achieving global prevention of cardiovascular disease morbidity and mortality, AI-based hardware implementation is often regarded as one of the most cost-effective solutions for remote monitoring and prevention. Building an AI-based screening system could help physicians find the potential patient suffered from specific heart disease and temporarily monitor the health to implement graded medical care and allow doctors to spend more time treating people who need more detailed examinations or who are in urgent need of medical attention resources. However, since 2012, the vigorous development of deep learning networks has not driven its realization in hardware. The reason may be related to the depth of layers and structural complexity of the deep learning architecture. Therefore, a hardware-friendly-CNN-based hardware implementation system was proposed to a) screen arrhythmia and structural heart disease, b) evaluate the rating scales of pulse amplitude variability as an assessment index for the severity of atrial fibrillation with embedded AI computing devices utilized in this thesis, i.e., STM32F746ZG Nucleo-144 (STM32) and Jetson TX2.
In the part of arrhythmia and structural heart disease, continuous-wavelet-transformation-based SEmbedNet/simplified AlexNet and GoogLeNet with pixel 56/112 of input size were compared to choose the best and efficient combination to implement in the hardware. The result showed that in the MIT-BIH Arrhythmia ECG database, using a 5-layer CNN (SEmbedNet) with an input image of pixel 56 could get better performance than an 8-layer CNN (Simplified AlexNet) using ECG database with an accuracy of 99.89%. Besides, the combination of SEmbedNet with an input image size of pixel 56 and STM32 can achieve the best benefits with 1.3 seconds and 1.1W per unit in the MIT-BIH Arrhythmia Database. In addition, according to the 10-second PAG passed the medical safety IEC60601-1, SEmbedNet can achieve the best accuracy of 95.26% for the classification of sinus rhythm, arrhythmia, and structural heart disease. Moreover, it can reach an accuracy of 99.83% in sinus rhythm and structural heart disease classification, and it only takes about 4 seconds.
For pulse amplitude variability prediction, YOLOv4 based QRS complex detection implemented on Jetson TX2 was utilized to extract the amplitude and peak-to-peak interval on 10 seconds ECG and PAG data. At the same time, PAG amplitude was adopted to define rating scales of 75%, 50%, and 25% pulse amplitude variability according to the peak information. The result showed that based on the average mAP of 98.63% and F1-score of 0.96 on QRS complex detection, pulse amplitude variability prediction combined with QRS complex detection and rating ratio evaluation was evaluated preliminarily on 14 patients suffered from atrial fibrillation with a computing time of 8.9 seconds and power consumption of 7.23W individually.
In summary, the high performance on specific heart disease classification and low energy consumption improved the feasibility of developing a rapid screening system that assists doctors in judgment and the possibility of using the concept of graded medical care to assist remote monitoring and testing of community medical care in the further future.
[1] O. Russakovsky, J. Deng, H. Su, J. Krause S. Satheesh and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015.
[2] V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, R. Kim, R. Raman, P. C. Nelson, J. L. Mega, D. R. Webster, “Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs,” The Journal of the American Medical Association, vol. 316, no. 22, pp. 2402-2410, 2016.
[3] J. P. Simon, “Artificial Intelligence: Scope, Players, Markets and Geography,” Digital Policy, Regulation and Governance, vol. 21, no. 3, 2019.
[4] K. Ashton, “That Internet of Things thing,” RFID journal, vol. 22, no. 7, pp. 97-114, 2009.
[5] “Global Cloud Index Projects Cloud Traffic to Represent 95 Percent of Total Data Center Traffic by 2021,” Global Cloud Index Projects Strong Multicloud Traffic Growth The Network, 05-Feb-2018. [Online]. Available: https://newsroom.cisco.com/press-release-content?articleId=1908858. [Accessed: 13-Feb-2021].
[6] “Cisco Annual Internet Report (2018–2023) White Paper,” Cisco, 10-Mar-2020. [Online]. Available: https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html. [Accessed: 13-Feb-2021].
[7] W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet of Things Journal vol. 3, no. 5, pp. 637-646, 2016.
[8] “The Top 10 Causes of Death,” World Health Organization. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death. [Accessed: 12-Feb-2021].
[9] G. A. Roth, C. Johnson, A. Abajobir, F. Abd-Allah, S. F. Abera, G. Abyu and K. N. Ukwaja, “Global, Regional, and National Burden of Cardiovascular Diseases for 10 Causes, 1990 to 2015,” Journal of the American College of Cardiology, vol. 70, no. 1, pp. 1-25, 2017.
[10] M. P. Turakhia, J. Shafrin, K. Bognar, J. Trocio, Y. Abdulsattar, D. Wiederkehr and D. P. Goldman, “Estimated Prevalence of Undiagnosed Atrial Fibrillation in the United States,” PLOS One, vol. 13, no. 4, p. e0195088, 2018.
[11] “Cardiovascular Diseases,” World Health Organization. [Online]. Available: https://www.who.int/health-topics/cardiovascular-diseases#tab=tab_1. [Accessed: 08-Feb-2021].
[12] Y. E. M. Portnoy and L. D. Richardson, “Pulse Deficit as a New Innovative Non-invasive Measure to Risk-stratify Atrial Fibrillation Patients,” Circulation, vol. 138, no. 1, 2018.
[13] M. E. Williams, “Geriatric Physical Diagnosis: A Guide to Observation and Assessment,” McFarland, 2010.
[14] EA. McConnell, “Assessing Pulse Deficit,” Nursing, vol. 23, no. 11, p. 18, 1993
[15] E. Bossone, V. Rampoldi, C. A Nienaber, S. Trimarchi, A. Ballotta, J. V. Cooper, D. E Smith, K. A. Eagle, R. H. Mehta, “Usefulness of Pulse Deficit to Predict In-hospital Complications and Mortality in Patients with Acute Type A Aortic Dissection,” The American Journal of Cardiology, vol. 89, no. 7, pp. 851-855, 2002.
[16] W. Liu, X. Fang, Q. Chen, Y. Li, and T. Li, “Reliability Analysis of an Integrated Device of ECG, PPG and Pressure Pulse Wave for Cardiovascular Disease,” Microelectronics Reliability, vol. 87, pp. 183-187, 2018.
[17] J. Y. Chen, C. C. K. Lin, C. W. Lin, F. M. Yu, K. J. Li, and L.M. Tsai, “Development of Radial Artery Pulse Audiogram Sensing System for Fast Detection of Atrial Fibrillation and Pulse Amplitude Variation,” IEEE Access, vol. 8, pp. 178770-178781, 2020.
[18] C. W. Lin, Y. Chang, C. C. K. Lin, L.M. Tsai, and J. Y. Chen, “Development of an AI-based Non-invasive Pulse AudioGram Monitoring Device for Arrhythmia Screening,” 2017 IEEE Healthcare Innovations and Point of Care Technologies, pp. 40-43, 2017.
[19] L. A. Conraria, and M. J. Soares, “The Continuous Wavelet Transform: A Primer,” NIPE Working Paper, no. 23, 2011. Available: http://sites.google.com/site/aguiarconraria/joanasoares-wavelets.
[20] D. Griffin and J. Lim, “Signal Estimation from Modified Short-time Fourier Transform,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 32, no. 2, pp. 236-243, 1984.
[21] Y. L. Xie, “Evaluation of Deep Learning and Image Feature Extraction in Automatic Cardiovascular Disease Recognition based on the Time-Frequency Transformation of ECG and Pulse-audiogram,” M. S. thesis, department of biomedical engineering, NCKU, Tainan, Taiwan (R.O.C), 2019.
[22] Association for the Advancement of Medical Instrumentation, “Testing and Reporting Performance Results of Cardiac Rhythm and ST Segment Measurement Algorithms,” ANSI/AAMI EC38, 1998.
[23] S. Raj and K. C. Ray, “A Personalized Arrhythmia Monitoring Platform,” Scientific Reports, vol. 8, no. 1, pp. 1-11, 2018.
[24] Y. Xu, Z. Chen, F. Li, and J. Meng, “A Granular Resampling Method and Adaptive Speculative Mechanism-based Energy-efficient Architecture for Multiclass Heartbeat Classification,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 38, no. 11, pp. 2172-2176, 2018.
[25] Y. Zhao, Z. Shang, and Y. Lian, “A 13.34 μW Event-Driven Patient-Specific ANN Cardiac Arrhythmia Classifier for Wearable ECG Sensors,” IEEE Transactions on Biomedical Circuits and Systems, vol. 14, no. 2, pp. 186-197, 2019.
[26] N. Wang, J. Zhou, G. Dai, J. Huang, and Y. Xie, “Energy-efficient intelligent ECG monitoring for wearable devices,” IEEE Transactions on Biomedical Circuits and Systems, vol. 13 no. 5, pp. 1112-1121, 2019.
[27] H. K. Chatterjee, M. Mitra, and R. Gupta, “Real-time Detection of Electrocardiogram Wave Features using Template Matching and Implementation in FPGA,” International Journal of Biomedical Engineering and Technology, vol. 17, no. 3, pp. 290-313, 2015.
[28] E. El Mimouni and M. Karim, “Novel Real-time FPGA-based QRS Detector using Adaptive Threshold with the Previous Smallest Peak of ECG Signal,” Journal of Theoretical and Applied Information Technology, vol. 50, no. 1, pp. 33-43, 2013.
[29] K. Meddah, M. K. Talha, M. Bahoura and H. Zairi, “FPGA-based System for Heart Rate Monitoring.” IET Circuits, Devices & Systems, vol. 13, no. 6, pp. 771-782, 2019.
[30] D. Panigrahy, M. Rakshit, and P. K. Sahu, “FPGA Implementation of Heart Rate Monitoring System,” Journal of medical systems, vol. 40, pp. 3, pp. 1-12, 2016
[31] M. A. Kumar and K. M. Chari, “Efficient FPGA-based VLSI Architecture for Detecting R-peaks in Electrocardiogram Signal by Combining Shannon Energy with Hilbert Transform,” IET Signal Processing, vol. 12, no. 6, pp. 748-755, 2018
[32] S. R. Chowdhury, “Field Programmable Gate Array based Fuzzy Neural Signal Processing System for Differential Diagnosis of QRS Complex Tachycardia and Tachyarrhythmia in Noisy ECG Signals,” Journal of Medical Systems, vol. 36, no. 2, pp. 765-775, 2012.
[33] M. Qasaimeh, K. Denolf, J. Lo, K. Vissers, J. Zambreno and P. H. Jones, “Comparing energy efficiency of CPU, GPU and FPGA implementations for vision kernels.” In 2019 IEEE International Conference on Embedded Software and Systems , pp. 1-8, 2019.
[34] B. Saeedi, “The Jetson Artificial Intelligence Tool Chain (JAI-TC),” Master of Science, 2019, [online] Available: https://ir.library.oregonstate.edu/concern/graduate_projects/xw42nf01n.
[35] A. A. Süzen, B. Duman, and B. Şen, “Benchmark Analysis of Jetson TX2, Jetson Nano and Raspberry PI using Deep-CNN,” 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications , pp. 1-5, 2020
[36] L. Liu, H. Li, and M. Gruteser, “Edge assisted real-time object detection for mobile augmented reality,” The 25th Annual International Conference on Mobile Computing and Networking, pp. 1-16, 2019
[37] H. Yang, M. Huang, Z. Cai, Y. Yao, and C. Liu, “A Faster R CNN-Based Real-Time QRS Detector.” 2019 Computing in Cardiology, pp. 1-4, 2019
[38] R. Yu, Y. Gao, X. Duan, T. Zhu, Z. Wang and B. Jiao, “QRS Detection and Measurement Method of ECG Paper Based on Convolutional Neural Networks,” 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4636-4639, 2018.
[39] “IEC 60601-1:2021 SER | IEC Webstore | electromagnetic compatibility,” EMC, smart city. [Online]. Available: https://webstore.iec.ch/publication/2603. [Accessed: 19-Mar-2021].
[40] S. Patro and K. K. Sahu, “Normalization: A Preprocessing Stage.” ArXiv Preprint ArXiv:1503.06462, pp. 1-4, 2015.
[41] L. Rabiner, “Linear Program Design of Finite Impulse Response (FIR) Digital Filters,” IEEE Transactions on Audio and Electroacoustics, vol. 20, no. 4, pp. 280-288, 1972.
[42] R. Polikar, “The Wavelet Tutorial Part I,” The Wavelet Tutorial Part I, [Online]. Available: https://web.archive.org/web/20040407015627/http://users.rowan.edu/~polikar/WAVELETS/WTpart1.html. [Accessed: 19-Mar-2021].
[43] W. K. Ngui, M. S. Leong, L. M. Hee, and A. M. Abdelrhman, “Wavelet Analysis: Mother Wavelet Selection Methods,” Applied Mechanics and Materials, Trans Tech Publications Ltd. vol. 393, pp. 953-958, 2013.
[44] S. J. Huang, C. T. Hsieh and C. L. Huang, “Application of Morlet Wavelets to Supervise Power System Disturbances,” IEEE Transactions on Power Delivery, vol. 14, no. 1, pp. 235-243, 1999.
[45] J. Lin and L. Qu, “Feature Extraction based on Morlet Wavelet and its Application for Mechanical Fault Diagnosis,” Journal of Sound and Vibration, vol. 234, no. 1, pp. 135-148, 2000.
[46] M. Z. Alom, T. M. Taha, C. Yakopcic, S. Westberg, P. Sidike, M. S. Nasrin, B. C. V. Esesn, A. A. S. Awwal and V. K. Asari, “The History Began from Alexnet: A Comprehensive Survey on Deep Learning Approaches,” ArXiv Preprint ArXiv:1803.01164, pp. 1-39, 2018.
[47] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks.” Advances in Neural Information Processing Systems, pp. 1106–1114, 2012.
[48] D. Anguita, L. Ghelardoni, A. Ghio, L. Oneto, S. Ridella, “The ‘K’ in K-fold Cross Validation,” 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 441-446, 2012.
[49] J. Ng Yue-Hei, F. Yang and L. S. Davis, “Exploiting Local Features from Deep Networks for Image Retrieval.” Proceedings of the IEEE conference on Computer Vision and Pattern Recognition Workshops, pp. 53-61, in 2015
[50] X. Wen, Y. Huang, X. Wu and B. Zhang, “A Feasible Feature Extraction Method for Atrial Fibrillation Detection From BCG,” IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 4, pp. 1093-1103, 2020.
[51] D. Lai, Y. Bu, Y. Su, X. Zhang and C. -S. Ma, “Non-Standardized Patch-Based ECG Lead Together With Deep Learning Based Algorithm for Automatic Screening of Atrial Fibrillation,” IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 6, pp. 1569-1578, 2020.
[52] H. Dang, M. Sun, G. Zhang, X. Qi, X. Zhou and Q. Chang, “A Novel Deep Arrhythmia-Diagnosis Network for Atrial Fibrillation Classification Using Electrocardiogram Signals,” IEEE Access, vol. 7, pp. 75577-75590, 2019
[53] L. M. Eerikäinen, A. G. Bonomi, F. Schipper, L. R. C. Dekker,H. M. de Morree, R. Vullings, Rik and R. M. Aarts, “Detecting Atrial Fibrillation and Atrial Flutter in Daily Life Using Photoplethysmography Data,” IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 6, pp. 1610-1618, 2020.
[54] Y. Chang, “Development of an AI-based Arrhythmia and Structural Heart Disease Classification Algorithm based on Pulse AudioGram,” M. S. thesis, department of biomedical engineering, NCKU, Tainan, Taiwan (R.O.C), 2018.
[55] C. W. Lee, “Development of an Abnormal ECG Detection Algorithm in Long-term ECG using Faster Region Convolution Neural Network and Deep Convolution Neural Network,” M. S. thesis, department of biomedical engineering, NCKU, Tainan, Taiwan (R.O.C), 2020.
[56] L. Lu, Y. Zheng, G. Carneiro, and L. Yang, “Deep learning and convolutional neural networks for medical image computing,” Advances in Computer Vision and Pattern Recognition, vol. 10, pp. 978, 2017.
[57] H. Khorrami, M. Moavenian, “A Comparative Study of DWT, CWT and DCT Transformations in ECG Arrhythmias Classification,” Expert systems with Applications, vol. 37, no. 8, pp. 5751-5757, 2010.
[58] A. Bochkovskiy, C. Y. Wang, and H. Y. M. Liao, “Yolov4: Optimal Speed and Accuracy of Object Detection,” ArXiv preprint ArXiv:2004.10934, pp. 1-17, 2020.
校內：2024-07-01公開校外：2024-07-01公開