成功大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	林冠宏 Lin, Kuan-Hung
論文名稱：	基於深度學習演算法辨識手術視野清晰度並用於調控肩關節鏡注液幫浦壓力之研究 Research on Recognizing the Visual Field Clarity during Surgery based on Deep Learning Algorithms and using it to Adjust the Pressure of the Shoulder Arthroscopy Irrigation Pump
指導教授：	林哲偉 Lin, Che-Wei
學位類別：	碩士 Master
系所名稱：	工學院 - 生物醫學工程學系 Department of BioMedical Engineering
論文出版年：	2021
畢業學年度：	109
語文別：	英文
論文頁數：	84
中文關鍵詞：	肩關節鏡手術、術後復原、術野清晰度、卷積神經網路、深度學習、出血遮罩、NVIDIA Jetson TX2 、注液幫浦
外文關鍵詞：	Shoulder arthroscopy surgery, Postoperative recovery, Visual field clarity during surgery, Convolutional neural network, Deep learning, Bleeding mask, NVIDIA Jetson TX2, Irrigation pump
相關次數：	點閱：87 下載：3
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

肩關節鏡手術中注液幫浦壓力不但與術野清晰程度有直接相關，與患者復原狀況亦有高關聯性，若可透過機電整合即時調控壓力、取得清晰的手術視野、並最小化灌溉量，預期可使手術患者復原狀況較佳。本研究開發了「基於深度學習辨識手術視野清晰度之演算法並用於實現自動調整肩關節鏡注水幫浦壓力」的模擬平台，此模擬平台研究包括已實現於NVIDIA Jetson TX2邊緣運算裝置的深度學習手術視野清晰度辨識演算法、以及可自動調控注液幫浦壓力的機電整合模擬平台。手術視野清晰度辨識演算法的資料來源為成大醫院骨科部施行肩關節鏡手術之影像（共22部影片，影片錄製規格為每秒30幀影像），全部資料集共有57472幀影像。在骨科醫師的標註下，手術視野品質標註為清晰(Clear)、普通(Normal)、不良(Not good)三種，之後分類經過調整增加反光(Reflective)與未知(Unknown)至五類。本研究以Inception V3, MobileNet和Xception三種深度學習網路建構手術視野清晰度辨識演算法，以5-fold交叉驗證，分別得到97.68%, 97.66% 和 98.31%的辨識正確率，上述深度學習網路在NVIDIA Jetson TX2邊緣運算裝置執行每一幀影像的運算時間分別約為121、32和87毫秒(ms)，因此可達到在每秒30幀影像的錄製品質下接近即時辨識的效果。而為了達到更進一步的辨識效能，使邊界更能區分開來，必須將比較多誤判的分類—中間段數Normal的準確率提升，此研究測試加入出血遮罩用來濾掉血液與沖洗液浮動的動態畫面，剩餘之影像分類效能皆得到明顯提升，更在Xception的精確率 (Precision) 與召回率 (Recall)得到99%以上的效能。且為了連續辨識每幀影像的清晰度並自動調控壓力而設計一套軟、韌、硬體結合的系統，本研究已完成「深度學習演算法辨識手術視野清晰度並用於調控肩關節鏡注液幫浦壓力」模擬平台的建構及先導研究。

The irrigation pump pressure during shoulder arthroscopy surgery is directly related to the visual field clarity during the surgery and highly related to the patient's prognosis status. If the pressure can be regulated in real-time through electromechanical integration, a clear surgical visual field can be obtained, and the amount of irrigation can be minimized. It is expected that the recovery of surgical patients will be better. This research has developed a simulation platform for “An algorithm based on deep learning to recognize the visual field clarity during surgery and used to adjust the pressure of the shoulder arthroscopy irrigation pump automatically.” This simulation platform research includes the deep learning implemented on the NVIDIA Jetson TX2 edge computing device. The visual field recognition algorithm and the electromechanical integrated simulation platform can automatically adjust the pressure of the irrigation pump. The data source of the algorithm for recognizing the visual field clarity during surgery is the images of the shoulder arthroscopy surgery performed from the Department of Orthopedics, National Cheng Kung University Hospital (22 videos in total and the video recording specification is 30 frames per second). The entire data set has a total of 57472 frames of images. Under the label of the orthopedic surgeon, the quality of the surgical field was labeled as Clear, Normal, and Not good. Afterward, the classification was adjusted to add Reflective and Unknown to five categories. This study used three deep learning networks, Inception V3, MobileNet, and Xception, to construct the clarity recognition algorithm. With 5-fold cross-validation, the recognition accuracy rates of 97.68%, 97.66%, and 98.31% were obtained, respectively. The above-mentioned deep learning network. The calculation time for each frame of an image on the NVIDIA Jetson TX2 edge computing device is approximately 121, 32, and 87 milliseconds (ms), respectively, to achieve it near-real-time recognition at a recording quality of 30 frames per second. To achieve further recognition performance and make the boundary more distinguishable, it is necessary to improve the accuracy of the more misidentification—Normal in the middle segment. This research test adds the bleeding mask to filter out the dynamics of blood and flushing fluid floating image. The remaining image classification performance has been significantly improved, and the performance of Xception's Precision and Recall has been over 99%. To continuously recognize the clarity of each frame of images and automatically adjust the pressure, a complete set of software, hardware, and firmware combinations are designed. This research has completed the “Deep learning algorithm to recognize the visual field clarity during surgery and use it to adjust the pressure of the irrigation pump.” Its simulation platform construction and pilot research will be developed.

摘 要	ii
Abstract	iv
Acknowledgements	vi
Table of Contents	vii
List of Tables	ix
List of Figures	x
Chapter 1  Introduction	1
1 Introduction	1
2 Literature Review	2
2.1 The importance of visual field clarity during surgery	3
2.2 The influence of fluid irrigation control on postoperative recovery	5
2.3 The influence of fluid irrigation control on visual field clarity	6
2.4 The surgical visual field clarity recognition algorithm	7
2.5 The development of AI surgical aids	9
3 Motivation	9
4 Organization of this thesis	10
Chapter 2 Deep Learning Model for Visual Field Clarity Recognition	12
1 Algorithm Architecture	12
2 Dataset	13
2.1 Definition of surgical visual field clarity	13
2.2 Data feature observation	17
3 Preprocessing	21
4 Data Generation (Data Augmentation)	21
5 Deep Learning Algorithm – CNN	23
5.1 Inception V3	24
5.2 MobileNet	26
5.3 Xception	29
6 CV	31
6.1 K-fold CV	31
7 Model Evaluation	32
7.1 Confusion matrix (CM)	32
7.2 PR curve and AUC values	32
8 Judgment method – Majority Vote	33
9 Image Processing Algorithm (Bleeding part)	34
9.1 HSV	34
9.2 BM	36
Chapter 3 Simulation platform for irrigation pump	37
1 System Architecture	37
1.1 NVIDIA Jetson TX2	39
2 Hardware Architecture	39
3 The Threads of Software and Firmware on TX2 (CNN)	41
4 Judgment Method – Majority Vote	43
5 The Threads of Software and Firmware on TX2 (Adding BM)	44
6 Comparison of Actual Platform and Simulation Platform	45
Chapter 4 Experimental Results	46
1 Recognition Accuracy Experimental Results	46
1.1 Result 1 - Compare with three different 5-fold CV CNN architectures	48
1.2 Result 2 - BM performance	51
1.3 Result 3 - Compare with three 5-fold CV CNNs on the no bleeding part	52
1.4 Result 4 – Test the CNN model on the NVIDIA Jetson TX2 platform	54
1.5 Result 5 – Combine BM with CNN model and test on the TX2	57
1.6 Result 6 – Combine BM with CNN model (no bleeding) on the TX2	60
2 Recognition Speed Results	63
Chapter 5  Discussion and Conclusion	64
1 Discussions	64
1.1 Comparison of CNN part and adding the BM	64
1.2 Comparison of three models when adding the BM	66
1.3 Compare the sensitivity and precision of the normal part	66
2 Algorithm Limitations	69
2.1 Image processing – Bleeding part	69
2.2.1 Blur Detection	70
3 Hardware Limitations	72
4 Conclusion and Future Work	73
References	75
Appendix A – Dataset of 22 videos this study used	84
                                    

[1] M. Pribicevic, “The epidemiology of shoulder pain: A narrative review of the literature,” Pain in Perspective, London, UK, 2012, ch. 7, pp. 147–186.
[2] SCMH, “Understanding arthroscopy,” scmh.org.tw, [Online]. Available: https://www.scmh.org.tw/internet/show_org/headline_detail1.asp?no=712&kind=1&img=1. [Accessed June 30, 2021].
[3] P. Berjano, “Complications in arthroscopic shoulder surgery,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol.14, pp. 785–788, 1998.
[4] K. H. Jensen, K. Werther, V. Stryger, K. Schultz, B. Falkenberg, “Arthroscopic shoulder surgery with epinephrine saline irrigation,” Arthroscopy, vol. 17, pp. 578–581, 2001.
[5] B. C. Bomberg, P. E. Hurley, C. A. Clark and C. S. McLaughlin, “Complications associated with the use of an infusion pump during knee arthroscopy,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 8, pp. 224–228, 1992.
[6] G. J. M. Tuijthof, M. M. de Vaal, I. N. Sierevelt, L. Blankevoort and M. P. vander List, “Performance of arthroscopic irrigation systems assessed with automatic blood detection,” Knee Surg Sports Traumatol Arthrosc, vol. 19, pp. 1948–1954, 2011.
[7] S. S. Burkhart, S. M. Danaceau and K. A. Athanasiou, “Turbulence control as a factor in improving visualization during subacromial shoulder arthroscopy,” Arthroscopy, vol. 17, pp. 209–212, 2001.
[8] C. Zhang, M. Xiao and D. Lu, “Temporary application of ox muzzle drainage in knee arthroscopic surgery,” Research Square, 2021.
[9] Y. F. Liu, C. K. Hong, K. L. Hsu, F. C. Kuan, Y. Chen, M. L. Yeh and W. R. Su, “Intravenous administration of tranexamic acid significantly improved clarity of the visual field in arthroscopic shoulder surgery. A prospective, double-blind, and randomized controlled trial,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 36, pp. 640–647, 2020.
[10] M. A. Camarasa, G. Olle, M. Serra-Prat, A. Martín, M. Sánchez, P. Ricós, A. Pérez and L. Opisso, “Efficacy of aminocaproic, tranexamic acids in the control of bleeding during total knee replacement: A randomized clinical trial,” BJA: British Journal of Anaesthesia, vol. 96, pp. 576–582, 2006.
[11] L. Felli, S. Revello, G. Burastero, P. Gatto, A. Carletti, M. Formica, A. M. Mattia, “Single intravenous administration of tranexamic acid in anterior cruciate ligament reconstruction to reduce postoperative hemarthrosis and increase functional outcomes in the early phase of postoperative rehabilitation: A randomized controlled trial,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 35, pp. 149–157. 2019.
[12] F. Karaaslan, R. Seijas, A. Sallent, O. Ares-Rodriguez, W. Espinosa, P. Alvarez, R. Cugat and P. Lopez, “Tranexamic acid in bolus vs. infusion in hip arthroscopy; Bleeding improvement as a silent complication,” International Journal of Orthopaedics, vol. 4, no. 3, pp. 749–752, 2017.
[13] L. Good, E. Peterson and B. Lisander, “Tranexamic acid decreases external blood loss but not hidden blood loss in total knee replacement,” BJA: British Journal of Anaesthesia, vol. 90, pp. 596–599, 2003.
[14] E. R. Chiang, K. H. Chen, S. T. Wang, H. L. Ma, M. C. Chang, C. L. Liu and T. H. Chen, “Intra-articular injection of tranexamic acid reduced postoperative hemarthrosis in arthroscopic anterior cruciate ligament reconstruction: A prospective randomized study,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 35, pp. 2127–2132, 2019.
[15] L. Pauzenberger, M. A. Domej, P. R. Heuberer, M. Hexel, A. Grieb, B. Laky, J. Blasl and W. Anderl, “The effect of intravenous tranexamic acid on blood loss and early post-operative pain in total shoulder arthroplasty,” The Bone & Joint Journal, vol. 99–B, pp. 1073–1079, 2017.
[16] J. W. Belk, E. C. McCarty, D. A. Houck, J. L.Dragoo, F. H. Savoie and S. G. Thon, “Tranexamic acid use in knee and shoulder arthroscopy leads to improved outcomes and fewer hemarthrosis-related complications: a systematic review of level I and II studies,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 37, pp. 1323–1333, 2021.
[17] W. L. Johns, K. C. Walley, R. Seedat, B. Jackson III, K. Boukhemis and T. Gonzalez, “Tranexamic acid use in foot and ankle surgery,” Foot & Ankle Orthopaedics, vol. 5, no. 4, pp. 1–6, 2020.
[18] W. L. Johns, K. C. Walley, S. Hammoud, T. A. Gonzalez, M. G. Ciccotti and N. K. Patel, “Tranexamic acid in anterior cruciate ligament reconstruction: A systematic review and meta-analysis,” The American Journal of Sports Medicine, 2021.
[19] W. L. Johns, , Walley, K. C., B. Jackson III, T. A. Gonzalez, “Tranexamic acid in foot and ankle surgery: A topical review and value analysis,” Foot & Ankle Specialist, 2021.
[20] M. J. Alaia and A. M. Gipsman, “Editorial commentary: The benefits of tranexamic acid may outweigh risks in arthroscopy and sports medicine,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 37, pp. 1334–1336, 2021.
[21] A. W. Hartland, K. H. Teoh and M. S. Rashid, “Clinical effectiveness of intraoperative tranexamic acid use in shoulder surgery: A systematic review and meta-analysis,” The American Journal of Sports Medicine, vol. 49, pp. 3145–3154, 2021.
[22] E. Bayram, C. Yıldırım, A. K. Ertürk, M. Yılmaz, and D. Atlıhan, “Comparison of the efficacy of irrigation with epinephrine or tranexamic acid on visual clarity during arthroscopic rotator cuff repair: A double-blind, randomized-controlled study,” Joint Diseases and Related Surgery, vol. 32, pp. 115, 2021.
[23] W. B. Stetson, S. A. Morgan, S. Polinsky, B. Chung and N. J. Hung, “Cost effective technique of shoulder arthroscopy without the use of epinephrine in irrigation solution,” Arthroscopy Techniques, vol. 10, pp. e411–e418, 2021.
[24] D. O. van Montfoort, P. M. van Kampen and P. E. Huijsmans, “Epinephrine diluted saline–irrigation fluid in arthroscopic shoulder surgery: A significant improvement of clarity of visual field and shortening of total operation time. A randomized controlled trial,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 32, pp. 436–444, 2016.
[25] D. M. Avery III, B. W. Gibson, and G. F. Carolan, “Surgeon-rated visualization in shoulder arthroscopy: A randomized blinded controlled trial comparing irrigation fluid with and without epinephrine,” Arthroscopy: The Journal of Arthroscopic & Related Surgery, vol. 31, pp. 12–18, 2015.
[26] A. Borgeat, P Bird, G. Ekatodramis and C. Dumont, “Tracheal compression caused by periarticular fluid accumulation: A rare complication of shoulder surgery,” Journal of Shoulder Elbow Surgery, vol. 9, pp. 443–445, 2000.
[27] C. D. Smith and M. M. Shah, “Fluid gain during routine shoulder arthroscopy,” Journal of Shoulder and Elbow Surgery, vol. 17, pp. 415–417, 2008.
[28] G. J. M. Tuijthof, M. M. de Vaal, I. N. Sierevelt, L. Blankevoort and M. P. J. van der List, “Performance of arthroscopic irrigation systems assessed with automatic blood detection,” Knee Surgery, Sports Traumatology, Arthroscopy, vol. 19, pp. 1948–1954, 2011.
[29] N. Oretorp and S. Elmersson, “Arthroscopy and irrigation control,” The Journal of Arthroscopic & Related Surgery, vol. 2, pp. 46–50, 1986.
[30] L. T. Kuo, C. L. Chen, P. A. Yu, W. H. Hsu, C. C. Chi and J. C. Yoo, “Epinephrine in irrigation fluid for visual clarity in arthroscopic shoulder surgery: a systematic review and meta-analysis,” International Orthopedics, vol. 42, pp. 2881–2889, 2018.
[31] Y. F. Liu, C. K. Hong, K. L. Hsu, F. C. Kuan, Y. Chen, M. L. Yeh and W. R. Su, “Intravenous administration of tranexamic acid significantly improved clarity of the visual field in arthroscopic shoulder surgery. A prospective, double-blind, and randomized controlled trial,” Arthroscopy, vol. 36, pp. 640–647, 2020.
[32] G. J. M. Tuijthof, I. N. Sierevelt and C. N. van Dijk, “Disturbances in the arthroscopic view defined with video analysis,” Knee Surgery, Sports Traumatology, Arthroscopy, vol. 15, pp. 1101–1106, 2007.
[33] G. J. M. Tuijthof, M Abbink, I. N. Sierevelt and C. N. van Dijk, “Multirater agreement on arthroscopic image quality,” Journal of Engineering in Medicine, vol. 223, pp.179–187, 2008.
[34] F. L. Yu, J. Sun, A. Li, J. Cheng, C. Wan and J. Liu, “Image quality classification for DR screening using deep learning,” Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Jeju, Korea, vol. 2017, pp. 664–667, 2017.
[35] Y. Yuan and M. Q. H. Meng, “Deep learning for polyp recognition in wireless capsule endoscopy images,” Med Phys, vol. 44, pp. 1379–1389, 2017.
[36] M. Alicandri-Ciufelli, L. Pingani, D. Mariano, L. Anschuetz, G. Molinari, D. Marchioni, M. Bonali, G. M. Galeazzi and L. Presutti, “Rating surgical field quality in endoscopic ear surgery: proposal and validation of the "Modena Bleeding Score",” European Archives of Oto-Rhino-Laryngology, vol. 276, pp. 383–388, 2019.
[37] T. Athanasiadis, A. Beule, J. Embate, E. Steinmeier, J. Field and P. J. Wormald, “Standardized video-endoscopy and surgical field grading scale for endoscopic sinus surgery: a multi-centre study,” Laryngoscope, vol.118, pp. 314–319, 2008.
[38] J. Shihadeh, A. Ansari and T. Ozunfunmi, “Deep learning based image classification for remote medical diagnosis,” IEEE Global Humanitarian Technology Conference, CA, USA, pp. 1–8, 2018.
[39] C. Tang, S. Xia, M. Qian and B. Wang, “Deep learning-based vein localization on embedded system,” IEEE Access, vol. 9, pp. 27916–27927, 2021.
[40] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Communications of the ACM, vol. 60, pp. 84–90, 2017.
[41] L. Taylor and G. Nitschke, “Improving deep learning using generic data augmentation,” Aug. 2017, [online]. Available: https://arxiv.org/abs/1708.06020. [Accessed June 30, 2021].
[42] L. Perez and J. Wang, “The effectiveness of data augmentation in image classification using deep learning,” Dec. 2017. [Online]. Available: http://arxiv.org/abs/1712.04621. [Accessed June 30, 2021].
[43] I. Goodfellow, Y. Bengio and A. Courville, “Deep Learning,” MIT Press, vol. 1, pp. 326, 2016.
[44] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens and Z. Wojna, “Rethinking the inception architecture for computer vision,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 2818–2826, 2016.
[45] Jian Yi's reading notes, “Inception series – Inception v2, Inception v3,” Medium.net, Nov 7, 2020. [Online]. Available: https://medium.com/ching-i/inception-%E7%B3%BB%E5%88%97-inceptionv2-inceptionv3-93cd42054d23. [Accessed Juan. 30, 2021].
[46] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” Computer Vision and Pattern Recognition, Apr. 17, 2017. [online] Available: https://arxiv.org/abs/1704.04861. [Accessed Juan. 30, 2021].
[47] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” Computer Vision and Pattern Recognition, Oct. 7, 2016. [online] Available: https://arxiv.org/abs/1610.02357. [Accessed Juan. 30, 2021].
[48] M. Stone, “Cross-validatory choice and assessment of statistical predictions,” Journal of the Royal Statistical Society, vol. 36, pp. 111–147, 1974.
[49] M. Stone, “An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion,” Journal of the Royal Statistical Society, vol. 39, pp. 44–47, 1977.
[50] K. Ron, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, vol. 2, pp. 1137–1143, 1995.
[51] EXSILIO solutions, “Accuracy, precision, recall & F1 score: Interpretation of performance measures,” Sep. 2016, [Online]. Available: https://blog.exsilio.com/all/accuracy-precision-recall-f1-score-interpretation-of-performance-measures/. [Accessed Juan. 30, 2021].
[52] T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol. 27, pp. 861–874, 2006.
[53] LaptrinhX, “Gaining an intuitive understanding of Precision and Recall,” Apr. 2020, [Online]. Available: https://laptrinhx.com/gaining-an-intuitive-understanding-of-precision-and-recall-2631030324/. [Accessed Juan. 30, 2021].
[54] W. Chen, Y. Shi and G. Xuan, “Identifying computer graphics using HSV color model and statistical moments of characteristic functions,” IEEE International Conference on Multimedia and Expo, Beijing, China, pp. 1123–1126, 2007.
[55] B. Su, S. Lu and C. L. Tan, “Blurred image region detection and classification,” In ACM International Conference on Multimedia, pp. 1397–1400, 2011.

校內：2023-09-01公開
校外：2023-09-01公開

簡易檢索 / 詳目顯示

相關論文