簡易檢索 / 詳目顯示

研究生: 楊芮瑄
Yang, Ruei-Syuan
論文名稱: 應用深度學習與擴增實境技術建構傷口量測系統
Developing a Wound Measurement System using Deep Learning and Augmented Reality Technology
指導教授: 王泰裕
Wang, Tai-Yue
學位類別: 碩士
Master
系所名稱: 管理學院 - 工業與資訊管理學系
Department of Industrial and Information Management
論文出版年: 2021
畢業學年度: 109
語文別: 中文
論文頁數: 84
中文關鍵詞: 傷口量測擴增實境深度學習影像處理
外文關鍵詞: wound measurement, augmented reality, deep learning, image processing
相關次數: 點閱:170下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 現今傷口量測需放置紙尺、黏貼參考標記作為參考點,如同地圖上的比例尺。而在醫院急診室繁忙的環境下,此方法相當不便,醫療人員一旦忘記放置參考點,就需要重新量測。本研究探討傷口量測領域,提出一個新方法協助醫療人員進行傷口量測,以開發一個方便使用者量測傷口的手機應用程式為目標,並利用虛擬參考點進行傷口量測。本研究建置的傷口量測系統共包含3個子系統,分別是量測子系統、傷口偵測子系統與傷口辨識子系統,量測子系統使用擴增實境(Augmented Reality, AR)技術、相似三角形原理和距離公式計算傷口大小;而傷口偵測子系統使用深度學習(deep learning)方法訓練傷口偵測模型,最後傷口辨識子系統使用影像處理流程,進行傷口邊緣辨識。訓練傷口偵測模型需要過往的傷口影像,且由於蒐集到的五類傷口資料量不平均,透過資料擴增方式來產生充足且均勻的樣本。本研究亦提出使用放置在基準面上的虛擬矩形判斷拍攝當下手機傾斜程度,應用透視轉換(perspective transformation)校正手機傾斜產生的影像變形。由實證結果得知,在拍攝角45度與距離20公分時,量測結果最佳,誤差皆小於0.5公分,符合本研究訂定誤差小於1公分的目標。另外,不同拍攝角度的長與寬平均誤差降低量為0.184公分與0.062公分;不同拍攝距離的長與寬平均誤差降低量為0.093公分與0.01公分,平均誤差降低量皆大於0,足以確認本研究提出的校正方法能有效降低量測誤差。

    Measuring the surface area of a wound is a vital step in wound assessment. Nowadays, the most commonly used methods of wound assessment in hospital emergency room, including placing a paper ruler or sticking a reference mark around the wounded area, are cumbersome and time-consuming. The process must be repeated if the health care providers forget to place the reference point next to the wounded area. Comparatively little research has focused on usage environment and user experience. Therefore, the objective of this study is to design a wound measurement system which is easy-to-use, zero-cost and time-saving. In our research, we use a virtual rectangle as a reference point and implement it on an Android system. Our wound measurement system comprises three subsystems, namely: measurement, wound detection, and wound recognition. These subsystems are designed based on augmented reality (AR), deep learning, and digital image processing (DIP), respectively. In order to improve accurracy of the system, we use perspective transformation algorithm to rectify wound image deviation. The results show that the average measurement error is less than 0.5 cm, which is in line with the error formulated in this paper. From different shooting angles, the mean length and width error reduction are 0.184 cm and 0.062 cm, respectively. From different shooting distances, the mean error reduction of length and width are 0.093 cm and 0.001 cm, respectively. These results show that using a virtual reference point to rectify wound image can undeniably reduce errors.

    摘要 i 致謝 viii 目錄 x 表目錄 xii 圖目錄 xiv 第一章 緒論 1 第一節 研究背景與動機 1 第二節 研究目的 3 第三節 研究範圍與限制 3 第四節 研究流程 4 第五節 論文架構 5 第二章 文獻探討 6 第一節 傷口量測 6 第二節 擴增實境 11 第三節 深度學習 14 第四節 數位影像處理 23 第五節 小結 30 第三章 以影像處理為基礎之傷口量測系統架構設計 31 第一節 問題描述與使用情節 31 第二節 傷口量測系統架構 32 第三節 量測子系統 35 第四節 傷口偵測子系統 36 第五節 傷口辨識子系統 37 第六節 校正 39 第七節 評估指標 41 第八節 小結 42 第四章 系統開發與結果分析 43 第一節 資料說明與資料前處理 43 第二節 環境建置與系統開發 47 第三節 系統分析 56 第四節 實證結果 66 第五節 小結 69 第五章 結論及建議 71 第一節 結論 71 第二節 研究貢獻 72 第三節 未來研究建議與方向 73 參考文獻 74 附錄A 80 附錄B 81 附錄C 83 附錄D 84

    [1]"元智大學資訊管理學系第二十一屆專業實習成果報告." https://captainkuo1234.wixsite.com/mysite2/blank-3 (accessed Apr 17, 2021).
    [2]"手機王." https://www.sogi.com.tw/products/samsung_galaxy_a60/14781 (accessed Apr 27, 2021).
    [3]"如何估算前方人物的距離?" https://chtseng.wordpress.com/2018/09/18/%E5%A6%82%E4%BD%95%E4%BC%B0%E7%AE%97%E5%89%8D%E6%96%B9%E4%BA%BA%E7%89%A9%E7%9A%84%E8%B7%9D%E9%9B%A2/ (accessed Oct 19, 2020).
    [4]"醫療服務給付項目及支付標準查詢服務." https://www.nhi.gov.tw/query/Query2_Print.aspx?ChtName=%e5%82%b7%e5%8f%a3&EngName=&Q2ID=&StartDateYYY=&StartDateMM=&ShowType=2 (accessed Jan 4, 2021).
    [5]"全民健康保險醫療服務給付項目及支付標準." https://www.nhi.gov.tw/DL.aspx?sitessn=292&u=LzAwMS9VcGxvYWQvMjkyL3JlbGZpbGUvMC82NTcwL%2bmGq%2beZguacjeWLmee1puS7mOmgheebrigxMDkxMjMx5pu05pawKS5wZGY%3d&n=6Yar55mC5pyN5YuZ57Wm5LuY6aCF55uuKDEwOTEyMzHmm7TmlrApLnBkZg%3d%3d&ico%20=.pdf (accessed Jan 4, 2021).
    [6]"南山人壽醫療險附約." https://www.nanshanlife.com.tw/NanshanWeb/file/DOCUMENT/10400 (accessed Jan 4, 2021).
    [7]"國泰人壽-心安安醫療終身保險." https://www.cathaylife.com.tw/cathaylife/products/insurance/protection/health# (accessed Jan 4, 2021).
    [8]"Fundamental concepts | ARCore | Google Developers." Google. https://developers.google.com/ar/discover/concepts (accessed Oct 19, 2020).
    [9]T. Acharya and A. K. Ray, Image processing: principles and applications. John Wiley & Sons, 2005.
    [10]A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. García, S. Gil-López, D. Molina, and R. Benjamins, "Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI," Information Fusion, vol. 58, pp. 82-115, 2020.
    [11]R. T. Azuma, "A survey of augmented reality," Presence: Teleoperators & Virtual Environments, vol. 6, no. 4, pp. 355-385, 1997.
    [12]D. L. Baggio, OpenCV 3.0 computer vision with Java. Packt Publishing Ltd, 2015.
    [13]E. E. Baraboshkin, L. S. Ismailova, D. M. Orlov, E. A. Zhukovskaya, G. A. Kalmykov, O. V. Khotylev, E. Y. Baraboshkin, and D. A. Koroteev, "Deep convolutions for in-depth automated rock typing," Computers & Geosciences, vol. 135, p. 104330, 2020, doi: https://doi.org/10.1016/j.cageo.2019.104330.
    [14]S. Bhaskar, N. Lavassar, and S. Green. "Implementing optical character recognition on the android operating system for business cards." https://stacks.stanford.edu/file/druid:rz261ds9725/Bhaskar_Lavassar_Green_BusinessCardRecognition.pdf (accessed Jun 6, 2021).
    [15]J. D. Bills, S. J. Berriman, D. L. Noble, L. A. Lavery, and K. E. Davis, "Pilot study to evaluate a novel three‐dimensional wound measurement device," International Wound Journal, vol. 13, no. 6, pp. 1372-1377, 2016.
    [16]A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint arXiv:2004.10934, 2020.
    [17]A. Bousaid, T. Theodoridis, S. Nefti-Meziani, and S. Davis, "Perspective Distortion Modeling for Image Measurements," IEEE Access, vol. 8, pp. 15322-15331, 2020, doi: 10.1109/access.2020.2966716.
    [18]A. Buslaev, V. I. Iglovikov, E. Khvedchenya, A. Parinov, M. Druzhinin, and A. A. Kalinin, "Albumentations: Fast and Flexible Image Augmentations," Information, vol. 11, no. 2, p. 125, 2020. [Online]. Available: https://www.mdpi.com/2078-2489/11/2/125.
    [19]A. C. Chang, B. Dearman, and J. E. Greenwood, "A comparison of wound area measurement techniques: visitrak versus photography," Eplasty, vol. 11, pp. e18-e18, 2011. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3080766/.
    [20]P. Foltynski, P. Ladyzynski, and J. M. Wojcicki, "A new smartphone‐based method for wound area measurement," Artificial Organs, vol. 38, no. 4, pp. 346-352, 2014.
    [21]A. GeethaKiran and S. Murali, "Automatic rectification of perspective distortion from a single image using plane homography," International Journal of Computer Science & Applications, vol. 3, pp. 47-58, 2013.
    [22]P. Gholami, M. A. Ahmadi-pajouh, N. Abolftahi, G. Hamarneh, and M. Kayvanrad, "Segmentation and Measurement of Chronic Wounds for Bioprinting," IEEE Journal of Biomedical and Health Informatics, vol. 22, no. 4, pp. 1269-1277, 2018, doi: 10.1109/JBHI.2017.2743526.
    [23]R. Gonzalez and R. Woods, Digital image processing, 4th ed. Pearson, 2017.
    [24]T. Goto, G. Nakagami, A. Nakai, S. Noyori, S. Sasaki, C. Hayashi, T. Miyagaki, K. Akamata, and H. Sanada, "Utility of a three-dimensional wound measurement device in pressure ulcers," Chronic Wound Care Manag Res, vol. 4, no. 4, pp. 129-33, 2017.
    [25]C. E. Hammond, "The ARANZ medical silhouette[TM]: an innovative wound measurement and documentation system," Acute Care Perspectives, vol. 17, p. 12+, 2008.
    [26]A. Henrysson, M. Billinghurst, and M. Ollila, "Face to face collaborative AR on mobile phones," in Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ismar'05), Vienna, 2005, pp. 80-89.
    [27]G. E. Hinton, S. Osindero, and Y.-W. Teh, "A Fast Learning Algorithm for Deep Belief Nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.
    [28]S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
    [29]J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proceedings of the National Academy of Sciences of the United States of America, vol. 79, no. 8, pp. 2554-2558, 1982.
    [30]A. Körez, N. Barışçı, A. Çetin, and U. Ergün, "Weighted Ensemble Object Detection with Optimized Coefficients for Remote Sensing Images," ISPRS International Journal of Geo-Information, vol. 9, no. 6, p. 370, 2020. [Online]. Available: https://www.mdpi.com/2220-9964/9/6/370.
    [31]H. Kato and M. Billinghurst, "Marker tracking and HMD calibration for a video-based augmented reality conferencing system," in Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99), NW Washington, 1999, pp. 85-94, doi: 10.1109/IWAR.1999.803809.
    [32]K. Klinker, M. Wiesche, and H. Krcmar, "Digital transformation in health care: Augmented reality for hands-free service innovation," Information Systems Frontiers, vol. 22, no. 6, pp. 1419-1431, 2020.
    [33]A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Advances in Neural Information Processing Systems, vol. 25, pp. 1097-1105, 2012.
    [34]G. S. Lazarus, D. M. Cooper, D. R. Knighton, D. J. Margolis, R. E. Percoraro, G. Rodeheaver, and M. C. Robson, "Definitions and guidelines for assessment of wounds and evaluation of healing," Wound Repair and Regeneration, vol. 2, no. 3, pp. 165-170, 1994.
    [35]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
    [36]X. Li, S. Li, W. Bai, X. Cui, G. Yang, H. Zhou, and C. Zhang, "Method for rectifying image deviation based on perspective transformation," IOP Conference Series: Materials Science and Engineering, vol. 231, p. 012029, 2017, doi: 10.1088/1757-899x/231/1/012029.
    [37]M. Marvin and A. P. Seymour, Perceptrons: an introduction to computational geometry. MIT Press, 1969.
    [38]J. McCardle, M. Smith, E. Brewin, and M. Young, "Visitrak: wound measurement as an aid to making treatment decisions," Diabetic Foot, vol. 8, no. 4, p. 207, 2005.
    [39]N. Mo and L. Yan, "Improved Faster RCNN Based on Feature Amplification and Oversampling Data Augmentation for Oriented Vehicle Detection in Aerial Images," Remote Sensing, vol. 12, no. 16, p. 2558, 2020. [Online]. Available: https://www.mdpi.com/2072-4292/12/16/2558.
    [40]M. Negnevitsky, Artificial Intelligence: A Guide to Intelligent Systems, 3rd ed. Pearson Education, 2011.
    [41]M. Nemeth, S. Sprigle, and A. Gajjala, "Clinical usability of a wound measurement device," in 36th Annual American Spinal Injury Conference, Nashville, 2010.
    [42]D. Nixon, A. Philps, L. Jonsson, and M. Nixon, "Measurement of diabetic foot ulcers using SilhouetteMobile," J Wound Technol, vol. 3, pp. 37-8, 2009.
    [43]F. Patin. "An introduction to digital image processing." http://teachme.free.fr/ImageProc.pdf (accessed Jun 1, 2021).
    [44]R. Raizman, D. Dunham, L. Lindvere-Teene, L. M. Jones, K. Tapang, R. Linden, and M. Y. Rennie, "Use of a bacterial fluorescence imaging device: wound measurement, bacterial detection and targeted debridement," Journal of Wound Care, vol. 28, no. 12, pp. 824-834, 2019.
    [45]J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016, pp. 779-788.
    [46]J. Rogowska, K. Preston, and D. Sashin, "Evaluation of digital unsharp masking and local contrast stretching as applied to chest radiographs," IEEE Transactions on Biomedical Engineering, vol. 35, no. 10, pp. 817-827, 1988, doi: 10.1109/10.7288.
    [47]F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, no. 6, p. 386, 1958.
    [48]N. S. Rosni, Z. Abd Kadir, M. N. M. M. Noor, Z. H. A. Rahman, and N. A. Bakar, "Development of mobile markerless augmented reality for cardiovascular system in anatomy and physiology courses in physiotherapy education," in 14th International Conference on Ubiquitous Information Management and Communication (IMCOM), Taichung, 2020, pp. 1-5.
    [49]D. E. Rumelhart and J. L. McClelland, Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations. MIT Press, 1986.
    [50]J. Rychen, J. Goldberg, A. Raabe, and D. Bervini, "Augmented Reality in Superficial Temporal Artery to Middle Cerebral Artery Bypass Surgery: Technical Note," Operative Neurosurgery, vol. 18, no. 4, pp. 444-450, 2019, doi: 10.1093/ons/opz176.
    [51]S. Sharma and S. Sharma, "Activation functions in neural networks," Towards Data Science, vol. 6, no. 12, pp. 310-316, 2017.
    [52]R. Shetty, H. Sreekar, S. Lamba, and A. K. Gupta, "A novel and accurate technique of photographic wound measurement," Indian journal of plastic surgery: official publication of the Association of Plastic Surgeons of India, vol. 45, no. 2, p. 425, 2012.
    [53]J. E. Solem, Programming Computer Vision with Python: Tools and algorithms for analyzing images. O'Reilly Media, Inc., 2012.
    [54]J. Sugama, Y. Matsui, H. Sanada, C. Konya, M. Okuwa, and A. Kitagawa, "A study of the efficiency and convenience of an advanced portable wound measurement system (VISITRAKTM)," Journal of Clinical Nursing, vol. 16, no. 7, pp. 1265-1269, 2007.
    [55]I. E. Sutherland, "A head-mounted three dimensional display," in Proceedings of the December 9-11, 1968, fall joint computer conference, part I, San Francisco, California, 1968: Association for Computing Machinery, pp. 757–764, doi: 10.1145/1476589.1476686. [Online]. Available: https://doi.org/10.1145/1476589.1476686
    [56]Y. Tai, L. Wei, H. Zhou, J. Peng, Q. Li, F. Li, J. Zhang, and J. Shi, "Augmented-reality-driven medical simulation platform for percutaneous nephrolithotomy with cybersecurity awareness," International Journal of Distributed Sensor Networks, vol. 15, no. 4, p. 1550147719840173, 2019, doi: 10.1177/1550147719840173.
    [57]B. Thomas, B. Close, J. Donoghue, J. Squires, P. De Bondi, M. Morris, and W. Piekarski, "ARQuake: An outdoor/indoor augmented reality first person application," in Digest of Papers. Fourth International Symposium on Wearable Computers, Atlanta, 2000, pp. 139-146.
    [58]P. Thomas and W. David, "Augmented reality: An application of heads-up display technology to manual manufacturing processes," in Hawaii International Conference on System Sciences, Kauai, 1992, pp. 659-669.
    [59]Y. Tian, G. Yang, Z. Wang, H. Wang, E. Li, and Z. Liang, "Apple detection during different growth stages in orchards using the improved YOLO-V3 model," Computers and Electronics in Agriculture, vol. 157, pp. 417-426, 2019, doi: https://doi.org/10.1016/j.compag.2019.01.012.
    [60]S.-H. Wang, C. Tang, J. Sun, J. Yang, C. Huang, P. Phillips, and Y.-D. Zhang, "Multiple Sclerosis Identification by 14-Layer Convolutional Neural Network With Batch Normalization, Dropout, and Stochastic Pooling," Frontiers in Neuroscience, vol. 12, p. 818, 2018.
    [61]S. C. Wang, J. A. Anderson, R. Evans, K. Woo, B. Beland, D. Sasseville, and L. Moreau, "Point-of-care wound visioning technology: Reproducibility and accuracy of a wound measurement app," PloS One, vol. 12, no. 8, p. e0183139, 2017.
    [62]P. Werbos, "Beyond regression : new tools for prediction and analysis in the behavioral sciences," Ph. D. dissertation, Harvard University, 1974.
    [63]L. Wu, J. Ma, Y. Zhao, and H. Liu, "Apple Detection in Complex Scene Using the Improved YOLOv4 Model," Agronomy, vol. 11, no. 3, p. 476, 2021. [Online]. Available: https://www.mdpi.com/2073-4395/11/3/476.
    [64]W. Wu, K. Yong, M. Federico, and S. Gan, "The APD Skin Monitoring App for wound monitoring: image processing, area plot, and colour histogram," Scientific Phone Apps Mobile Device, vol. 5, p. 3, 2018.
    [65]R. Yamashita, M. Nishio, R. K. G. Do, and K. Togashi, "Convolutional neural networks: an overview and application in radiology," Insights into Imaging, vol. 9, no. 4, pp. 611-629, 2018, doi: 10.1007/s13244-018-0639-9.
    [66]I. T. Young, J. J. Gerbrands, and L. J. Van Vliet, Fundamentals of Image Processing. Delft University of Technology, 1998.

    無法下載圖示
    校外:不公開
    電子論文及紙本論文均尚未授權公開
    QR CODE