| 研究生: |
阮玉勝 Nguyen Ngoc Thang |
|---|---|
| 論文名稱: |
結合多個動作捕捉裝置開發的自動化指關節角度量測系統及驗證於健康受測者手部功能性活動 Development of an Automatically Finger Joints Angle Measurement System using Multiple Motion Capture Devices and its Verification on Healthy Subjects’ Functional Movements |
| 指導教授: |
林哲偉
Lin, Che-Wei |
| 學位類別: |
碩士 Master |
| 系所名稱: |
工學院 - 生物醫學工程學系 Department of BioMedical Engineering |
| 論文出版年: | 2021 |
| 畢業學年度: | 109 |
| 語文別: | 英文 |
| 論文頁數: | 79 |
| 中文關鍵詞: | 多通道的Leap Motion控制器 、關節角度 、運動範圍 、功能性運動 、SVM 、單樣本 t 檢 |
| 外文關鍵詞: | Multiple Leap Motion controllers, Joints angle, Range of Motion, Functional movement, SVM, One-sample t-test |
| 相關次數: | 點閱:178 下載:1 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
中風是全球疾病中併發症發生率高的常見疾病,中風會導致上肢肢體無力,當患者上肢出現問題時將會影響手部動作表現,如:抓握、握持、書寫以及涉及手部的日常活動,生活品質將會隨著疾病發展而逐漸下降。本研究結合多個動作捕捉裝置開發的自動化指關節角度量測系統,提供一種新的方法用來加強監測患者在家中風後康復過程,以及驗證健康受試者功能運動。
自動手指角度測量系統是使用三個 leap motion controller (LMC) 設備開發,用以優化中風患者在復健治療中的手部運動追蹤,系統使用面對面 (face-to-face) 和正交 (orthogonal) 兩種設置方式,藉此克服單個 LMC 系統的缺點,例如:遮擋。此外能偵測到 3 個關節 (遠端指間關節(distal interphalangeal joint, DIP)、近端指間關節 (proximal interphalangeal joint, PIP) 和掌指骨 (metacarpophalangeal, MCP)) 大部分的運動,實驗過程中受測者的任務包含:伸直運動、桌上運動、握拳、鉤指和拇指屈曲/伸展,這些運動可以清晰地觀察每個關節的運動範圍變化。在數據分析中,採用單樣本t檢定,將角度和活動範圍 (range of motion, ROM) 的測量值,與美國手外科學會提供的參考值進行比較,並且使用機器學習中的四分類決策樹 (four classifiers decision tree)、貝氏分類器 (Naive Bayes)、K-近鄰演算法 (k-nearest neighbor, KNN)、支援向量機 (support vector machine, SVM) 對健康受試者的手部運動進行分類。此外,使用特徵選擇 (feature selection) 技術確保分類準確性、優化模型,並減少計算時間及運算成本。所有實驗都使用 5 折交叉驗證 (5-fold cross validation) 來評估機器學習模型。
統計結果分別將每個關節的 ROM 與美國手外科醫學會 (American Society for Surgery of the Hand, ASSH) 的參考值 (DIP = 80、PIP = 100 和 MCP = 90 )進行比較,在 LMC #2 (LMC #2 的控制器直接指向手掌) 中對每個關節使用單樣本 t 檢定時的 p 值大於 0.001 (alpha = 5%),證明 LMC #2 提供的手部運動追蹤結果最接近參考值,這意味著 LMC #2 控制器方向可以提供最佳位置。在關節角度的功能運動分類結果中 SVM 分類器中的準確率最高,單一 LMC 控制器在 LMC #2的結果為 89.15%,在 LMC #1 加上 LMC #2 的雙控制器為 92.48% (LMC #1 的控制器直接指向手背),和三個 LMC 為 93.27%。在ROM 的功能運動分類結果中,SVM 分類器中具有最高的準確率,單一 LMC控制器在 LMC #1可達90.17%,雙 LMC控制器在 LMC #1 和 LMC #3 (LMC #3 控制器位於左側手部) 結果為 95.42%,對於三個 LMC控制器可達 96.33% 的準確率。所有分類器的拇指屈伸分類結果幾乎超過90%,其中SVM的分類結果最好,準確率分別為94.71% (無特徵選擇)、94.80% (NCA (neighborhood component analysis) 特徵選擇)、94.42% (SBS (sequential backward selection) 特徵選擇)、93.23% (SFS (sequential forward selection) 特徵選擇)。
根據本研究的結果顯示,有其必要性去開發一個多通道的Leap Motion控制器系統來測量手指關節角度,以提高分類準確率以及克服單個Leap Motion系統中所存在控制器自遮擋和追踪區域受限的問題。結果還顯示了使用基於ROM特徵的SVM 分類器將比其他基於關節角度的分類器帶來更高的效率。此系統具有替代傳統工具 (如測角儀) 和大型系統 (如運動捕捉系統) 的巨大潛力,以在家中監測中風後患者的康復程度。
Stroke is a common disease with a high complication rate worldwide. Stroke can cause strength loss of the upper extremities. A patient who experiences a problem in the upper extremities gets difficulties in grasping, holding, writing as well as daily activities involving the hands. Consequently, their quality of life can decrease over time. This research develops an automatically finger joints angle measurement system by using multiple motion capture devices, aims to provide a new approach to enhance the monitoring of patient’s rehabilitation progress after stroke at home as well as its verification on healthy subjects' functional movement.
This automatic finger angle measurement system was developed using three LMCs devices to optimize the tracking hand movement range for the post-stroke patient's rehabilitation tasks. Using two setups, face-to-face and orthogonal, this system can overcome the disadvantages of a single LMC system such as self-occlusion, cover almost all movements of 3 joints DIP, PIP, and MCP. The tasks for the experimenters were Straight, Tabletop, Full fist, Hook, and Thumb flexion/extension. These movements ensure the clearest observation of the change in range of motion at each joint. In the data analysis, a one-sample t-test was used to compare the measured value of angle and ROM with the reference value provided by the American Society for Surgery of the Hand. Next, Machine Learning with four classifiers Decision Tree, Naive Bayes, KNN, SVM is used to classify motions obtained on healthy subjects. In addition, feature selection techniques are used to optimize the model by reducing computation time and cost while maintaining classification accuracy. All experiments used 5-fold cross validation to evaluate machine learning models.
The statistical result compares the ROM of each joint with ASSH reference values of DIP = 80, PIP = 100, and MCP = 90, respectively. A p-value when using one-sample t-test of each joint in LMC #2 (LMC #2's sensor points directly to the palm of the hand) is greater than 0.001 (alpha = 5%) demonstrate that LMC #2 gives the most approximate tracking hand movement results to the reference value. That means LMC #2 has the most optimal placement. The classification results of functional movement based on joint angle have the highest accuracy for SVM classifiers with results of 89.15% at LMC #2 for single LMC, 92.48% at LMC #1 (LMC #1's sensor points directly to the back of the hand) and LMC #2 for dual LMCs, and 93.27% for three LMCs, respectively. The classification results of functional movement based on ROM have the highest accuracy for SVM classifiers with results of 90.17% at LMC #1 for single LMC, 95.42% at LMC #1 and LMC #3 (LMC #3’s sensor in the left side of the hand) for dual LMCs, and 96.33% for three LMCs, respectively. The classification results of thumb flexion/extension of all classifiers almost over 90% in which SVM gives the best classification result with the accuracy are 94.71%, 94.80%, 94.42%, 93.23% respectively for without feature selection, NCA, SBS and SFS feature selection method.
For the results obtained in this research, it is necessary to develop a multiple Leap Motion controller system to measure the finger joints angle to increase the classification accuracy as well as to overcome the existing problems in the single Leap Motion system. controller as self-occlusion and tracking area limitation. The results also show that using SVM classifier based on ROM features will bring greater efficiency than other classifiers based on joints angle. This system has great potential to replace traditional tools such as goniometers and large systems such as motion capture systems to monitor recovery progress for post-stroke patients at home.
[1] A. H. Association. "2021 Heart Disease & Stroke Statistical Update Fact Sheet Global Burden of Disease." [Online]. Available: https://bitly.com.vn/o9raxa. [Accessed: 2021].
[2] M. Clinic. "Stroke." [Online]. Available: https://bitly.com.vn/rsiqr0. [Accessed: Sep.15, 2021].
[3] L. Hui-chin. "Deaths from strokes rising: MOHW." [Online]. Available: https://bitly.com.vn/rfq2o7. [Accessed: Apr. 06, 2021].
[4] N. I. o. N. D. a. Stroke. "Post-Stroke Rehabilitation Fact Sheet." [Online]. Available: https://bitly.com.vn/yrsu76. [Accessed: Sep.15, 2021].
[5] H. B. Bosworth, "How can innovative uses of technology be harnessed to improve medication adherence?," Expert Review of Pharmacoeconomics & Outcomes Research, vol. 12, no. 2, pp. 133-135, 2012/04/01 2012.
[6] Elemental. "VR May Change the Future of Therapy." [Online]. Available: https://bitly.com.vn/twf1l5. [Accessed: Oct.23, 2019].
[7] L. Motion. "5 Medical and Assistive Technologies Being Transformed with Leap Motion." [Online]. Available: https://bitly.com.vn/a3ksb0. [Accessed: Aug.29, 2015].
[8] Z. R. Wang, P. Wang, L. Xing, L. P. Mei, J. Zhao, and T. Zhang, "Leap Motion-based virtual reality training for improving motor functional recovery of upper limbs and neural reorganization in subacute stroke patients," (in eng), Neural Regen Res, vol. 12, no. 11, pp. 1823-1831, Nov 2017.
[9] R. L. Gajdosik and R. W. Bohannon, "Clinical measurement of range of motion. Review of goniometry emphasizing reliability and validity," (in eng), Phys Ther, vol. 67, no. 12, pp. 1867-72, Dec 1987.
[10] K. Nizamis, N. H. M. Rijken, A. Mendes, M. Janssen, A. Bergsma, and B. Koopman, "A Novel Setup and Protocol to Measure the Range of Motion of the Wrist and the Hand," Sensors (Basel), vol. 18, no. 10, Sep 25 2018.
[11] R. M. F. de Carvalho, N. Mazzer, and C. H. Barbieri, "Analysis of the reliability and reproducibility of goniometry compared to hand photogrammetry," (in eng), Acta Ortop Bras, vol. 20, no. 3, pp. 139-149, 2012.
[12] E. Lewis, L. Fors, and W. Tharion, "Interrater and Intrarater Reliability of Finger Goniometric Measurements," The American journal of occupational therapy : official publication of the American Occupational Therapy Association, vol. 64, pp. 555-61, 07/06 2010.
[13] R. D. Lea and J. J. Gerhardt, "Range-of-motion measurements," (in eng), J Bone Joint Surg Am, vol. 77, no. 5, pp. 784-98, May 1995,.
[14] G. S. Rash, P. P. Belliappa, M. P. Wachowiak, N. N. Somia, and A. Gupta, "A demonstration of validity of 3-D video motion analysis method for measuring finger flexion and extension," J Biomech, vol. 32, no. 12, pp. 1337-41, Dec 1999.
[15] R. B. Davis, S. Õunpuu, D. Tyburski, and J. R. Gage, "A gait analysis data collection and reduction technique," Human Movement Science, vol. 10, no. 5, pp. 575-587, 1991/10/01/ 1991.
[16] H. Y. Chiu, S. C. Lin, F. C. Su, S. T. Wang, and H. Y. Hsu, "The use of the motion analysis system for evaluation of loss of movement in the finger," J Hand Surg Br, vol. 25, no. 2, pp. 195-9, Apr 2000.
[17] A. Cappozzo, U. Della Croce, A. Leardini, and L. Chiari, "Human movement analysis using stereophotogrammetry. Part 1: theoretical background," Gait Posture, vol. 21, no. 2, pp. 186-96, Feb 2005.
[18] K. S. Lee and M. C. Jung, "Ergonomic evaluation of biomechanical hand function," Saf Health Work, vol. 6, no. 1, pp. 9-17, Mar 2015.
[19] L. Reissner, G. Fischer, R. List, W. R. Taylor, P. Giovanoli, and M. Calcagni, "Minimal detectable difference of the finger and wrist range of motion: comparison of goniometry and 3D motion analysis," Journal of Orthopaedic Surgery and Research, vol. 14, no. 1, p. 173, 2019/06/10 2019.
[20] Y. Zhu, W. Lu, W. Gan, and W. Hou, "A contactless method to measure real-time finger motion using depth-based pose estimation," Comput Biol Med, vol. 131, p. 104282, Apr 2021.
[21] A. H. Butt, E. Rovini, C. Dolciotti, G. De Petris, P. Bongioanni, M. C. Carboncini, and F. Cavallo, “Objective and automatic classification of parkinson disease with Leap Motion Controller,” BioMedical Engineering OnLine, vol. 17, no. 1, 2018.
[22] M. Cohen and D. Regazzoni, "Hand rehabilitation assessment system using leap motion controller," AI & SOCIETY, vol. 35, 09/01 2020.
[23] Y.-T. Wu, K.-H. Chen, S.-L. Ban, K.-Y. Tung, and L.-R. Chen, "Evaluation of leap motion control for hand rehabilitation in burn patients: An experience in the dust explosion disaster in Formosa Fun Coast," Burns, vol. 45, no. 1, pp. 157-164, 2019/02/01/ 2019.
[24] A. Houston, V. Walters, T. Corbett, and R. Coppack, "Evaluation of a multi-sensor Leap Motion setup for biomechanical motion capture of the hand," Journal of Biomechanics, vol. 127, p. 110713, 2021/10/11/ 2021.
[25] H. Jin, Q. Chen, Z. Chen, Y. Hu, and J. Zhang, "Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task," CAAI Transactions on Intelligence Technology, vol. 1, no. 1, pp. 104-113, 2016/01/01/ 2016.
[26] Y. Wang, Y. Wu, S. Jung, S. Hoermann, S. Yao, and R. W. Lindeman, "Enlarging the Usable Hand Tracking Area by Using Multiple Leap Motion Controllers in VR," IEEE Sensors Journal, vol. 21, no. 16, pp. 17947-17961, 2021.
[27] F. Zhou, R. Yin, G. Trajcevski, K. Zhang, J. Wu, and A. Khokhar, "Improving human mobility identification with trajectory augmentation," GeoInformatica, vol. 25, no. 3, pp. 453-483, 2021/07/01 2021.
[28] A. Moshkova, A. Samorodov, N. Voinova, A. Volkov, E. Ivanova, and E. Fedotova, "Parkinson’s Disease Detection by Using Machine Learning Algorithms and Hand Movement Signal from LeapMotion Sensor," in 2020 26th Conference of Open Innovations Association (FRUCT), 20-24 April 2020 2020, pp. 321-327, doi: 10.23919/FRUCT48808.2020.9087433.
[29] M. W. Cohen, I. Voldman, D. Regazzoni, and A. Vitali, "Hand Rehabilitation via Gesture Recognition Using Leap Motion Controller," in 2018 11th International Conference on Human System Interaction (HSI), 4-6 July 2018 2018, pp. 404-410.
[30] K. Morita, P. Chan, M. Nii, N. Nakagawa, and S. Kobashi, "Finger Joint Detection Method for the Automatic Estimation of Rheumatoid Arthritis Progression Using Machine Learning," in 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 7-10 Oct. 2018 2018, pp. 1315-1320.
[31] K. Anam and A. Al-Jumaily, "A robust myoelectric pattern recognition using online sequential extreme learning machine for finger movement classification," in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 25-29 Aug. 2015 2015, pp. 7266-7269.
[32] R. Nogales and M. E. Benalcázar, "Real-Time Hand Gesture Recognition Using the Leap Motion Controller and Machine Learning," 2019 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pp. 1-7, 2019.
[33] A. Vysocky et al., "Analysis of Precision and Stability of Hand Tracking with Leap Motion Sensor," Sensors (Basel), vol. 20, no. 15, Jul 22 2020.
[34] S. Okazaki, Y. Muraoka, and R. Suzuki, "Validity and reliability of Leap Motion Controller for assessing grasping and releasing finger movements," Journal of ergonomic technology, vol. 17, p. 2017, 09/23 2017.
[35] Y. Kim, P. C. W. Kim, R. Selle, A. Shademan, and A. Krieger, "Experimental evaluation of contact-less hand tracking systems for tele-operation of surgical tasks," in 2014 IEEE International Conference on Robotics and Automation (ICRA), 31 May-7 June 2014 2014, pp. 3502-3509.
[36] Ultraleap. "Experimental Release #2: Multiple Device Support." [Online]. Available: https://blog.leapmotion.com/multiple-devices/. [Accessed.
[37] M. C. B. Seixas, J. C. S. Cardoso, and M. T. G. Dias, "The Leap Motion movement for 2D pointing tasks: Characterisation and comparison to other devices," in 2015 International Conference on Pervasive and Embedded Computing and Communication Systems (PECCS), 11-13 Feb. 2015 2015, pp. 15-24.
[38] Ultraleap. "Leap Motion Controller." [Online]. Available: https://bitly.com.vn/14zp9x. [Accessed: Sep. 08, 2021].
[39] A. H. Smeragliuolo, N. J. Hill, L. Disla, and D. Putrino, "Validation of the Leap Motion Controller using markered motion capture technology," Journal of Biomechanics, vol. 49, no. 9, pp. 1742-1750, 2016/06/14/ 2016.
[40] Intel. "Intel® NUC 10 Performance kit - NUC10i3FNH." [Online]. Available: https://bitly.com.vn/ty0076. [Accessed: Sep. 08, 2021].
[41] L. L. Cheng and H. B. Liu, "Examples of quadrocopter control on ROS," in 2015 IEEE 9th International Conference on Anti-counterfeiting, Security, and Identification (ASID), 25-27 Sept. 2015 2015, pp. 92-96.
[42] A. Sarkar, K. A. Patel, R. K. G. Ram, and G. K. Capoor, "Gesture control of drone using a motion controller," in 2016 International Conference on Industrial Informatics and Computer Systems (CIICS), 13-15 March 2016 2016, pp. 1-5.
[43] R. A. S. Fernández, J. L. Sanchez-Lopez, C. Sampedro, H. Bavle, M. Molina, and P. Campoy, "Natural user interfaces for human-drone multi-modal interaction," in 2016 International Conference on Unmanned Aircraft Systems (ICUAS), 7-10 June 2016 2016, pp. 1013-1022.
[44] R. W. Partridge, F. S. Brown, P. M. Brennan, I. A. Hennessey, and M. A. Hughes, "The LEAP™ Gesture Interface Device and Take-Home Laparoscopic Simulators: A Study of Construct and Concurrent Validity," (in eng), Surg Innov, vol. 23, no. 1, pp. 70-7, Feb 2016.
[45] M. Carrieri et al., "Prefrontal Cortex Activation Upon a Demanding Virtual Hand-Controlled Task: A New Frontier for Neuroergonomics," (in eng), Front Hum Neurosci, vol. 10, p. 53, 2016.
[46] C. Naidu and A. Ghotkar, "Hand Gesture Recognition Using Leap Motion Controller," 2016.
[47] ElectroPeak. "CP2102 USB to TTL Uart Serial Convertor Module." [Online]. Available: https://bitly.com.vn/mqgulg. [Accessed: Sep.08, 2021].
[48] S. Laboratories. "SINGLE-CHIP USB TO UART BRIDGE." [Online]. Available: https://bitly.com.vn/5l5hxb. [Accessed: Sep.08, 2021].
[49] C. Corner. "What Can C# Do For You." [Online]. Available: https://bitly.com.vn/fs6mhg. [Accessed: Sep.11, 2021].
[50] MathWorks. "What is MATLAB." [Online]. Available: https://bitly.com.vn/ezlppa. [Accessed: Sep.08, 2021].
[51] IBM. "IBM SPSS software." [Online]. Available: https://bitly.com.vn/3k5tpc. [Accessed: Sep.08, 2021].
[52] JMP. "Statistics Knowledge Portal." [Online]. Available: https://bitly.com.vn/pt4u6o. [Accessed: Sep.08, 2021].
[53] L. Barhum. "The Anatomy of the Finger Joints." [Online]. Available: https://bitly.com.vn/vi5zma. [Accessed: May.06, 2021].
[54] Handcare. "Joints." [Online]. Available: https://bitly.com.vn/6h04sy. [Accessed: Sep.08, 2021].
[55] J. C. Segen, Concise Dictionary of Modern Medicine. McGraw-Hill, 2006.
[56] Physiopedia. "Range of Motion." [Online]. Available: https://bitly.com.vn/qwhvbw. [Accessed: Sep.08, 2021].
[57] K. S. University. "SPSS Tutorials: One Sample T-Test." [Online]. Available: https://libguides.library.kent.edu/SPSS/OneSampletTest. [Accessed: 10, 2021].
[58] A. S. f. S. o. t. Hand. [Online]. Available: https://bitly.com.vn/yvilvh. [Accessed: Sep, 29, 2021].
[59] A. Marcano-Cedeño, J. Quintanilla-Domínguez, M. G. Cortina-Januchs, and D. Andina, "Feature selection using Sequential Forward Selection and classification applying Artificial Metaplasticity Neural Network," in IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society, 7-10 Nov. 2010 2010, pp. 2845-2850.
[60] L. Burrell, O. Smart, G. Georgoulas, E. Marsh, and G. Vachtsevanos, Evaluation of Feature Selection Techniques for Analysis of Functional MRI and EEG. 2007, pp. 256-262.
[61] W. Yang, K. Wang, and W. Zuo, "Neighborhood Component Feature Selection for High-Dimensional Data," JCP, vol. 7, pp. 161-168, 01/01 2012.
[62] J. A. Fairley, G. Georgoulas, and G. Vachtsevanos, "Sequential feature selection methods for Parkinsonian human sleep analysis," 2009 17th Mediterranean Conference on Control and Automation, pp. 1468-1473, 2009.
[63] O. Yaman, "An automated faults classification method based on binary pattern and neighborhood component analysis using induction motor," Measurements, vol. 168, p. 108323, January 01, 2021 2021.
[64] V. Podgorelec, P. Kokol, B. Stiglic, and I. Rozman, "Decision Trees: An Overview and Their Use in Medicine," Journal of Medical Systems, vol. 26, no. 5, pp. 445-463, 2002/10/01 2002.
[65] O. Yaman, H. Yetis, and M. Karakose, "Decision Tree Based Customer Analysis Method for Energy Planning in Smart Cities," in 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), 2020: IEEE, pp. 1-4.
[66] Y. Freund and L. Mason, "The alternating decision tree learning algorithm," in icml, 1999, vol. 99: Citeseer, pp. 124-133.
[67] K. Lee, H. O. Choi, S. D. Min, J. Lee, B. B. Gupta, and Y. Nam, "A Comparative Evaluation of Atrial Fibrillation Detection Methods in Koreans Based on Optical Recordings Using a Smartphone," IEEE Access, vol. 5, pp. 11437-11443, 2017.
[68] Z. Zhang, "Introduction to machine learning: k-nearest neighbors," Annals of translational medicine, vol. 4, no. 11, 2016.
[69] N. S. Altman, "An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression," The American Statistician, vol. 46, no. 3, pp. 175-185, 1992.
[70] R. K. Begg, M. Palaniswami, and B. Owen, "Support vector machines for automated gait classification," IEEE Transactions on Biomedical Engineering, vol. 52, no. 5, pp. 828-838, 2005.
[71] T. Nakano et al., "Gaits classification of normal vs. patients by wireless gait sensor and Support Vector Machine (SVM) classifier," in 2016 IEEE/ACIS 15th International Conference on Computer and Information Science (ICIS), 26-29 June 2016 2016, pp. 1-6.
[72] A. Widodo and B.-S. Yang, "Support vector machine in machine condition monitoring and fault diagnosis," Mechanical systems and signal processing, vol. 21, no. 6, pp. 2560-2574, 2007.
[73] S. Okazaki, Y. Muraoka, and R. Suzuki, "Validity and reliability of Leap Motion Controller for assessing grasping and releasing finger movements," Journal of ergonomic technology, vol. 17, p. 2017, 09/23 2017.