中風是造成腦部損傷的主要病症之一，而偏癱則是一種常見的中風後遺症。近年來，研究發現鏡像治療能改善輕偏癱患者的上肢動作功能。在傳統的鏡像治療中，患者藉由健側手執行復健動作、並透過鏡箱反射健側的影像於患側的位置、患側放置於鏡箱中，患者在視覺上會感覺患側似乎也能活動，進而達到修復腦中鏡像神經元的效果。然而傳統的鏡像治療方式礙於鏡箱尺寸，患者的活動範圍也因此受限。除此之外，鏡箱與實際景物混合出現在患者視野範圍中，也可能讓使用者從鏡像治療感受到的鏡像影像回饋真實感下降。為了解決傳統鏡像治療動作大小受限及真實度不足的問題。本論文開發一套基於鏡像治療理論應用於中風偏癱患者復健的虛擬實境系統。本論文的虛擬實境治療系統的硬體系統由動作捕捉感測器(Leap Motion Controller, LMC)及虛擬實境頭盔(Oculus Rift)所組成。LMC為一種基於影像感測器搭配人工智慧演算法的動作捕捉感測器，此裝置於臨床上使用最大的優點為不需要於使用者身上配戴任何的標記點(Marker)，LMC就可達到即時分析並產生22個手指及掌關節的座標的功效，本研究透過LMC捕捉使用者的健側手指/手掌關節座標；接著透過演算法的運算及映射，產生患側手在鏡像中對應的手指/手掌關節座標，最後透過使用Oculus Rift虛擬實境頭盔即時生成使用者健側手及患側手的座標，使用者於虛擬實境頭盔中直接感受到虛擬實境頭盔的鏡像治療效果。本論文將「基於鏡像治療理論應用於中風偏癱患者復健的虛擬實境系統」應用於臨床實際驗證，本研究招募30位健康人並隨機分為兩組，一組先接受傳統鏡像介入、另一組先接受虛擬實境鏡像介入。在兩週之後，兩組接受的介入互換。本研究使用了抓握提舉(Pinch-holding up activity, PHUA)測試、Semmes-Weinstein 單纖維(Semmes-Weinstein Monofilament, SWM)測試、明尼蘇達手部靈巧度測驗(Minnesota Hand dexterity test, MMDT)及普渡手功能測驗測試(Purdue Pegboard test, PPT)來評估受測者接受虛擬鏡像治療及傳統鏡像治療的成效，研究結果顯示在明尼蘇達手部靈巧度測驗及普渡手功能測驗測試中，不論是虛擬實境鏡像治療或傳統鏡像治療都可以提升使用者的手部靈巧度（統計上有顯著差異）。而虛擬實境鏡像治療在抓握提舉(PHUA)測試方面較傳統鏡像治療有更佳的效果（統計上有顯著差異）。
在虛擬實境鏡像治療中，本研究發現LMC帶來一個附加價值：可以連續記錄受測者在鏡像治療復健過程中關節移動的軌跡，關節移動的軌跡或許可作為進一步復健成效的評估基礎。為了進一步的驗證LMC在手指/手掌關節的精準度，本研究使用生物力學研究中標準的動作捕捉感測設備Motion Lab：以光學反射原理並配戴於軀幹關節上的標記點(Marker)來進行手指/手掌關節的移動軌跡精準度比較驗證。驗證分為兩個部分，第一部分為沿著動作捕捉感測器的單一軸向，移動真手或假手來進行規律的運動。第二部分則包含常見的復健動作如手掌開闔、拇指繞圈等動作，本研究收集了20位受測者的數據，若比較LMC與Motion Lab的關節軌跡，LMC所量測到的關節軌跡並無法完全與Motion Lab關節軌跡相符。因此本研究進一步使用機器學習的迴歸模型用於校正LMC所量測到的軌跡、希望LMC所量測到的關節移動軌跡可以接近Motion Lab所量測到的關節移動軌跡。本研究將LMC的各個關節在空間中三個維度的位置數據作為迴歸模型的輸入(Input)、Motion Lab的各個關節在空間中三個維度的位置數據作為迴歸模型的理想輸出(Desired output)。本研究測試了19種常見的迴歸模型演算法，經過初步測試，高斯迴歸模型(Gaussian regression model)能獲得最佳之結果，因此本研究採用高斯迴歸模型來進行進一步的研究探討。本研究進一步的基於高斯迴歸模型將輸入以現下時間中以及過去一段時間LMC各個維度的三維的位置數據進行迴歸，以達到更好的結果。本研究數據顯示若能以現下時間以及過去20毫秒的軌跡共同進行迴歸，LMC經迴歸修正後的軌跡可達到平均小於10毫米的誤差（與Motion Lab系統所得到的軌跡比較）。基於上述說明，本研究成功的開發一套虛擬實境鏡像治療系統，在明尼蘇達手部靈巧度測驗及普渡手功能測驗測試中，虛擬實境鏡像治療系統不但有與傳統鏡像治療一致的功效，更發現在抓握提舉測試方面上、虛擬實境鏡像治療系統較傳統鏡像治療更有效果。進一步透過機器學習的迴歸演算法，可讓LMC此種低價且無須黏貼標記點(Marker)於患者身上的動作捕捉系統能有更高的精確度。本研究認為LMC不但可用於虛擬實境相關的復健應用，LMC所量測到的軌跡搭配機器學習的迴歸，可得到更精準的關節移動軌跡，或許也可成為另一種評估復健成效的指標。

Stroke is one of the main causes of brain damage, and hemiplegia is a common sequela after stroke. In recent years, plenty of studies have found that mirror therapy can improve the upper limb movement function of patients with hemiparesis. In the traditional mirror therapy treatment, the patient achieves the effect of repairing the mirror neurons in the brain by observing the image of the healthy side’s reflection in the mirror box, which may look like their affected side can move again. However, traditional mirror therapy is limited by the size of the mirror box. Hence, the patient’s perspective of motion is also limited. At the same time, when a mirror box exists in the actual scene and appears in the patient’s perspective, it may also cause the user to experience a decrease in the sense of reality from the mirror image perceived by the mirror therapy. To solve the problem of the movement is limited by size and lack of realism in traditional mirror therapy — this thesis aimed at developing a mirror therapy-based virtual reality system for rehabilitation for stroke hemiplegia patient. The hardware system of the proposed virtual reality therapy system consists of a Leap Motion Controller (LMC) and a virtual reality headset (Oculus Rift). LMC is a camera-based motion capture sensor with an artificial intelligence algorithm. The advantage of using LMC in the clinical scenario is that it does not need to wear any marker on the user, but still can instantly analyze and generate coordinates of 22 markers of fingers and palm. The proposed system captures the movement trajectories of the user’s finger/palm joint in healthy side by using LMC. The corresponding finger/palm joint coordinates of the affected hand are generated in the mirror by the self-developed algorithm. Finally, the Oculus Rift virtual reality headset is used to display the coordinates of the healthy side and the affected side instantly. The user can directly feel the mirror therapy effect of virtual reality therapy in the virtual reality headset.
To validate the effectiveness of the proposed system, this thesis applies the “development of a mirror therapy-based virtual reality system for rehabilitation of stroke hemiplegia patient” to clinical trials. Thirty healthy subjects were recruited and were divided into two groups. One group received traditional mirror therapy intervention, and the other group received virtual reality mirror therapy intervention. After two weeks of washout time, two groups will receive interchange intervention. This study used the pinch-holding up activity (PHUA) test, the Semmes-Weinstein Monofilament (SWM) test, the Minnesota Hand dexterity test (MMDT) and the Purdue Pegboard test (PPT) to assess the subject’s improvement after the intervention of the virtual mirror therapy and the traditional mirror therapy. The results of the study show that in the Minnesota hand dexterity test (MMDT) and the Purdue pegboard test (PPT), both virtual reality and traditional mirror therapy intervention can improve the user’s hand dexterity (significant differences in statistical analysis). The virtual reality mirror therapy is better than that of the traditional mirror therapy in terms of PHUA test (significant differences in statistical analysis). Therefore, it can be proved that the virtual reality mirror therapy not only has the advantages of the traditional reality mirror therapy in MMDT and PPT, but also has a better effect on the PHUA test.
In the virtual reality mirror therapy, this study found that LMC brings an added value: it can continuously record the trajectory of the joints’ movement of the subject during the rehabilitation process, and the trajectory of joint movement may be able to use as the basis for further evaluation of rehabilitation effectiveness. To further verify the accuracy of the LMC in the finger/palm joint, this study used the standard motion capture system Motion Lab in biomechanical research: use the optical reflective markers which used to wear on the torso joints to verify the accuracy of the movement of the finger/palm joints. The verification is divided into two parts; the first part is to move the real hand or the fake hand along the single axis of the system for regular movement. The second part contains common rehabilitation movements such as palm open and close, thumb circling, and other movements. So far, 20 subjects’ activities have been collected. If we compared the joint trajectories of LMC with Motion Lab, the joint trajectories measured by LMC do not fully match the Motion Lab joint trajectories. Therefore, this study further uses a machine learning regression model to calibrate the trajectories measured by LMC. In this study, the position data in the three-dimensional space of each joint of LMC in space is used as the input of the regression model, and the position data in the three-dimensional space of each joint of Motion Lab as the desired output of the regression model. This study tested 19 common regression model algorithms, and the Gaussian regression model obtained the best results. Therefore, this study used a Gaussian regression model for further research.
Based on the Gaussian regression model, this research further discovers that better results can be achieved by regressing the three-dimensional position data of each dimension of the LMC in the current time and a period in the past. The data of this study show that if the regression can be performed together with the current time and the trajectory of the past 20 milliseconds, the corrected trajectory of the LMC can reach an error of less than 10 mm in average (compared with the trajectory obtained by the Motion Lab system). Based on the above description, this study successfully developed a virtual reality mirror therapy system. Based on the clinical trial in the healthy subjects, it is found that the proposed virtual reality mirror therapy system can have the same effectiveness on MMDT and Purdue Pegboard test. Furthermore, the virtual reality mirror therapy system is more effective than the traditional mirror therapy in the PHUA test. Through the machine learning regression algorithm, LMC can achieve higher accuracy with the low-cost motion capture system (LMC) that does not need to stick Marker on the patient. The trajectories of joints during rehabilitation activities may be potential to become a new index to evaluate the rehabilitation effectiveness in the future in the clinical scenario with the aid of LMC.

[1] H.-H. Hu et al., “Incidence of stroke in Taiwan,” Stroke, vol. 23, pp. 1237-1241, 1992.
[2] J. Broderick et al., “The Greater Cincinnati/Northern Kentucky Stroke Study: Preliminary First-Ever and Total Incidence Rates of Stroke Among Blacks.,” Stroke, vol. 29, pp. 415-421, 1998.
[3] H.-H. Hu et al., “Prevalence of stroke in Taiwan,” Stroke, vol. 20, pp. 858-63, 1989.
[4] J. P. Casus et al., “Homocysteine and stroke: evidence on a causal link from mendelian readomisation,” Lancet, vol. 365, pp. 224-32, 2005.
[5] V. L. Feigin et al., “Stroke epidemiology: a review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century,” Lancet Neurology, vol. 2, pp. 43-53, 2003.
[6] E. Sadler et al., “Developing a novel peer support intervention to promote resilience after stroke, ” Health & social care in the community, vol. 25, no. 5, pp. 1590-1600, 2017.
[7] 邱弘毅, “亞急性暨長期照護之現況與未來～以中風為例,” Formosan Journal of Stroke, vol.22, no. 3, pp.4-7, 2018.
[8] H.-P. Wang, C.-C. Yang, and H.-C. Hsu, “Exploring Resilience and Related Factors Among Patients With Stroke in the Recovery Stage,” The Journal of Nursing, vol. 65, no. 5, pp. 56-67, 2018.
[9] D. R.-R. V. S. Ramachandran, and S. Cobb, “Touching the phantom limb,” Nature, vol. 3777, no. 6549, pp. 489-490, 1995.
[10] V. S. R. a. D. Rogers-Ramachandran, “Synaesthesia in phantom limbs induced with mirrors,” Proceedings of the Royal Society of London, vol. 263, pp. 377-386, 1996.
[11] E. L. A. et. al., “Rehabilitation of hemiparesis after stroke with a mirror,” Lancet, vol. 353, no. 9169, pp. 2035-2036, 1999.
[12] F. Steinberg, N. H. Pixa, and M. Doppelmayr, “Mirror visual feedback training improves intermanual transfer in a sport-specific task: A comparison between different skill levels, ” Neural plasticity, vol. 2016, pp. 1-11, 2016.
[13] J. K. J. P. Hunter, and K. D. Davis, “The effect of tactile and visual sensory inputs on phantom limb awareness,” Brain, vol. 126, no. 3, pp. 579-589, 2003.
[14] C. C. M. Diers, C. Koeppe, M. Ruf, and H. Flor, “Mirrored, imagined, and executed movements differentially activate sensorimotor cortex in amputees with and without phantom limb pain,” Pain medicine, vol. 149, no.2, pp. 296-304, 2010.
[15] S. Saleh, S. V. Adamovich, and E. Tunik, “Mirrored feedback in chronic stroke: recruitment and effective connectivity of ipsilesional sensorimotor networks, ” Neurorehabilitation and neural repair, vol. 28, no. 4, pp. 344-354, 2014.
[16] N. Darbois, A. Guillaud, and N. Pinsault, “Do Robotics and Virtual Reality Add Real Progress to Mirror Therapy Rehabilitation? A Scoping Review, ” Rehabilitation research and practice, vol. 2018, pp. 1-15, 2018.
[17] D. Perez-Marcos, M. Bieler-Aeschlimann, and A. Serino, “Virtual Reality as a Vehicle to Empower Motor-Cognitive Neurorehabilitation, ” Frontiers in psychology, vol. 9, pp. 1-8, 2018.
[18] J. Crosbie, S. Lennon, M. McGoldrick, M. McNeill, and S. McDonough, “Virtual reality in the rehabilitation of the arm after hemiplegic stroke: a randomized controlled pilot study,” Clinical Rehabilitation, vol. 26, no. 9, pp. 798-806, 2012.
[19] G. Castelnuovo, C. L. Priore, D. Liccione, and G. Cioffi, “Virtual Reality based tools for the rehabilitation of cognitive and executive functions: the V-STORE,” PsychNology Journal, vol. 1, no. 3, pp. 310-325, 2003.
[20] K. Eng et al., “Interactive visuo-motor therapy system for stroke rehabilitation, ” Medical & biological engineering & computing, vol. 45, no. 9, pp. 901-907, 2007.
[21] T. Guzsvinecz, V. Szucs, and C. Sik-Lanyi, “Suitability of the Kinect Sensor and Leap Motion Controller-A Literature Review,” Sensors, vol. 19, no. 5, p. 1072, 2019.
[22] E. Niechwiej-Szwedo, D. Gonzalez, M. Nouredanesh, and J. Tung, “Evaluation of the Leap Motion Controller during the performance of visually-guided upper limb movements,” PloS one, vol. 13, no. 3, March. 12, 2018. [Online]. Available: https://doi.org/10.1371/journal.pone.0193639. [Accessed July 13, 2019].
[23] F. Weichert, D. Bachmann, B. Rudak, and D. Fisseler, “Analysis of the accuracy and robustness of the leap motion controller,” Sensors, vol. 13, no. 5, pp. 6380-6393, 2013.
[24] Leap Motion, “Leap Motion for Mac and PC,” Leap Motion, [Online]. Available: https://www.leapmotion.com/product/desktop-controller/ [Accessed July 13, 2019].
[25] Leap Motion, “Gestures” Leap Motion, [Online]. Available: https://developer-archive.leapmotion.com/documentation/csharp/devguide/Leap_Gestures.html [Accessed July 13, 2019].
[26] R. Narain, R. A. Albert, A. Bulbul, G. J. Ward, M. S. Banks, and J. F. O'Brien, “Optimal presentation of imagery with focus cues on multi-plane displays, ” ACM Transactions on Graphics (TOG), vol. 34, no. 4, pp. 59:1-59:12, 2015.
[27] N. Matsuda, A. Fix, and D. Lanman, “Focal surface displays, ” ACM Transactions on Graphics (TOG), vol. 36, no. 4, pp. 86:1-86:14, 2017.
[28] C. E. Rasmussen and C. Williams, Gaussian Processes for Machine Learning. MA: The MIT Press, 2006.
[29] C. J. Kincaid, “The Feasibility of Using a Markerless Motion Capture Sensor (Leap Motion TM Controller) for Quantitative Motor Assessment Intended for a Clinical Setting,” 2016.
[30] B. Henriksen, R. Nielsen, L. Szabo, N. Evers, M. Kraus, and B. Geng, “An affordable virtual reality system for treatment of phantom limb pain,” in Proceedings of the 2016 Virtual Reality International Conference, Laval, France, 2016
[31] C. Mercier and A. Sirigu, “Training with virtual visual feedback to alleviate phantom limb pain,” Neurorehabilitation and neural repair, vol. 23, no. 6, pp. 587-594, 2009.
[32] C. D. Murray et al., “The treatment of phantom limb pain using immersive virtual reality: three case studies,” Disability and rehabilitation, vol. 29, no. 18, pp. 1465-1469, 2007.
[33] M. Ortiz-Catalan, N. Sander, M. B. Kristoffersen, B. Håkansson, and R. Brånemark, “Treatment of phantom limb pain (PLP) based on augmented reality and gaming controlled by myoelectric pattern recognition: a case study of a chronic PLP patient,” Frontiers in neuroscience, vol. 8, p. 24, 2014.
[34] T. Massetti et al., “The Clinical Utility of Virtual Reality in Neurorehabilitation: A Systematic Review,” Journal of Central Nervous System Disease, vol. 10, Nov, 27, 2018. [Online]. Available: https://doi.org/10.1177/1179573518813541. [Accessed July 13, 2019].
[35] G. C. Burdea, “Virtual rehabilitation-benefits and challenges,” Methods of information in medicine, vol. 42, no. 05, pp. 519-523, 2003.
[36] F. Cordella, F. Di Corato, L. Zollo, B. Siciliano, and P. van der Smagt, “Patient performance evaluation using Kinect and Monte Carlo-based finger tracking,” in Biomedical Robotics and Biomechatronics (BioRob), 2012 4th IEEE RAS & EMBS International Conference on, pp. 1967-1972, 2012.
[37] B. Henriksen, R. Nielsen, M. Kraus, and B. Geng, “Comparison of movements in a virtual reality mirror box therapy for treatment of lower limb phantom iain," in Proc. of VISIGRAPP (1: GRAPP), pp. 167-174, 2018.
[38] M. Carrozzo and F. Lacquaniti, “Virtual reality: a tutorial,” Electroencephalography and Clinical Neurophysiology/Electromyography and Motor Control, vol. 109, no. 1, pp. 1-9, 1998.
[39] J. A. Lozano et al., “VR-mirror: A virtual reality system for mental practice in post-stroke rehabilitation,” in Proc. of International Symposium on Smart Graphics, pp. 241-251, 2005.
[40] K. Sato et al., “Nonimmersive virtual reality mirror visual feedback therapy and its application for the treatment of complex regional pain syndrome: an open-label pilot study,” Pain medicine, vol. 11, no. 4, pp. 622-629, 2010.
[41] P. L. Weiss, P. Bialik, and R. Kizony, “Virtual reality provides leisure time opportunities for young adults with physical and intellectual disabilities,” CyberPsychology & Behavior, vol. 6, no. 3, pp. 335-342, 2003.
[42] The Internet Stroke Center, “Stroke Statistics,” [Online]. Available: http://www.strokecenter.org/patients/about-stroke/stroke-statistics/.[Accessed July 13, 2019].