本論文開發了可用於中風偏癱患者復健的擴增實境鏡像治療軟體，一種相較於沉浸性虛擬實境復健系統、可直接在使用者iOS手機的方案，旨在提供傳統鏡像治療的便利性，同時回饋高沉浸的視覺刺激以提供較好的上肢復健效果。本軟體基於Apple iOS作業系統開發，由手機後鏡頭、人體語義分割神經網路與支持圖形運算加速的渲染器構成。模擬鏡像治療原理、將使用者手部輪廓影像以最高60幀的更新率即時渲染於對側視野中。本論文驗證擴增實境鏡像治療軟體成效的研究招募了三十名年輕的健康受試者參加臨床試驗，每位受試者均在前後一周的時間分別被施以一次30分鐘的傳統鏡像治療與擴增實境鏡像治療的上肢功能介入實驗，在實驗開始後的前十分鐘使用了功能性近紅外光譜，估測受試者在不同介入條件下，執行十次一分鐘捏取運動時的前額葉與運動感覺皮質區的血流灌注量；後二十分鐘執行上肢運動功能訓練，包含前臂/拇指旋轉60次、手腕/手指屈伸60次、對掌運動60次以及肌腱滑動訓練60次。評估其在抓握提取測試、普渡釘板測試、明尼蘇達手動敏捷測試、兩點距離測試以及單絲觸覺測試中的前後測表現，並使用重複測量變異數分析統計組間差異。結果發現擴增實境鏡像治療在提升手指捏取協調性、手指靈活度、上肢粗大運動以及降低兩點距離閥值的表現上，均優於傳統鏡像治療與前測基準，並且存在顯著差異。功能性近紅外光譜的測量則顯示兩種介入方式下的前額葉左右腦區之時間血流變化量，其相關係數均達0.9以上；運動感覺皮質區的相關性則分別為0.3（擴增實境鏡像治療）與0.7（傳統鏡像治療）以上。研究結果顯示該軟體具有應用在臨床居家中風上肢復健的潛力。

This paper has developed an augmented reality mirror therapy software that can be used for the rehabilitation of stroke patients with hemiplegia. Compared with the immersive virtual reality rehabilitation system, it can be directly installed on the user's iOS mobile phone. Aiming to provide the convenience of traditional mirror therapy while giving highly immersive visual stimulation to provide better upper limb rehabilitation. This software is developed based on the Apple iOS operating system, and consists of a mobile phone rear camera, a human body semantic segmentation neural network, and a renderer that supports graphics computing acceleration. Simulating the principle of mirror therapy, the contour image of the user's hand is instantly rendered in the contralateral view at a maximum update rate of 60 frames. Thirty young healthy subjects were recruited to participate in clinical trials in this paper to verify the effectiveness of augmented reality mirror therapy software. Each subject was given a 30-minute traditional mirror therapy in one week before and after. In the upper limb function intervention experiment with augmented reality mirror therapy, functional near-infrared spectroscopy was used in the first ten minutes after the start of the experiment to estimate the subject's blood perfusion in the prefrontal cortex and sensorimotor cortex area by performing ten one-minute pinch tasks under different intervention conditions; after 20 minutes, perform upper limb motor function training, including forearm/thumb rotation 60 times, wrist/finger flexion and extension 60 times, palm movement 60 times and tendon sliding training 60 times . The pre- and post-test performance in the Pinch-Holding-Up-Activity test, Purdue Peg board test, Minnesota Manual Dexterity test, Two-point Discrimination test, and Semmes-Weinstein Monofilament was evaluated. The differences between groups were statistically analyzed using repeated measures variance analysis. It was found that augmented reality mirror therapy was superior to traditional mirror therapy and pre-test benchmarks in improving finger pinch coordination, finger dexterity, upper limb gross movement, and reducing the distance threshold between two points, and there were significant differences. The measurement of functional near-infrared spectroscopy showed that under the two intervention methods, the correlation coefficients of the temporal blood flow changes in the left and right brain regions of the prefrontal cortex were all above 0.9; the correlation coefficients in the sensorimotor cortex were 0.3 (augmented reality mirror therapy) and above 0.7 (traditional mirror therapy). The results of the study show that the software has the potential in clinical home stroke upper limb rehabilitation.

[1] B. H. Dobkin, The clinical science of neurologic rehabilitation. Oxford University Press, 2003.
[2] R. Teasell et al., “Canadian stroke best practice recommendations: rehabilitation, recovery, and community participation following stroke. Part one: rehabilitation and recovery following stroke; update 2019,” International Journal of Stroke, vol. 15, no. 7, pp. 763-788, 2020.
[3] V. S. Ramachandran, D. Rogers-Ramachandran, and S. Cobb, “Touching the phantom limb,” Nature, vol. 377, pp. 489-490, 1995.
[4] C.-W. Lin, L.-C. Kuo, Y.-C. Lin, F.-C. Su, Y.-A. Lin, and H.-Y. Hsu, “Development and testing of a virtual reality mirror therapy system for the sensorimotor performance of upper extremity: A pilot randomized controlled trial,” IEEE Access, vol. 9, pp. 14725-14734, 2021.
[5] C.-W. Lin, L.-C. Kuo, Y.-C. Lin, F.-C. Su, T.-H. Yang, and H.-Y. Hsu, “Effects of a virtual reality–based mirror therapy program on improving sensorimotor function of hands in chronic stroke patients: a randomized controlled trial,” Neurorehabilitation and Neural Repair, vol. 36, no. 6, pp. 335-345, 2022.
[6] (2021). Global strategy on digital health 2020-2025.
[7] M. Costandi, Neuroplasticity. MIt Press, 2016.
[8] M. Maier, B. R. Ballester, and P. F. Verschure, “Principles of neurorehabilitation after stroke based on motor learning and brain plasticity mechanisms,” Frontiers in systems neuroscience, vol. 13, p. 74, 2019.
[9] E. Fuchs and G. Flügge, “Adult neuroplasticity: more than 40 years of research,” Neural plasticity, vol. 2014, 2014.
[10] J. Shaffner, “Neuroplasticity and clinical practice: building brain power for health. Front Psychol. 2016; 7: 1118,” ed, 2016.
[11] M. Hallett, “Neuroplasticity and rehabilitation,” Journal of Rehabilitation Research and Development, vol. 42, no. 4, p. R17, 2005.
[12] A. Pascual-Leone, “Modulation of motor cortical outputs to the reading hand of braille readers,” Annals of Neurology, vol. 34, pp. 33-37, 1993, doi: 10.1002/ana.410340108.
[13] R. J. Nudo, “Functional and structural plasticity in motor cortex: implications for stroke recovery,” Physical Medicine and Rehabilitation Clinics, vol. 14, no. 1, pp. S57-S76, 2003.
[14] J. C. Grotta et al., “Constraint-induced movement therapy,” Stroke, vol. 35, no. 11_suppl_1, pp. 2699-2701, 2004.
[15] M. Hallett, “Plasticity of the human motor cortex and recovery from stroke,” Brain research reviews, vol. 36, no. 2-3, pp. 169-174, 2001.
[16] J. Bernhardt, H. Dewey, A. Thrift, and G. Donnan, “Inactive and alone: physical activity within the first 14 days of acute stroke unit care,” Stroke, vol. 35, no. 4, pp. 1005-1009, 2004.
[17] J. Livingston-Thomas et al., “Exercise and environmental enrichment as enablers of task-specific neuroplasticity and stroke recovery,” Neurotherapeutics, vol. 13, pp. 395-402, 2016.
[18] X. Chen et al., “Therapeutic effects of sensory input training on motor function rehabilitation after stroke,” Medicine, vol. 97, no. 48, 2018.
[19] A. Rören, D. M. Yagappa, C. Théry, M.-M. Lefèvre-Colau, F. Rannou, and C. Nguyen, “Remote telerehabilitation to maintain adherence to home-based exercise therapy in people with musculoskeletal disorders: A pilot study,” Annals of physical and rehabilitation medicine, vol. 66, no. 5, p. 101723, 2023.
[20] M. White, J. N. Stinson, P. Lingley-Pottie, P. J. McGrath, N. Gill, and A. Vijenthira, “Exploring therapeutic alliance with an internet-based self-management program with brief telephone support for youth with arthritis: a pilot study,” Telemedicine and e-Health, vol. 18, no. 4, pp. 271-276, 2012.
[21] M. A. Cottrell, O. A. Galea, S. P. O’Leary, A. J. Hill, and T. G. Russell, “Real-time telerehabilitation for the treatment of musculoskeletal conditions is effective and comparable to standard practice: a systematic review and meta-analysis,” Clinical rehabilitation, vol. 31, no. 5, pp. 625-638, 2017.
[22] A. Gover-Chamlou and J. W. Tsao, “Telepain management of phantom limb pain using mirror therapy,” Telemedicine and e-Health, vol. 22, no. 2, pp. 176-179, 2016.
[23] T. G. Russell, “Physical rehabilitation using telemedicine,” Journal of telemedicine and telecare, vol. 13, no. 5, pp. 217-220, 2007.
[24] T. Hoffmann, T. Russell, L. Thompson, A. Vincent, and M. Nelson, “Using the Internet to assess activities of daily living and hand function in people with Parkinson's disease,” NeuroRehabilitation, vol. 23, no. 3, pp. 253-261, 2008.
[25] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, and B. G. Celler, “Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring,” IEEE transactions on information technology in biomedicine, vol. 10, no. 1, pp. 156-167, 2006.
[26] T. Wark, M. Karunanithi, and W. Chan, “A framework for linking gait characteristics of patients with accelerations of the waist,” in 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, 2006: IEEE, pp. 7695-7698.
[27] H. M.K, “Virtual environments for motor rehabilitation,” vol. 8, ed: MARY ANN LIEBERT INC 140 HUGUENOT STREET, 3RD FL, NEW ROCHELLE, NY 10801 USA, 2005, pp. 212-212.
[28] K. Laver, S. George, S. Thomas, J. Deutsch, and M. Crotty, “Virtual reality for stroke rehabilitation: an abridged version of a Cochrane review,” European journal of physical and rehabilitation medicine, vol. 51, no. 4, pp. 497-506, 2015.
[29] A. Rothgangel and R. Bekrater-Bodmann, “Mirror therapy versus augmented/virtual reality applications: towards a tailored mechanism-based treatment for phantom limb pain,” Pain management, vol. 9, no. 2, pp. 151-159, 2019.
[30] R. Kizony, L. Raz, N. Katz, H. Weingarden, and P. L. T. Weiss, “Video-capture virtual reality system for patients with paraplegic spinal cord injury,” Journal of Rehabilitation Research & Development, vol. 42, no. 5, 2005.
[31] J. E. Deutsch, A. S. Merians, S. Adamovich, H. Poizner, and G. C. Burdea, “Development and application of virtual reality technology to improve hand use and gait of individuals post-stroke,” Restorative neurology and neuroscience, vol. 22, no. 3-5, pp. 371-386, 2004.
[32] M. Park et al., “Effects of virtual reality-based planar motion exercises on upper extremity function, range of motion, and health-related quality of life: a multicenter, single-blinded, randomized, controlled pilot study,” Journal of neuroengineering and rehabilitation, vol. 16, no. 1, pp. 1-13, 2019.
[33] R. Miclaus et al., “Non-immersive virtual reality for post-stroke upper extremity rehabilitation: a small cohort randomized trial,” Brain Sciences, vol. 10, no. 9, p. 655, 2020.
[34] L. M. Weber, D. M. Nilsen, G. Gillen, J. Yoon, and J. Stein, “Immersive virtual reality mirror therapy for upper limb recovery following stroke: A pilot study,” American journal of physical medicine & rehabilitation, vol. 98, no. 9, p. 783, 2019.
[35] K. Marek, I. Zubrycki, and E. Miller, “Immersion Therapy with Head-Mounted Display for Rehabilitation of the Upper Limb after Stroke,” Sensors, vol. 22, no. 24, p. 9962, 2022.
[36] S. Hoermann et al., “Computerised mirror therapy with augmented reflection technology for early stroke rehabilitation: clinical feasibility and integration as an adjunct therapy,” Disability and Rehabilitation, vol. 39, no. 15, pp. 1503-1514, 2017.
[37] G. A. d. Assis, A. G. D. Corrêa, M. B. R. Martins, W. G. Pedrozo, and R. d. D. Lopes, “An augmented reality system for upper-limb post-stroke motor rehabilitation: a feasibility study,” Disability and Rehabilitation: Assistive Technology, vol. 11, no. 6, pp. 521-528, 2016.
[38] A. Boschmann, D. Neuhaus, S. Vogt, C. Kaltschmidt, M. Platzner, and S. Dosen, “Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis,” Journal of neuroengineering and rehabilitation, vol. 18, no. 1, pp. 1-15, 2021.
[39] C. Zirbel, X. Zhang, and C. Hughes, “The VRehab system: a low-cost mobile virtual reality system for post-stroke upper limb rehabilitation for medically underserved populations,” in 2018 IEEE Global Humanitarian Technology Conference (GHTC), 2018: IEEE, pp. 1-8.
[40] T. Labs, “The Evolution of the Myo armband,” ed, 2014.
[41] N. LaPiana et al., “Acceptability of a mobile phone–based augmented reality game for rehabilitation of patients with upper limb deficits from stroke: Case study,” JMIR rehabilitation and assistive technologies, vol. 7, no. 2, p. e17822, 2020.
[42] M. Fiala, “Artag, a fiducial marker system using digital techniques, vol. 2,” ed: July, 2005.
[43] Y.-A. Barde, D. Edgar, and H. Thoenen, “Purification of a new neurotrophic factor from mammalian brain,” The EMBO journal, vol. 1, no. 5, pp. 549-553, 1982.
[44] E. S. Koroleva et al., “Serum BDNF’s role as a biomarker for motor training in the context of AR-based rehabilitation after ischemic stroke,” Brain sciences, vol. 10, no. 9, p. 623, 2020.
[45] M. Ferrari and V. Quaresima, “A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application,” Neuroimage, vol. 63, no. 2, pp. 921-935, 2012.
[46] J. Mehnert, M. Brunetti, J. Steinbrink, M. Niedeggen, and C. Dohle, “Effect of a mirror-like illusion on activation in the precuneus assessed with functional near-infrared spectroscopy,” Journal of Biomedical Optics, vol. 18, no. 6, pp. 066001-066001, 2013.
[47] D. H. Kim, K.-D. Lee, T. C. Bulea, and H.-S. Park, “Increasing motor cortex activation during grasping via novel robotic mirror hand therapy: a pilot fNIRS study,” Journal of NeuroEngineering and Rehabilitation, vol. 19, no. 1, pp. 1-14, 2022.
[48] J. J. Zhang, K. N. Fong, N. Welage, and K. P. Liu, “The activation of the mirror neuron system during action observation and action execution with mirror visual feedback in stroke: a systematic review,” Neural plasticity, vol. 2018, 2018.
[49] F. G. S. Velez et al., “Real-time video projection in an mri for characterization of neural correlates associated with mirror therapy for phantom limb pain,” JoVE (Journal of Visualized Experiments), no. 146, p. e58800, 2019.
[50] C. Neiger. “Virtual reality is too expensive for most people — but that's about to change.” https://www.businessinsider.com/why-is-virtual-reality-so-expensive-2016-9 (accessed).
[51] “Omdia research reveals 12.5m consumer VR headsets sold in 2021 with content spend exceeding $2bn.” OMDIA. https://omdia.tech.informa.com/pr/2021-dec/omdia-research-reveals-12m-consumer-vr-headsets-sold-in-2021-with-content-spend-exceeding-2bn (accessed).
[52] AVDepthData. Apple Developer. [Online]. Available: https://developer.apple.com/documentation/avfoundation/avdepthdata
[53] Y. Guo, Y. Liu, T. Georgiou, and M. S. Lew, “A review of semantic segmentation using deep neural networks,” International journal of multimedia information retrieval, vol. 7, pp. 87-93, 2018.
[54] Z. Qu and L. Zhang, “Research on image segmentation based on the improved Otsu algorithm,” in 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics, 2010, vol. 2: IEEE, pp. 228-231.
[55] A. Dos Anjos and H. R. Shahbazkia, “Bi-level image thresholding,” Biosignals, vol. 2, pp. 70-76, 2008.
[56] A. Betancourt, “EgoHands: a unified framework for hand-based methods in first person vision videos,” 2017.
[57] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 2015: Springer, pp. 234-241.
[58] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520.
[59] L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,” arXiv preprint arXiv:1706.05587, 2017.
[60] “Core ML Models.” APPLE Developer. https://developer.apple.com/machine-learning/models/ (accessed).
[61] P. Turner, “ARHeadsetKit: Bringing Affordable AR Headset Technology to the Masses,” Ocean Lakes High School, 2022.
[62] M. Mihara et al., “Cortical control of postural balance in patients with hemiplegic stroke,” Neuroreport, vol. 23, no. 5, pp. 314-319, 2012.
[63] S. B. Moro et al., “A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: a functional near-infrared spectroscopy study,” Neuroimage, vol. 85, pp. 451-460, 2014.
[64] J. R. Flanagan and A. M. Wing, “Modulation of grip force with load force during point-to-point arm movements,” Experimental brain research, vol. 95, pp. 131-131, 1993.
[65] T. J. Huppert, S. G. Diamond, M. A. Franceschini, and D. A. Boas, “HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain,” Applied optics, vol. 48, no. 10, pp. D280-D298, 2009.
[66] A. von Lühmann, A. Ortega-Martinez, D. A. Boas, and M. A. Yücel, “Using the general linear model to improve performance in fNIRS single trial analysis and classification: a perspective,” Frontiers in human neuroscience, vol. 14, p. 30, 2020.
[67] A. von Lühmann, X. Li, K.-R. Müller, D. A. Boas, and M. A. Yücel, “Improved physiological noise regression in fNIRS: a multimodal extension of the general linear model using temporally embedded canonical correlation analysis,” NeuroImage, vol. 208, p. 116472, 2020.
[68] T. Kawakami. “MirrorBox Lite.” https://reurl.cc/p64Dl8 (accessed).
[69] A. GmbH. “AS-Mirror.” https://reurl.cc/mD4nLj (accessed).
[70] S. R. P. T. P.A. “Mirror Box: CRPS & RSD, Stroke.” https://reurl.cc/VLQ68Y (accessed).