過去研究顯示，患者以動作想像控制腦機介面進行復健，可以改善運動功能，但僅少數受試者的動作意念能被準確地辨識，大多需經長期訓練或藉由較複雜系統才能正常操作腦機介面。
本研究目的為發展一混合型腦機介面，讓動作想像腦波特徵不明顯的使用者也能藉其得到與運動意念同步的體感回饋。本研究增加了動畫視覺引導和穩態光源刺激，提升受試者事件相關去同步化333程度以及增加更容易辨識的腦波特徵。系統以快速傅立葉分析每0.04秒提取一次C3, C4, Oz通道的腦波特徵，並以相同速率進行辨識和更新復健矯形手的作動狀態，偵測到動作意念時牽引受試者伸展手指。本研究徵招六位未經訓練的常人測試雛型系統，在三種狀態間的控制達到平均83%的線上控制成功率和9.74位元/分的資料傳輸率。受試者產生穩態視覺刺激電位時並不會抑制事件相關去同步化，顯示混合特徵方法可在不影響運動相關腦波特徵為前提下，有效地提升腦機介面辨識成功率。動畫視覺引導在腦波調變實驗及功能性磁振造影結果，顯示未增強事件相關去同步化或腦部運動區活化，但有受試者回報其可助於減輕運動想像時的精神負擔。
結論，本研究發展之混合型腦機介面有可能取代單純基於動作想像的腦機介面以應用於復健，但動畫視覺引導並未立即改善受試者運動相關腦波特徵。

Stroke rehabilitation with motor-imagery-based brain-computer-interface (MI-BCI) can induce motor function improvements. However, it does not work for all users, very few percentage of subjects can effectively use MI-BCI without intensive training. The aim of this study was to develop a hybrid system for those who cannot effectively use the MI-BCI to attain adequate BCI accuracy for neuro rehabilitation. The proposed system integrates the action observation event-related desynchronization (ERD) and steady state visual evoked potential (SSVEP) features with a MI-BCI to enhance its performance. Features for the hybrid BCI were extracted from EEG of C3, C4 and Oz by performing fast Fourier transform every 40ms. The motion states of the controlled orthotic hands were updated at same rate to extend the subject’s hands when his/her intent of motion was detected. Six able-bodied subjects without previous BCI experience were recruited to test the functions of the prototype system. Experimental results show an averaged success rate of 83% and information transfer rate of 9.74 bit/min. EEG modulation and fMRI experiment results show that observation of dynamic visual guiding does not enhance the ERD but it can ease the mental load on subjects during motor imagery. When performing the hybrid tasks, the ERD and the activation in brain motor area were not interfered by the SSVEP. In conclusion, a SSVEP-ERD hybrid BCI system was developed and it may have potential to outperform conventional MI-BCI for assisting the rehabilitation of stroke patients. Although the visual guide may be beneficial for the stroke patient, it does not enhance BCI accuracy significantly for first-time users.

[1] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, T. M. Vaughan, “Brain–computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, no. 6, pp. 767-791, 2002.
[2] "The Future in Brain/Neural-Computer Interaction: Horizon 2020,", 2015; http://bnci-horizon-2020.eu/roadmap.
[3] U. Hoffmann, J. M. Vesin, T. Ebrahimi, K. Diserens, “An efficient P300-based brain-computer interface for disabled subjects,” Neuroscience Methods, vol. 167, no. 1, pp. 115-125, 2008.
[4] G. R. Muller-Putz, G. Pfurtscheller, “Control of an electrical prosthesis with an SSVEP-based BCI,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 1, pp. 361-364, 2008.
[5] T. Kaufmann, A. Herweg, A. Kübler, “Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials,” NeuroEngineering and Rehabilitation, vol. 11, no.7 (7 pages), 2014.
[6] L. M. Vaca Benitez, M. Tabie, N. Will, S. Schmidt, et al, “Exoskeleton Technology in Rehabilitation: Towards an EMG-Based Orthosis System for Upper Limb Neuromotor Rehabilitation,” Robotics, vol. 2013, pp. 1-13, 2013.
[7] P. Maciejasz, J. Eschweiler, K. Gerlach-Hahn, et al, “A survey on robotic devices for upper limb rehabilitation,” NeuroEngineering and Rehabilitation, vol. 11, no. 3 (29 pages) , 2014.
[8] H. D, The Organization of behavior, New York: John Wiley, 1949.
[9] K. A. Asanuma H, “Neurological basis of motor learning and memory,” Concepts in Neuroscience, vol 2, no 1, pp. 1-30, 1991.
[10] A. Ramos-Murguialday, D. Broetz, M. Rea, et al, “Brain-Machine-Interface in Chronic Stroke Rehabilitation: A Controlled Study,” Annals of neurology, vol. 74, no. 1, pp. 100-108, 2013.
[11] J. J. Daly, J. R. Wolpaw, “Brain–computer interfaces in neurological rehabilitation,” The Lancet Neurology, vol. 7, no. 11, pp. 1032-1043, 2008.
[12] B. C. Osuagwu, L. Wallace, M. Fraser, A. Vuckovic, “Rehabilitation of hand in subacute tetraplegic patients based on brain computer interface and functional electrical stimulation: a randomised pilot study,” Neural Engineering, vol. 13, no. 6, 065002 (13 pages), 2016.
[13] K. K. Ang, K. S. Chua, K. S. Phua, et al, “A Randomized Controlled Trial of EEG-Based Motor Imagery Brain-Computer Interface Robotic Rehabilitation for Stroke,” Clinical EEG and Neuroscience, vol. 46, no. 4, pp. 310-320, 2015.
[14] K. Brodmann, “Brodmann's‘Localisation in the Cerebral Cortex,” 2000.
[15] C. S. Herrmann, “Human EEG responses to 1~100Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena,” Experimental Brain Research, vol. 137, no. 3-4, pp. 346-353, 2001.
[16] D. Zhu, J. Bieger, G. Garcia Molina, R. M. Aarts, “A survey of stimulation methods used in SSVEP-based BCIs,” Computational Intelligence and Neuroscience, vol. 2010, 702357 (12 pages), 2010.
[17] Y. Liu, X. Jiang, T. Cao, F. Wan, et al, "Implementation of SSVEP based BCI with Emotiv EPOC." EWU Masters Thesis Collection, Paper 39, 2012.
[18] A. Materka, M. Byczuk, “Alternate half-field stimulation technique for SSVEP-based brain–computer interfaces,” Electronics Letters, vol. 42, no. 6, pp. 321-322, 2006.
[19] B. Z. Allison, D. J. McFarland, G. Schalk, et al, “Towards an Independent Brain - Computer Interface Using Steady State Visual Evoked Potentials,” Clinical neurophysiology, vol. 119, no. 2, pp. 399-408, 2008.
[20] X. Chen, Z. Chen, S. Gao, X. Gao, “Brain-computer interface based on intermodulation frequency,” Neural Engineering, vol. 10, no. 6, 066009 (9 pages), 2013.
[21] J. Annett, “Motor imagery: Perception or action?,” Neuropsychologia, vol. 33, no. 11, pp. 1395-1417, 1995.
[22] C. Schuster, R. Hilfiker, O. Amft, et al, “Best practice for motor imagery: a systematic literature review on motor imagery training elements in five different disciplines,” BMC Medicine, vol. 9, no. 75 (35 pages), 2011.
[23] J. Munzert, B. Lorey, K. Zentgraf, “Cognitive motor processes: the role of motor imagery in the study of motor representations,” Brain Research Reviews, vol. 60, no. 2, pp. 306-326, 2009.
[24] T. Mulder, “Motor imagery and action observation: cognitive tools for rehabilitation,” Neural Transmission, vol. 114, no. 10, pp. 1265-1278, 2007.
[25] N. Sharma, V. M. Pomeroy, J. C. Baron, “Motor imagery: a backdoor to the motor system after stroke?,” Stroke, vol. 37, no. 7, pp. 1941-1952, 2006.
[26] S. B. i. Badia, A. G. Morgade, H. Samaha, P. F. M. J. Verschure, “Using a Hybrid Brain Computer Interface and Virtual Reality System to Monitor and Promote Cortical Reorganization through Motor Activity and Motor Imagery Training,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 21, no. 2, pp. 174-181, 2013.
[27] H. E. Rossiter, M. R. Borrelli, R. J. Borchert, et al, “Cortical mechanisms of mirror therapy after stroke,” Neurorehabilitation & Neural Repair, vol. 29, no. 5, pp. 444-452, 2015.
[28] B. L. Chan, R. Witt, A. P. Charrow, et al, “Mirror Therapy for Phantom Limb Pain,” New England Journal of Medicine, vol. 357, no. 21, pp. 2206-2207, 2007.
[29] V. S. Ramachandran, E. L. Altschuler, “The use of visual feedback, in particular mirror visual feedback, in restoring brain function,” Brain, vol. 132, no. 7, pp. 1693-1710, 2009.
[30] H. Thieme, J. Mehrholz, M. Pohl, J. Behrens, C. Dohle, “Mirror therapy for improving motor function after stroke,” Stroke, vol. 44, no. 1, pp. e1-e2, 2013.
[31] B. Abibullaev, J. An, S. H. Lee, J. I. Moon, “Design and evaluation of action observation and motor imagery based BCIs using Near-Infrared Spectroscopy,” Measurement, vol. 98, pp. 250-261, 2017.
[32] G. Pfurtscheller, F. H. Lopes da Silva, “Event-related EEG/MEG synchronization and desynchronization: basic principles,” Clinical Neurophysiology, vol. 110, no. 11, pp. 1842-1857, 1999.
[33] A. Streltsova, C. Berchio, V. Gallese, M. A. Umilta, “Time course and specificity of sensory-motor alpha modulation during the observation of hand motor acts and gestures: a high density EEG study,” Experimental Brain Research, vol. 205, no. 3, pp. 363-373, 2010.
[34] M. Jeannerod, “Neural simulation of action: a unifying mechanism for motor cognition,” Neuroimage, vol. 14, no. 1, pp. S103-S109, 2001.
[35] J. D. Millan, R. Rupp, G. R. Muller-Putz, R. Murray-Smith, et al, “Combining Brain-Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges,” Frontiers in Neuroscience, vol. 4, 00161 (15 pages), 2010.
[36] G. Pfurtscheller, B. Z. Allison, C. Brunner, et al, “The hybrid BCI,” Frontiers in Neuroscience, vol. 4, 00042 (11 pages), 2010.
[37] G. R. Muller-Putz, C. Breitwieser, F. Cincotti, et al, “Tools for Brain-Computer Interaction: A General Concept for a Hybrid BCI,” Frontiers in Neuroscience, vol. 5, 00030 (10 pages), 2011.
[38] B. Rebsamen, E. Burdet, Q. Zeng, et al, “Hybrid P300 and mu-beta brain computer interface to operate a brain controlled wheelchair,” Proceedings of the 2nd International Convention on Rehabilitation Engineering & Assistive Technology, Bangkok, pp. 51-55, 2008.
[39] A. Riccio, E. M. Holz, P. Arico, et al, “Hybrid P300-based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task,” Archives of Physical Medicine and Rehabilitation, vol. 96, no. 3, pp. S54-S61, 2015.
[40] R. Leeb, H. Sagha, R. Chavarriaga, R. Millan Jdel, “A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities,” Neural Engineering, vol. 8, no. 2, 025011 (5 pages), 2011.
[41] C. Brunner, B. Z. Allison, C. Altstatter, C. Neuper, “A comparison of three brain-computer interfaces based on event-related desynchronization, steady state visual evoked potentials, or a hybrid approach using both signals,” Neural Engineering, vol. 8, no. 2, 025010 (8 pages), 2011.
[42] S. Amiri, R. Fazel-Rezai, V. Asadpour, “A Review of Hybrid Brain-Computer Interface Systems,” Advances in Human-Computer Interaction, vol. 2013, 187024 (8 pages), 2013.
[43] B. Z. Allison, C. Brunner, V. Kaiser, et al, “Toward a hybrid brain-computer interface based on imagined movement and visual attention,” Neural Engineering, vol. 7, no. 2, 26007 (9 pages), 2010.
[44] G. Pfurtscheller, T. Solis-Escalante, R. Ortner, et al, “Self-paced operation of an SSVEP-Based orthosis with and without an imagery-based "brain switch:" a feasibility study towards a hybrid BCI,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 4, pp. 409-414, 2010.
[45] L. Cao, J. Li, H. Ji, C. Jiang, “A hybrid brain computer interface system based on the neurophysiological protocol and brain-actuated switch for wheelchair control,” Neuroscience Methods, vol. 229, pp. 33-43, 2014.
[46] R. C. Panicker, S. Puthusserypady, Y. Sun, “An asynchronous P300 BCI with SSVEP-based control state detection,” IEEE Transactions on Biomedical Engineering, vol. 58, no. 6, pp. 1781-1788, 2011.
[47] G. Edlinger, C. Holzner, C. Guger, "A Hybrid Brain-Computer Interface for Smart Home Control," Proceedings of Human-Computer Interaction. Interaction Techniques and Environments: 14th International Conference, Orlando, pp. 417-426, 2011.
[48] E. Yin, T. Zeyl, R. Saab, et al, “A Hybrid Brain-Computer Interface Based on the Fusion of P300 and SSVEP Scores,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 23, no. 4, pp. 693-701, 2015.
[49] Y. Su, Y. Qi, J.-x. Luo, et al, “A hybrid brain-computer interface control strategy in a virtual environment,” Journal of Zhejiang University SCIENCE C, vol. 12, no. 5, pp. 351-361, 2011.
[50] H. Riechmann, N. Hachmeister, H. Ritter, A. Finke, "Asynchronous, parallel on-line classification of P300 and ERD for an efficient hybrid BCI." Proceedings of the 5th International IEEE EMBS Conference on Neural Engineering Cancun, Mexico, pp. 412-415, 2011.
[51] S. Fazli, J. Mehnert, J. Steinbrink, et al, “Enhanced performance by a hybrid NIRS-EEG brain computer interface,” Neuroimage, vol. 59, no. 1, pp. 519-529, 2012.
[52] Y. Punsawad, Y. Wongsawat, M. Parnichkun, "Hybrid EEG-EOG brain-computer interface system for practical machine control." Proceedings of 32nd Annual International Conference of the IEEE EMBS, Buenos Aires, pp. 1360-1363, 2010.
[53] X. Yong, M. Fatourechi, R. K. Ward, G. E. Birch, “The Design of a Point-and-Click System by Integrating a Self-Paced Brain Computer Interface With an Eye-Tracker,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 1, no. 4, pp. 590-602, 2011.
[54] B. Choi, S. Jo, “A low-cost EEG system-based hybrid brain-computer interface for humanoid robot navigation and recognition,” PLoS One, vol. 8, no. 9, e74583 (13 pages), 2013.
[55] 林宙晴, 朱銘祥, 王政道, “運用μ波為腦機介面控制源之研究”, 八十七年醫學工程科技研討會論文集, 1998.
[56] 陳志瑋, “研究以小波神經網路做μ波即時鑑別," 國立成功大學機械工程學系碩士論文, 2002.
[57] C. C. K. Lin, M. S. Ju, Y. N. Sun., “Comparison of Adaptive Cancellation and Laplacian Operation in Removing Eye Blinking Artifacts in EEG,” Medical and Biological Engineering, vol. 24, no. 1, pp. 9-15, 2004.
[58] C. C. K. Lin, M. S. Ju, C. W. Hsu, Y. N. Sun, “Applying stochastic resonance to magnify mu and beta wave suppression,” Computers in Biology and Medicine, vol. 38, no. 10, pp. 1068-1075, 2008.
[59] C. W. Chen, C. C. Lin, M. S. Ju, “Detecting movement-related EEG change by wavelet decomposition-based neural networks trained with single thumb movement,” Clinical Neurophysiology, vol. 118, no. 4, pp. 802-814, 2007.
[60] C. W. Chen, M. S. Ju, Y. N. Sun, C. C. K. Lin, “Model analyses of visual biofeedback training for EEG-based brain-computer interface,” Computational Neuroscience, vol. 27, no. 3, pp. 357-368, 2009.
[61] C. C. K. Lin, C. W. Chen, M. S. Ju, “Hand Orthosis Controlled by Using Brain-Computer Interface,” Medical and Biological Engineering, vol. 29, no.5, pp. 234-241, 2009.
[62] J. M. , J. Y. Lee, C. C. K. Lin, S. M. Chen, “EEG-Controlled Robot For Hand Rehabilitation of a Stroke Patient and Evaluation by Functional Magnetic Resonance Imaging,” in World Congress on Bioengineering, Tainan, 2011.
[63] L. J. , M. S. Ju, C. C. K. Lin, S. M. Chen, “EEG Braincomputer-interface-controlled Orthosis for Neuro-rehabilitation of Hands of Stroke Patients-A Case Study,” in World Congress on Medical Physics and Biomedical Engineering, Beijing, 2012.
[64] L. C. , M. S. Ju, C. H. Chan, “Brain-connectivity assessment of rehabilitation of hands of stroke patients by brain-computer-interface controlled robot,” in World Congress on Bioengineering, Beijing, 2013.
[65] L. Y. , K. C. Chao, C. C. K. Lin, M. S. Ju, “Integrate Transcranial Direct Current Stimulation with Brain-Computer-Interface Controlled Orthotic Hands for Rehabilitation of Stroke Patients," Proceedings of the 8th Asian-Pacific Conference on Biomechanics,” Sapporo, 2015.
[66] "10/20 System Positionaing MANUAL," T. C. Technologies, ed.
[67] 李阮曜, “腦機介面控制復健機械手於中風病患手部復健及功能性磁振造影評估," 國立成功大學機械工程學系碩士論文, 2011.
[68] 陳志瑋, “以mu波為基礎之大腦電腦介面之實現與強化," 國立成功大學機械工程學系博士論文, 2009.
[69] J. Gómez, M. Aguilar, E. Horna, J. Minguez, "Quantification of event-related desynchronization/synchronization at low frequencies in a semantic memory task," Engineering in Medicine and Biology Society, 2012 Annual International Conference of the IEEE , pp. 2522-2526, 2012.
[70] G. Pfurtscheller, C. Neuper, “Motor imagery activates primary sensorimotor area in humans,” Neuroscience Letters, vol. 239, no. 2–3, pp. 65-68, 1997.
[71] M. Wilke, V. J. Schmithorst, “A combined bootstrap/histogram analysis approach for computing a lateralization index from neuroimaging data,” Neuroimage, vol. 33, no. 2, pp. 522-530, 2006.
[72] B. M. Young, Z. Nigogosyan, L. M. Walton, et al, “Changes in functional brain organization and behavioral correlations after rehabilitative therapy using a brain-computer interface,” Frontiers in Neuroscience, vol. 7, 00026 (15 pages), 2014.
[73] L. Bottou , C. j. Lin, “Support Vector Machine Solvers,” 2006, https://www.csie.ntu.edu.tw/~cjlin/papers/bottou_lin.pdf.
[74] R. Bousseta, S. Tayeb, I. E. Ouakouak, et al, "EEG efficient classification of imagined hand movement using RBF kernel SVM," Proceedings of 11th International Conference on Intelligent Systems: Theories and Applications, pp. 1-6, 2016.
[75] H. Lohninger. "Structure of Measured Data," Last Update: 2012-10-08; http://www.statistics4u.com/fundstat_eng/cc_data_structure.html.
[76] J. Wolpaw, H. Ramoser, “EEG-based communication: improved accuracy by response verification,” IEEE Transactions on Rehabilitation Engineering, vol. 6, no. 3, pp. 326-333, 1998.
[77] E. Combrisson, K. Jerbi, “Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy,” Neuroscience Methods, vol. 250, pp. 126-136, 2015.
[78] K. J. Friston, J. Ashburner, S. J. Kiebel, T. E. Nichols, "Statistical Parametric Mapping: The Analysis of Functional Brain Images," Elsevier/Academic Press, 2007.
[79] F. Filimon, J. D. Nelson, D. J. Hagler, M. I. Sereno, “Human cortical representations for reaching: Mirror neurons for execution, observation, and imagery,” NeuroImage, vol. 37, no. 4, pp. 1315-1328, 2007.
[80] J. C. Taylor, A. J. Wiggett, P. E. Downing, “Functional MRI analysis of body and body part representations in the extrastriate and fusiform body areas,” Neurophysiol, vol. 98, no. 3, pp. 1626-1633, 2007.
[81] S. H. Johnson-Frey, F. R. Maloof, R. Newman-Norlund, et al, “Actions or Hand-Object Interactions? Human Inferior Frontal Cortex and Action Observation,” Neuron, vol. 39, no. 6, pp. 1053-1058, 2003.
[82] L. Li, J. Wang, G. Xu, et al, “The Study of Object-Oriented Motor Imagery Based on EEG Suppression,” PLoS One, vol. 10, no. 12, e0144256 (10 pages), 2015.
[83] A. Vuckovic, B. A. Osuagwu, “Using a motor imagery questionnaire to estimate the performance of a Brain-Computer Interface based on object oriented motor imagery,” Clinical Neurophysiology, vol. 124, no. 8, pp. 1586-1595, 2013.
[84] M. M. Mannan, M. Y. Jeong, M. A. Kamran, “Hybrid ICA-Regression: Automatic Identification and Removal of Ocular Artifacts from Electroencephalographic Signals,” Frontiers in Neuroscience, vol. 10, 00193 (17 pages), 2016.
[85] 龔品誠, “神經復健機器人於中風病患肩肘與前臂關節不正常協同動作之評估與治療,” 國立成功大學機械工程學系博士論文, 2011.