本研究旨在探討將超音波浮空觸覺回饋整合至三維視覺空間認知訓練系統之成效，利用自製的超音波相位陣列在8×8×10立方公分的空間區域內產生浮空觸覺刺激，並與基於Unity開發的動態接球互動式訓練遊戲進行同步，本研究招募20名健康成年志願者，隨機分配至實驗組（含觸覺回饋）或對照組（無觸覺回饋），此外，透過實驗室與場域型認知控制任務（如：側翼任務）評估介入前後的認知與動作認知表現，以檢驗潛在的遷移效應，結果顯示，實驗組展現出顯著較快的學習進度，至訓練第6天時，漂白的紅色球體數量多出26%，且比對照組提前3天達到表現穩定，混合設計變異數分析結果證實，兩組在訓練趨勢（p = 0.035）與關卡得分（p = 0.051）上存在統計學上的顯著差異，且具備中至大的效應量，儘管兩組在介入後於實驗室任務中的認知控制表現皆有提升（即反應時間的側翼效應減少），但僅實驗組在場域任務的知覺認知層面上，顯示出反應時間側翼效應的進一步降低（p = 0.003），此研究結果顯示，結合模擬體感皮層活動之觸覺回饋的認知訓練，對於動作任務期間的認知控制處理具有更佳效益，整合超音波觸覺回饋能顯著增強視覺空間訓練成效，並為認知復健提供了一種具擴展性且無需手持設備的解決方案，凸顯了多感官方法在優化認知健康上的潛力。

This study examined the effectiveness of incorporating ultrasonic mid-air haptic feedback into a 3D visuospatial cognitive training system. A custom-built ultrasonic phased array generated mid-air tactile stimuli within an 8×8×10 cm region, synchronized with a Unity-based interactive training game involving dynamic ball-catching tasks. This work assigned 20 healthy adult volunteers to the experimental (with haptic feedback) or control (without haptic feedback) groups. In addition, we assessed the cognitive and motor-cognitive performance before and after the intervention using lab-based and field-based cognitive control tasks (e.g., the flanker task) to examine potential transfer effects. The results showed that the experimental group demonstrated significantly faster learning progress by bleaching 26% more red spheres by day 6, and reaching performance stability 3 days earlier. Mixed-design ANOVA confirmed statistically significant differences in training trends (p = 0.035) and level scores (p = 0.051), with medium to large effect sizes. Besides, although both groups demonstrated improved cognitive control performance (i.e., reduced flanker effect in reaction time [RT]) following the intervention in the lab-based task, only the experimental group showed a further reduction in the flanker effect of RT at the perceptual-cognitive level on the field-based task (p = 0.003). This finding suggests that cognitive training with haptic feedback simulating somatosensory cortex activity may yield greater benefits for cognitive control processing during a motor task. This study showed that integrating ultrasonic haptic feedback meaningfully enhances visuospatial training outcomes, and offers a scalable, hands-free solution for cognitive rehabilitation, underscoring the promise of multisensory approaches to optimize cognitive health.

[1] A. C. Rosen, L. Sugiura, J. H. Kramer, S. Whitfield-Gabrieli, and J. D. Gabrieli, “Cognitive Training Changes Hippocampal Function in Mild Cognitive Impairment: A Pilot Study,” Journal of Alzheimer’s Disease, vol. 26, no. s3, pp. 349–357, Oct. 2011, doi: https://doi.org/10.3233/jad-2011-0009.
[2] D. Mapelli, E. Di Rosa, R. Nocita, and D. Sava, “Cognitive Stimulation in Patients with Dementia: Randomized Controlled Trial,” Dementia and Geriatric Cognitive Disorders Extra, vol. 3, no. 1, pp. 263–271, 2013, doi: https://doi.org/10.1159/000353457.
[3] W. J. Tippett and M. N. Rizkalla, “Brain training: rationale, methods, and pilot data for a specific visuomotor/visuospatial activity program to change progressive cognitive decline,” Brain and Behavior, vol. 4, no. 2, pp. 171–179, Dec. 2013, doi: https://doi.org/10.1002/brb3.196.
[4] R. S. Wilson, E. Segawa, P. A. Boyle, and D. A. Bennett, “Influence of late-life cognitive activity on cognitive health,” Neurology, vol. 78, no. 15, pp. 1123–1129, Apr. 2012, doi: https://doi.org/10.1212/wnl.0b013e31824f8c03.
[5] S. L. Willis et al., “Long-term Effects of Cognitive Training on Everyday Functional Outcomes in Older Adults,” JAMA, vol. 296, no. 23, p. 2805, Dec. 2006, doi: https://doi.org/10.1001/jama.296.23.2805.
[6] J. D. Van Horn et al., “Changing patterns of brain activation during maze learning,” Brain Research, vol. 793, no. 1, pp. 29–38, May 1998, doi: https://doi.org/10.1016/S0006-8993(98)00051-1.
[7] A. Jacotte-Simancas, D. Costa-Miserachs, M. Torras-Garcia, M. Coll-Andreu, and I. Portell-Cortés, “Effect of voluntary physical exercise and post-training epinephrine on acquisition of a spatial task in the barnes maze,” Behavioural Brain Research, vol. 247, pp. 178–181, Jun. 2013, doi: https://doi.org/10.1016/j.bbr.2013.03.038.
[8] A. Bauer and G. Andringa, “The Potential of Immersive Virtual Reality for Cognitive Training in Elderly,” Gerontology, vol. 66, no. 6, pp. 1–10, Sep. 2020, doi: https://doi.org/10.1159/000509830.
[9] M. M. Assed et al., “Memory training combined with 3D visuospatial stimulus improves cognitive performance in the elderly: pilot study,” Dementia & Neuropsychologia, vol. 14, no. 3, pp. 290–299, Sep. 2020, doi: https://doi.org/10.1590/1980-57642020dn14-030010.
[10] A. M. Kueider, J. M. Parisi, A. L. Gross, and G. W. Rebok, “Computerized Cognitive Training with Older Adults: A Systematic Review,” PLoS ONE, vol. 7, no. 7, p. e40588, Jul. 2012, doi: https://doi.org/10.1371/journal.pone.0040588.
[11] H. Coyle, V. Traynor, and N. Solowij, “Computerized and Virtual Reality Cognitive Training for Individuals at High Risk of Cognitive Decline: Systematic Review of the Literature,” The American Journal of Geriatric Psychiatry, vol. 23, no. 4, pp. 335–359, Apr. 2015, doi: https://doi.org/10.1016/j.jagp.2014.04.009.
[12] J. Lumsden, E. A. Edwards, N. S. Lawrence, D. Coyle, and M. R. Munafò, “Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy,” JMIR Serious Games, vol. 4, no. 2, p. e11, Jul. 2016, doi: https://doi.org/10.2196/games.5888.
[13] J. M. Kang et al., “Effect of Cognitive Training in Fully Immersive Virtual Reality on Visuospatial Function and Frontal-Occipital Functional Connectivity in Predementia: Randomized Controlled Trial,” Journal of Medical Internet Research, vol. 23, no. 5, p. e24526, May 2021, doi: https://doi.org/10.2196/24526.
[14] N. T. M. Hill, L. Mowszowski, S. L. Naismith, V. L. Chadwick, M. Valenzuela, and A. Lampit, “Computerized Cognitive Training in Older Adults With Mild Cognitive Impairment or Dementia: A Systematic Review and Meta-Analysis,” American Journal of Psychiatry, vol. 174, no. 4, pp. 329–340, Apr. 2017, doi: https://doi.org/10.1176/appi.ajp.2016.16030360.
[15] J.-H. Park, “Effects of virtual reality-based spatial cognitive training on hippocampal function of older adults with mild cognitive impairment,” International Psychogeriatrics, vol. 34, no. 2, pp. 1–7, Jul. 2022, doi: https://doi.org/10.1017/s1041610220001131.
[16] O. D. Kothgassner, A. Goreis, J. X. Kafka, R. L. Van Eickels, P. L. Plener, and A. Felnhofer, “Virtual Reality Exposure Therapy for Posttraumatic Stress Disorder (PTSD): a Meta-analysis,” European Journal of Psychotraumatology, vol. 10, no. 1, Aug. 2019, doi: https://doi.org/10.1080/20008198.2019.1654782.
[17] L. Shams and A. R. Seitz, “Benefits of Multisensory Learning,” Trends in Cognitive Sciences, vol. 12, no. 11, pp. 411–417, Nov. 2008, doi: https://doi.org/10.1016/j.tics.2008.07.006.
[18] A. Parisi et al., “Efficacy of Multisensory Technology in Post-Stroke Cognitive Rehabilitation: A Systematic Review,” Journal of Clinical Medicine, vol. 11, no. 21, p. 6324, Jan. 2022, doi: https://doi.org/10.3390/jcm11216324.
[19] K. von Kriegstein and A.-L. Giraud, “Implicit Multisensory Associations Influence Voice Recognition,” PLoS Biology, vol. 4, no. 10, p. e326, Sep. 2006, doi: https://doi.org/10.1371/journal.pbio.0040326.
[20] J. M. Rogers, J. Duckworth, S. Middleton, B. Steenbergen, and P. H. Wilson, “Elements virtual rehabilitation improves motor, cognitive, and functional outcomes in adult stroke: evidence from a randomized controlled pilot study,” Journal of NeuroEngineering and Rehabilitation, vol. 16, no. 1, May 2019, doi: https://doi.org/10.1186/s12984-019-0531-y.
[21] T. T. Schmidt and F. Blankenburg, “Brain regions that retain the spatial layout of tactile stimuli during working memory – A ‘tactospatial sketchpad’?,” NeuroImage, vol. 178, pp. 531–539, Sep. 2018, doi: https://doi.org/10.1016/j.neuroimage.2018.05.076.
[22] T. Guo et al., “Improving Visual Working Memory With Training on a Tactile Orientation Sequence Task in Humans,” SAGE Open, vol. 11, no. 3, p. 215824402110315, Jul. 2021, doi: https://doi.org/10.1177/21582440211031549.
[23] T. Yoshida et al., "RePro3D: full-parallax 3D display with haptic feedback using retro-reflective projection technology," 2011 IEEE International Symposium on VR Innovation, Singapore, Mar. 2011, pp. 49-54, doi: 10.1109/ISVRI.2011.5759601.
[24] Z. Zhang, J. Li, Z. Yan, J. Nishida, and H. Peng, “JetUnit: Rendering Diverse Force Feedback in Virtual Reality Using Water Jets,” Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, pp. 1–15, Oct. 2024, doi: https://doi.org/10.1145/3654777.3676440.
[25] G. Korres and M. Eid, “Characterization of Ultrasound Tactile Display,” Lecture Notes in Computer Science, pp. 78–89, 2016, doi: https://doi.org/10.1007/978-3-319-42321-0_8.
[26] S. Gupta, D. Morris, S. N. Patel, and D. Tan, “AirWave,” Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, pp. 419–428, Sep. 2013, doi: https://doi.org/10.1145/2493432.2493463.
[27] V. Shen, C. Harrison, and C. Shultz, “Expressive, Scalable, Mid-Air Haptics with Synthetic Jets,” ACM transactions on computer-human interaction, vol. 31, no. 2, Dec. 2023, doi: https://doi.org/10.1145/3635150.
[28] Y. Suzuki and M. Kobayashi, “Air jet driven force feedback in virtual reality,” IEEE Computer Graphics and Applications, vol. 25, no. 1, pp. 44-47, Jan.-Feb. 2005, doi: https://doi.org/10.1109/MCG.2005.
[29] T. Iwamoto, M. Tatezono, and H. Shinoda, “Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound,” Haptics: Perception, Devices and Scenarios, vol. 5024, pp. 504–513, 2008, doi: https://doi.org/10.1007/978-3-540-69057-3_64.
[30] R. Hirayama, D. Martinez Plasencia, N. Masuda, and S. Subramanian, “A volumetric display for visual, tactile and audio presentation using acoustic trapping,” Nature, vol. 575, no. 7782, pp. 320–323, Nov. 2019, doi: https://doi.org/10.1038/s41586-019-1739-5.
[31] Y. Monnai, K. Hasegawa, M. Fujiwara, K. Yoshino, S. Inoue, and H. Shinoda, “HaptoMime,” Proceedings of the 27th annual ACM symposium on User interface software and technology, Oct. 2014, doi: https://doi.org/10.1145/2642918.2647407.
[32] S. Inoue, Y. Makino, and H. Shinoda, “Active touch perception produced by airborne ultrasonic haptic hologram,” 2015 IEEE World Haptics Conference (WHC), Jun. 2015, doi: https://ieeexplore.ieee.org/abstract/document/7177739.
[33] A. Biswas, M. Manivannan, and M. A. Srinivasan, “Vibrotactile sensitivity threshold: nonlinear stochastic mechanotransduction model of the Pacinian Corpuscle,” IEEE transactions on haptics, vol. 8, no. 1, pp. 102–13, 2015, doi: https://doi.org/10.1109/TOH.2014.2369422.
[34] T. Howard, G. Gallagher, A. Lecuyer, C. Pacchierotti, and M. Marchal, “Investigating the Recognition of Local Shapes Using Mid-air Ultrasound Haptics,” 2019 IEEE World Haptics Conference (WHC), pp. 503–508, Jul. 2019, doi: https://doi.org/10.1109/whc.2019.8816127.
[35] K. R. Ridderinkhof, S. A. Wylie, W. P. M. van den Wildenberg, T. R. Bashore, and M. W. van der Molen, “The arrow of time: Advancing insights into action control from the arrow version of the Eriksen flanker task,” Attention, Perception, & Psychophysics, vol. 83, no. 2, pp. 700–721, Feb. 2021, doi: https://doi.org/10.3758/s13414-020-02167-z.
[36] C.-H. Wang, C.-T. Yang, D. Moreau, and N. G. Muggleton, “Motor expertise modulates neural oscillations and temporal dynamics of cognitive control,” NeuroImage, vol. 158, pp. 260–270, Sep. 2017, doi: https://doi.org/10.1016/j.neuroimage.2017.07.009.
[37] W.-L. Hsieh, S.-C. Kao, D. Moreau, C.-T. Wu, and C.-H. Wang, “Examining the relationship between response inhibition and Taekwondo performance: The importance of ecological validity,” Psychology of Sport and Exercise, vol. 78, p. 102817, Jan. 2025, doi: https://doi.org/10.1016/j.psychsport.2025.102817.
[38] L.-X. Lee, K.-H. Yeh, C.-C. Tsai, D. Moreau, S.-C. Kao, and C.-H. Wang, “Toward ecological validity: A psychometric evaluation of a motor-cognitive task integrating executive function and physical performance,” presented at the Int. Soc. Sport Psychol. (ISSP) 16th World Congr., Dec. 2025.