隨著科技演進，人機互動介面逐漸由物理操作介面發展至虛擬操作介面，而虛擬操作介面若不能提供人類適當的感官回饋，將降低操作效率與使用者操作體驗。為增加互動介面的操作性，本論文提出一非接觸式超音波觸覺回饋系統，透過疊合超音波於空中定點，形成駐波並產生回饋，不僅解決穿戴式回饋系統造成的操作負擔問題，更可產生回饋於手掌各處，提升觸覺回饋應用性。
本系統使用雙鏡頭進行多視角手部骨架辨識，擴增系統辨識區域，解決手勢遮擋問題，讓系統在進行Unity物件操作辨識及超音波匯聚點計算有更好的效果。計算完超音波匯聚點後，匯聚點資訊將由電腦端傳至FPGA (Field Programmable Gate Array)端，以輸出控制波形至自行設計之超音波驅動電路，此超音波驅動電路設計目標為使用較少的控制訊號驅動多顆超音波發射器，達到系統驅動單元效率最佳化。為取得匯聚點成效，本研究透過三維量測平台對平面進行掃描，根據量測數據得知，匯聚點大小約為10 mm × 10 mm。此外，為取得人類手部最敏感之觸覺回饋受器觸發頻率，本研究分別從20-400 Hz間各頻率量測人體可感受之最低操作電壓，由實驗結果得知，在AM (Amplitude Modulation) 回饋模式下以100 Hz處為最敏感頻率；在STM (Spatio-Temporal Modulation)回饋模式下以50 Hz為最敏感頻率。最後本研究將AM及STM應用於Unity互動介面中實際進行測試，驗證觸覺回饋對於使用者操作效率之助益，由實驗結果得知，加入觸覺回饋可降低操作時間24.63%，證明超音波觸覺回饋應用於互動平台之有效性。希望未來本研究所提出之「基於超音波觸覺回饋技術之非接觸式互動平台」，能廣泛運用於各場域，提升使用者體驗與操作效率。

With the development of technology, HCI (human-machine interface) has gradually changed from physical interface to virtual interface. However, the virtual interface’s shortcoming is the lack of physical feedback, which causes malfunction. This work presents a non-contact interactive platform, which creates tactile virtual objects in mid-air by ultrasound haptic feedback technology. This research utilizes two depth cameras to capture hand images and calculates the three-dimensional coordinate position of the hand skeleton, which is used to calculate feedback points. The information of feedback points is transferred to FPGA (Field Programmable Gate Array) from PC (Personal Computer) for generating control signal to ultrasound driving circuit. The ultrasound driving circuit is designed to drive ultrasound transducers. To measure the result of acoustic pressure field around the feedback points, a 3-Dimentional measuring machine is constructed and used. The measure results show that the size of feedback point’s diameter is 10 mm. Furthermore, this work conducts a subjective experiment to get the most sensitive frequency of the mechanoreceptors, which makes people to receive tactile sensation. The experiment results show that the minimum of the vibration detection frequency is 100 Hz in AM (Amplitude Modulation) driving condition, and in STM (Spatio-Temporal Modulation) driving condition, the minimum of the vibration detection frequency is 50 Hz. Both AM and STM driving condition are applied in an interactive platform built by Unity to evaluate the effect of ultrasound haptic feedback. The experiment results which are induced on user experience show the average time taken by the task with the assistance of ultrasound haptic feedback reduces 24.63%, which demonstrates the functionality of the proposed system. It is hoped that in the future, the “Non-Contact Interactive Platform with Ultrasonic Haptic Feedback Technology” proposed by this research can be widely used in fields such as automobile controlling or public utilities operating to improve user experience and operation efficiency.

[1] Parity Innovations. (n.d.). “Floating Touch Display,” Retrieved July 31, 2022, from https://piq.co.jp/about_e.html/
[2] P .Martinez, S .Pirro, C .Vi, and S .Subramanian, “Agency in Mid-air Interfaces,” CHI ‘17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2426-2439, 2017.
[3] IDTechEx. (n.d.). “Haptics 2021-2031: Technologies, Market & Players,” Retrieved July 31, 2022, from https://www.idtechex.com/tw/research-report/haptics-2021-2031-technologies-market-and-players/794
[4] M. Maisto, C. Pacchierotti, F. Chinello, G. Salvietti, A. D. Luca and D. Prattichizzo, “Evaluation of Wearable Haptic Systems for the Fingers in Augmented Reality Applications,” IEEE Transactions on Haptics, vol. 10, pp. 511-522, 2017.
[5] D. Wang, Y. Wang, J. Pang, Z. Wang and B. Zi, “Development and Control of an MR Brake-Based Passive Force Feedback Data Glove,” IEEE Access, vol. 7, pp. 172477-172488, 2019.
[6] J. -B. Chossat, D. K. Y. Chen, Y. -L. Park and P. B. Shull, “Soft Wearable Skin-Stretch Device for Haptic Feedback Using Twisted and Coiled Polymer Actuators,” IEEE Transactions on Haptics, vol. 12, no. 4, pp. 521-532, 2019.
[7] R. Sodhi, I. Poupyrev, M. Glisson and A. Israr, “AIREAL: Interactive tactile experiences in free air,” ACM Transactions on Graphics, vol. 32, no. 4, 2013.
[8] K. Hasegawa and H. Shinoda, “Aerial vibrotactile display based on multiunit ultrasound phased array,” IEEE Trans. Haptics, vol. 11, no. 3, pp. 367–377, 2018.
[9] T. Carter, S. A. Seah, B. Long, B. Drinkwater and S. Subramanian. “UltraHaptics: multi-point mid-air haptic feedback for touch surfaces,” UIST’13: The 26th Annual ACM Symposium on User Interface Software and Technology, pp. 505-514, 2013.
[10] Ultraleap. (n.d.). “Stratos-Explore,” Retrieved July 31, 2022, from
https://www.ultraleap.com/product/stratos-explore/
[11] Ultraleap. (n.d.). “Leap Motion Controller,” Retrieved July 31, 2022, from
https://www.ultraleap.com/product/leap-motion-controller/
[12] Y. Wang, Y. Wu, S. Jung, S. Hoermann, S. Yao and R. W. Lindeman, “Enlarging the Usable Hand Tracking Area by Using Multiple Leap Motion Controllers in VR,” IEEE Sensors Journal, vol. 21, no. 16, pp. 17947-17961, 2021.
[13] J. Connolly, J. Condell, B. O’Flynn, J. T. Sanchez and P. Gardiner, “IMU Sensor-Based Electronic Goniometric Glove for Clinical Finger Movement Analysis,” IEEE Sensors Journal, vol. 18, no. 3, pp. 1273-1281, 2018.
[14] StretchSense. (n.d.). “MoCap Pro,” Retrieved July 31, 2022, from
https://stretchsense.com/
[15] S. Das, Y. Kishishita, T. Tsuji, C. Lowell, K. Ogawa and Y. Kurita, “ForceHand Glove: A Wearable Force-Feedback Glove With Pneumatic Artificial Muscles (PAMs),” IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2416-2423, 2018.
[16] W. Kim, B. Seo, S. Y. Yu and K. -J. Cho, “Deployable Soft Pneumatic Networks (D-PneuNets) Actuator With Dual-Morphing Origami Chambers for High-Compactness,” IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 1262-1269, 2022.
[17] Y. Suzuki and M. Kobayashi, “Air jet driven force feedback in virtual reality,” IEEE Computer Graphics and Applications, vol. 25, no. 1, pp. 44-47, 2005.
[18] H. Barreiro, S. Sinclair and M. A. Otaduy, “Path Routing Optimization for STM Ultrasound Rendering,” IEEE Transactions on Haptics, vol. 13, no. 1, pp. 45-51, 2020.
[19] K. Arun, T. Huang and S. Blostein, “Least-squares fitting of two 3-D point sets,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 698-700, Sep. 1987.
[20] S. Suzuki, S. Inoue, M. Fujiwara, Y. Makino and H. Shinoda, “AUTD3: Scalable Airborne Ultrasound Tactile Display,” IEEE Transactions on Haptics, vol. 14, no. 4, pp. 740-749, 2021.
[21] I. Rakkolainen, E. Freeman, A. Sand, R. Raisamo and S. Brewster, “A Survey of Mid-Air Ultrasound Haptics and Its Applications,” IEEE Transactions on Haptics, vol. 14, no. 1, pp. 2-19, 2021.
[22] R. Takahashi, K. Hasegawa, and H. Shinoda, “Tactile stimulation by repetitive lateral movement of midair ultrasound focus,” IEEE Transactions on Haptics, vol. 13, no. 2, pp. 334-342, 2020.
[23] D. Hajas, D. Pittera, A. Nasce, O. Georgiou and M. Obrist, “Mid-Air Haptic Rendering of 2D Geometric Shapes With a Dynamic Tactile Pointer,” IEEE Transactions on Haptics, vol. 13, no. 4, pp. 806-817, 2020.
[24] M. Barr, “Pulse width modulation,” Embedded System Programming, vol. 14, no. 10, pp. 103-104, 2001.
[25] D. Berg, G. Rebecca, D. Koning, K. Fernando, L. Jochem, P. Kurian, V. Prabhakar, S. Chandramani, T. Belma and V. Wijk, “Challenges in Haptic Communications Over the Tactile Internet,” IEEE Access, vol. 5, pp. 23502-23518, 2017.
[26] A. Marzo, T. Corkett and B. W. Drinkwater, “Ultraino: An Open Phased-Array System for Narrowband Airborne Ultrasound Transmission,” IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 65, no. 1, pp. 102-111, 2018.
[27] Krktl. (n.d.). “Snickerdoodle,” Retrieved July 31, 2022, from
https://krtkl.com/snickerdoodle/
[28] R. S. Johansson and J. R. Flanagan, “Coding and use of tactile signals from the fingertips in object manipulation tasks,” Nature Reviews Neuroscience, vol. 10, no. 5, pp. 345-359, 2009.
[29] W. Frier, D. Pittera, D. Ablart, M. Obrist and S. Subramanian, “Sampling strategy for ultrasonic mid-air haptics,” CHI Conference on Human Factors in Computing Systems Proceedings ACM, 2019.
[30] Ultraleap. (n.d.). “Understanding Latency : Part 2,” Retrieved August 18, 2022, from https://blog.leapmotion.com/understanding-latency-part-2/
[31] T. Hoshi, M. Takahashi, T. Iwamoto and H. Shinoda, “Noncontact Tactile Display Based on Radiation Pressure of Airborne Ultrasound,” IEEE Transactions on Haptics, vol. 3, no. 3, pp. 155-165, 2010.
[32] H. E. Bass, L. C. Sutherland, A. J. Zuckerwar, D. T. Blackstock, and D. Hester, “Atmospheric absorption of sound: Further developments,” The Journal of the Acoustical Society of America, vol. 97, no. 1, pp. 680–683, 1995.
[33] S. Suzuki, M. Fujiwara, Y. Makino and H. Shinoda, “Reducing Amplitude Fluctuation by Gradual Phase Shift in Midair Ultrasound Haptics,” IEEE Transactions on Haptics, vol. 13, no. 1, pp. 87-93, 2020.