現行的智慧型眼鏡裝置常使用觸控板或是語音輸入等方式作為操作互動介面。在智慧眼鏡的應用場景中，對於能空出雙手的非接觸式互動有很大的需求，特別是以影像作為輸入來源的手部追蹤應用。然而現行的智慧型眼鏡受到硬體規格限制，若要實現手部追蹤應用，目前主流是使用有線連接之外接裝置或是增加硬體規格，但此兩種方法均會造成使用者體驗不佳。
本論文提出利用運算卸載技術，使用影像傳輸方式，並搭配使用針對嵌入式裝置優化之手部追蹤函式庫，以設計一可運行於智慧型眼鏡且使用手部追蹤實現的遠端操控系統，使智慧型眼鏡得以實現基於手部追蹤之非接觸式輸入。本論文研究與設計之重點包含：(1)利用運算卸載技術，設計一遠端操控系統，讓智慧型眼鏡使用者獲得相當於在智慧型眼鏡使用手部追蹤作為互動介面之使用者體驗；(2)盡可能降低傳輸與圖形運算所造成之延遲；(3)分析智慧眼鏡使用者之使用者需求，設計一座標轉換演算法，並透過螢幕控制指令，讓使用者能直接對懸浮螢幕進行操控；(4)分析傳輸影像流之解析度與傳輸間隔對整體系統產生的延遲衝擊，盡量在系統能正常運作的前提下減少不必要之延遲。為驗證本系統之設計可用性，本論文設計一測試程式，透過該程式測試遠端操控系統對於點擊事件之座標精準度，以確保本研究開發之遠端操控系統能讓使用者正常控制懸浮螢幕而不發生誤觸等錯誤控制。
本論文主要貢獻為設計與實作一運行於智慧型眼鏡之基於手部追蹤的遠端操控系統，讓使用者能以直覺操控方式控制懸浮螢幕。透過控制Google地圖或第三方應用程式，可應用於導航等使用場景。

Nowadays, smart glasses devices often use touch pads or voice input as the methods for user operations. In the use case scenario of smart glasses, there is a great demand for non-contact interaction, especially for hand tracking interaction. However, the current smart glasses are limited by hardware specifications.

This study aims to design and implement a remote-control mechanism with the desirable feature of hand tracking that can operate on smart glasses to realize non-contact input. This study focuses on (1) designing a remote-control system which using hand tracking as an interactive interface in smart glasses; (2) minimizing the delay incurred by transmission; (3) Designing a landmark conversion algorithm, which allows users to directly control the floating screen.

To verify the usability of the system design, this study tested the accuracy of the remote-control system for click events to ensure that the user can use this system to control the floating screen properly without any unexpected mistake in operation. This study also tested the delay impact of different parameters of the transmitted image stream on the demo system in attempting to reduce the delay overhead.

The main contribution of this study is the design and implementation of a hand-tracking-based remote control mechanism working on smart glasses that allows users to control screen in an intuitive way. By controlling third-party applications, it can be applied to navigation and any other appropriate use case scenario.

[1] Lee Lik-Hang and Pan Hui, "Interaction methods for smart glasses: A survey," IEEE access, vol. 6, pp. 28712-28732, 2018.
[2] Lin Li et al., "Computation offloading toward edge computing," Proceedings of the IEEE, vol. 107, no. 8, pp. 1584-1607, 2019.
[3] Google, "Glass – Glass", https://www.google.com/glass/start/ (accessed on Aug. 21, 2022).
[4] Taiwan Epson, "V11H969154 | Moverio BT-40S | 智慧眼鏡 | 智慧穿戴裝置 | 家用系列 | Epson Taiwan", https://www.epson.com.tw/%E5%AE%B6%E7%94%A8%E7%B3%BB%E5%88%97/%E6%99%BA%E6%85%A7%E7%A9%BF%E6%88%B4%E8%A3%9D%E7%BD%AE/%E6%99%BA%E6%85%A7%E7%9C%BC%E9%8F%A1/Moverio-BT-40S/p/V11H969154 (accessed on Aug. 21, 2022).
[5] Microsoft, "Microsoft HoloLens | Mixed Reality Technology for Business", https://www.microsoft.com/en-us/hololens (accessed on Aug. 21, 2022).
[6] Jorjin, "J-Supporter Series J102 - The AR Smart Glasses for Industry | Jorjin Technologies Inc.｜A Pioneer of the AR Smart Glasses Industry", https://www.jorjin.com/products/ar-smart-glasses/j-supporter/j102/ (accessed on Aug. 21, 2022).
[7] Google, "Android Open Source Project", https://source.android.com/ (accessed on Aug. 21, 2022).
[8] Instruments Recon, "Intel's Recon Jet Pro: Smart glasses for the connected workforce", https://youtu.be/BhBtuNe_8is (accessed on Aug. 21, 2022).
[9] Fraga-Lamas Paula et al., "A review on industrial augmented reality systems for the industry 4.0 shipyard," IEEE access, vol. 6, pp. 13358-13375, 2018.
[10] Dalenogare Lucas Santos et al., "The contribution of Smart Glasses for PSS," Procedia CIRP, vol. 83, pp. 318-323, 2019.
[11] Wiedenmaier Stefan et al., "Augmented reality (AR) for assembly processes design and experimental evaluation," International journal of Human-Computer interaction, vol. 16, no. 3, pp. 497-514, 2003.
[12] Google, "Android – Google 行動服務", https://www.android.com/intl/zh-TW_tw/gms/ (accessed on Aug. 21, 2022).
[13] Research Grand View, "Augmented Reality Market Size & Share Report, 2030", https://www.grandviewresearch.com/industry-analysis/augmented-reality-market (accessed on Aug. 21, 2022).
[14] Kim Sun Kyung et al., "Adaptation of extended reality smart glasses for core nursing skill training among undergraduate nursing students: Usability and feasibility study," Journal of medical Internet research, vol. 23, no. 3, p. e24313, 2021.
[15] Bryson Steve T, "Effects of lag and frame rate on various tracking tasks," in Stereoscopic Displays and Applications IV, 1993, vol. 1915: SPIE, pp. 155-166.
[16] Schulzrinne H., "RFC 2326 - Real Time Streaming Protocol (RTSP)", https://datatracker.ietf.org/doc/html/rfc2326 (accessed on Aug. 21, 2022).
[17] Schulzrinne H., "RFC 7826 - Real-Time Streaming Protocol Version 2.0", https://datatracker.ietf.org/doc/html/rfc7826 (accessed on Aug. 21, 2022).
[18] Chiu Jih-Ching et al., "Design of Multidimension-media Streaming Protocol Based on RTSP," in 2020 International Computer Symposium (ICS), 2020: IEEE, pp. 341-347.
[19] Parmar H., "Adobe’s Real Time Messaging Protocol", https://web.archive.org/web/20171025123230/http://wwwimages.adobe.com/www.adobe.com/content/dam/acom/en/devnet/rtmp/pdf/rtmp_specification_1.0.pdf (accessed on Aug. 21, 2022).
[20] Thornburgh M., "RFC 7425 - Adobe's RTMFP Profile for Flash Communication", https://datatracker.ietf.org/doc/html/rfc7425 (accessed on Aug. 21, 2022).
[21] Siekkinen Matti et al., "A first look at quality of mobile live streaming experience: the case of periscope," in Proceedings of the 2016 Internet Measurement Conference, 2016, pp. 477-483.
[22] Rey Marina del, "RFC 793 - Transmission Control Protocol", https://datatracker.ietf.org/doc/html/rfc793 (accessed on Aug. 21, 2022).
[23] Larsen Steen et al., "Architectural breakdown of end-to-end latency in a TCP/IP network," International journal of parallel programming, vol. 37, no. 6, pp. 556-571, 2009.
[24] Google, "Google Chrome 遠端桌面", https://remotedesktop.google.com/support/ (accessed on Aug. 21, 2022).
[25] TeamViewer, "TeamViewer", https://www.teamviewer.com/ (accessed on Aug. 21, 2022).
[26] GmbH AnyDesk Software, "AnyDesk", https://anydesk.com/ (accessed on Aug. 21, 2022).
[27] Alliance Wi-Fi, "Miracast | Wi-Fi Alliance ", https://www.wi-fi.org/discover-wi-fi/miracast (accessed on Aug. 21, 2022).
[28] Genymobile, "GitHub - Genymobile/scrcpy: Display and control your Android device", https://github.com/Genymobile/scrcpy (accessed on Aug. 21, 2022).
[29] Abdul Anwar Mohammad, "scrcpy-android · GitLab", https://gitlab.com/las2mile/scrcpy-android (accessed on Aug. 21, 2022).
[30] team OpenCV, "Home - OpenCV", https://opencv.org/ (accessed on Aug. 21, 2022).
[31] Google, "Home - mediapipe", https://google.github.io/mediapipe/ (accessed on Aug. 21, 2022).
[32] Google, "Hands - mediapipe", https://google.github.io/mediapipe/solutions/hands.html (accessed on Aug. 21, 2022).
[33] Zhang Fan et al., "MediaPipe Hands: On-device Real-time Hand Tracking," arXiv preprint arXiv:2006.10214, 2020.
[34] Smith Alvy Ray, "Alpha and the history of digital compositing", https://www.cs.princeton.edu/courses/archive/spr05/cos426/papers/smith95c.pdf (accessed on Aug. 21, 2022).
[35] Google, "BitmapFactory | Android Developers", https://developer.android.com/reference/android/graphics/BitmapFactory (accessed on Aug. 21, 2022).
[36] Union International Telecommunication, "BT.601 : Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios", https://www.itu.int/rec/R-REC-BT.601/ (accessed on Aug. 21, 2022).
[37] Google, "Camera | Android Developers", https://developer.android.com/reference/android/hardware/Camera (accessed on Aug. 21, 2022).
[38] handy, "图解YU12、I420、YV12、NV12、NV21、YUV420P、YUV420SP、YUV422P、YUV444P的区别", https://blog.csdn.net/byhook/article/details/84037338 (accessed on Aug. 21, 2022).
[39] Google, "ServerSocket | Android Developers", https://developer.android.com/reference/java/net/ServerSocket (accessed on Aug. 21, 2022).
[40] Google, "Message | Android Developers", https://developer.android.com/reference/android/os/Message (accessed on Aug. 21, 2022).
[41] Google, "MediaProjection | Android Developers", https://developer.android.com/reference/android/media/projection/MediaProjection (accessed on Aug. 21, 2022).