近幾年AR在各個產業之應用相繼被提出，且都表明能夠有效提升工作效率，而在工業領域中常見的AR相關應用研究以維護最為熱門，因維護為產品生命週期中不可避免的部分，且主要為人工手動操作，導入AR可使人員更加了解現場狀況，避免因資訊不足而做出錯誤的判斷。但目前對於將AR導入工業場域尚存在許多問題需要克服，且隨著工業場域中的產線設備複雜化，所產生的數據也隨之增加，使得AR硬體於相關應用中所面臨的運算效能不足問題日益浮現。
為此，本研究提出「基於5G與霧運算之擴增實境監控與遠端協作系統」，並以CNC三軸加工機為應用案例，以自行實現之AR演算法進行拆分，將標記識別、影像校正、影像渲染等高能耗運算部署於霧節點中，以5G網路確保AR設備與霧節點間的影像傳輸延遲，藉此解決AR硬體運算效能不足問題，並提升AR系統的可擴展性。
為使維修任務順利進行，利用霧節點作為中繼站，將CNC感測數據、遠端維修平台等應用之數據整合於AR應用中，現場人員以CNC感測數據狀態可視化數據判斷設備異常，而遠端專家則透過遠端維修平台，在了解CNC設備異常後，以語音通話輔以虛擬標註之方式指導現場人員，使現場人員快速地透過遠端專家指導排除CNC設備異常。
將AR運算拆分至霧節點後，以5G網路進行影像傳輸，其傳輸延遲低於20毫秒，且相較於AR設備本地端之影像處理運算效能至少提升28％，以此為基礎開發之監控與遠端維修系統，系統以209毫秒之頻率更新AR可視化數據，使現場人員可即時發現數據異常，且經測試後發現對於不熟悉CNC維護任務之使用者，有著30％之維修效率提升，證明本研究之AR遠端維修系統架構於工業場域之效益與可行性，並為AR導入工業應用提供一種解決方案。

By deploying the image processing computation in the AR system in the fog node and using 5G network to ensure the image transmission delay between AR equipment and the fog node, we can solve the problem of insufficient AR hardware computing performance and enhance the scalability of AR system. We also develop AR monitoring and remote maintenance system based on this, using the fog node as a relay station to integrate and analyze data from CNC machines and remote maintenance platforms in the AR system. It enables field personnel to monitor CNC status and receive maintenance instructions from remote experts through AR devices, so that field personnel can quickly find CNC abnormalities and eliminate them immediately to reduce the additional cost caused by temporary downtime. This study uses 5G for image transmission to ensure that the latency is less than 20 ms and the image processing performance is at least 28% higher than that of the local side of the AR device. The monitoring system developed on this basis updates the AR visualization data at a frequency of 209 ms, and the maintenance assistance test conducted with this system shows that it achieves 30% improvement in maintenance efficiency for users who are not familiar with CNC maintenance tasks. These results demonstrate the effectiveness and feasibility of the AR remote maintenance system architecture in industrial settings and provide a solution for the introduction of AR into industrial applications.

[1] Z. Yusuf, V. Lukic, J. Heppelmann, C. Melrose, N. Ravi, U. Gill, A. Rosello, " Unleashing the Power of Data with IoT and Augmented Reality," Boston Consulting Group and PTC Inc., Boston, Massachusetts, United States, 2020.
[2] A.-O. Arwa and P. Master, "Usability Principles for Augmented Reality Applications in Education," IJCSNS, vol. 22, no. 1, pp. 49-54, 2022.
[3] R. Monterubbianesi, V. Tosco, F. Vitiello, G. Orilisi, F. Fraccastoro, A. Putignano, G. Orsini, "Augmented, Virtual and Mixed Reality in Dentistry: A Narrative Review on the Existing Platforms and Future Challenges," Applied Sciences, vol. 12, no. 2, 2022.
[4] B. W. Zatezalo, "Augmented reality to the rescue during the coronavirus pandemic.", https://www.asml.com/en/news/stories/2020/augmented-reality-to-the-rescue-coronavirus?fbclid=IwAR3ktX4UEeaMAZbwg0LpbCbWQEx0a9yc-JZMN-h-iU3jkBPWrWIIvWdhix4&ppc=SMC-1524, 訪問於2022/09/04.
[5] V. Reljić, I. Milenković, S. Dudić, J. Šulc, and B. Bajči, "Augmented Reality Applications in Industry 4.0 Environment," Applied Sciences, vol. 11, no. 12, 2021.
[6] D. Mourtzis, V. Zogopoulos, and E. Vlachou, "Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry," Procedia CIRP, vol. 63, pp. 46-51, 2017.
[7] G. M. Santi, A. Ceruti, A. Liverani, and F. Osti, "Augmented Reality in Industry 4.0 and Future Innovation Programs," Technologies, vol. 9, no. 2, 2021.
[8] S. Sinha, "State of IoT—Summer 2021 and the updated Cellular IoT & Low-Power Wide-Area (LPWA) Market Tracker (Q3 2021)," Iot Analytic, Germany, 2021.
[9] D. Mourtzis, V. Siatras, and J. Angelopoulos, "Real-Time Remote Maintenance Support Based on Augmented Reality (AR)," Applied Sciences, vol. 10, no. 5, 2020.
[10] Apple Inc., "Augmented Reality – Apple Developer", https://developer.apple.com/augmented-reality/, 訪問於2022/02/22.
[11] Google Inc., "Overview of ARCore and supported development environments", https://developers.google.com/ar/develop, 訪問於2022/02/22.
[12] PTC Inc., "Getting Started | VuforiaLibrary", https://library.vuforia.com/, 訪問於2022/02/22.
[13] D. K. Baroroh, C.-H. Chu, and L. Wang, "Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence," Journal of Manufacturing Systems, vol. 61, pp. 696-711, 2021.
[14] M. Torres Vega , C. Liaskos, S. Abadal, E. Papapetrou, A. Jain, B. Mouhouche, G. Kalem, S. Ergüt, M. Mach, T. Sabol, A. Cabellos-Aparicio, C. Grimm, F. De Turck and J. Famaey, "Immersive Interconnected Virtual and Augmented Reality: A 5G and IoT Perspective," Journal of Network and Systems Management, vol. 28, no. 4, pp. 796-826, 2020.
[15] A. Mishra, S. Karmakar, A. Bose, and A. Dutta, "Design and development of IoT-based latency-optimized augmented reality framework in home automation and telemetry for smart lifestyle," Journal of Reliable Intelligent Environments, vol. 6, no. 3, pp. 169-187, 2020.
[16] T. Schmiedinger, M. Petke, L. v. Czettritz, B. Wohlschläger, and M. Adam, "Augmented Reality as a tool for providing informational content in different production domains," Procedia Manufacturing, vol. 45, pp. 423-428, 2020.
[17] R. B. Patil, B. S. Kothavale, L. Y. Waghmode, and S. G. Joshi, "Reliability analysis of CNC turning center based on the assessment of trends in maintenance data," International Journal of Quality & Reliability Management, vol. 34, no. 9, pp. 1616-1638, 2017.
[18] R. B. Patil, B. S. Kothavale, and R. S. Powar, "Effects of Human and Organizational Factors on the Reliability and Maintainability of CNC Turning Center," in Reliability, Safety and Hazard Assessment for Risk-Based Technologies, (Lecture Notes in Mechanical Engineering, ch. Chapter 64, pp. 751-764, 2020.
[19] D. Mourtzis, A. Vlachou, and V. Zogopoulos, "Cloud-Based Augmented Reality Remote Maintenance Through Shop-Floor Monitoring: A Product-Service System Approach," Journal of Manufacturing Science and Engineering, vol. 139, no. 6, 2017.
[20] D. Mourtzis, J. Angelopoulos, and N. Panopoulos, "Intelligent Predictive Maintenance and Remote Monitoring Framework for Industrial Equipment Based on Mixed Reality," Frontiers in Mechanical Engineering, vol. 6, 2020.
[21] C. Kollatsch and P. Klimant, "Efficient integration process of production data into Augmented Reality based maintenance of machine tools," Production Engineering, vol. 15, no. 3-4, pp. 311-319, 2021.
[22] H. Heinonen, S. Siltanen, and P. Ahola, "Information Design for Small Screens: Toward Smart Glass Use in Guidance for Industrial Maintenance," IEEE Transactions on Professional Communication, vol. 64, 4, pp. 407-426, 2021.
[23] M. Aazam, S. Zeadally, and K. A. Harras, "Deploying Fog Computing in Industrial Internet of Things and Industry 4.0," IEEE Transactions on Industrial Informatics, vol. 14, no. 10, pp. 4674-4682, 2018.
[24] 呂冠廷, "三軸加工機之通用OPC UA訊息模型設計及行動化監控系統實現", 國立成功大學製造資訊與系統研究所碩士論文, 臺南市, 中華民國, 2020.
[25] T. M. Fernandez-Carames, P. Fraga-Lamas, M. Suarez-Albela, and M. Vilar-Montesinos, "A Fog Computing and Cloudlet Based Augmented Reality System for the Industry 4.0 Shipyard," Sensors (Basel), vol. 18, no. 6, Jun 2 2018.
[26] OpenCV.org, "About - OpenCV", https://opencv.org/about/, 訪問於2022/02/22.
[27] Khronos Group Inc., "OpenGL Platform & OS Implementations", https://www.opengl.org/archives/documentation/implementations/, 訪問於2022/02/22.
[28] S. Verykokou, A.-M. Boutsi, and C. Ioannidis, "Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering," Applied Sciences, vol. 11, no. 18, 2021.
[29] Z. Oufqir, A. El Abderrahmani, and K. Satori, "From Marker to Markerless in Augmented Reality," in Embedded Systems and Artificial Intelligence, (Advances in Intelligent Systems and Computing, 2020, ch. Chapter 57, pp. 599-612.
[30] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas, and M. J. Marín-Jiménez, "Automatic generation and detection of highly reliable fiducial markers under occlusion," Pattern Recognition, vol. 47, no. 6, pp. 2280-2292, 2014.
[31] M. F. Sani and G. Karimian, "Automatic navigation and landing of an indoor AR. drone quadrotor using ArUco marker and inertial sensors," 2017 International Conference on Computer and Drone Applications (IConDA), pp. 102-107, 2017.
[32] C. Shewell, J. Medina-Quero, M. Espinilla, C. Nugent, M. Donnelly, and H. Wang, "Comparison of fiducial marker detection and object interaction in activities of daily living utilising a wearable vision sensor," International Journal of Communication Systems, vol. 30, no. 5, 2017.
[33] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, "ORB: An efficient alternative to SIFT or SURF," International conference on computer vision, pp. 2564-2571, 2011.
[34] S. Shin, J. Yeom, Y. Son, and K.-S. Choi, "Implementation of Real-time Computer-generated Integral Imaging System for Head Mounted Display using OpenGL Shading Language," 2019 International Conference on Information and Communication Technology Convergence (ICTC), pp. 232-236, 2019.
[35] C. M. Costa, G. Veiga, A. Sousa, L. Rocha, A. A. Sousa, R. Rodrigues and U. Thomas, "Modeling of video projectors in OpenGL for implementing a spatial augmented reality teaching system for assembly operations," 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), pp. 1-8, 2019.
[36] Amy Tabb, "Converting OpenCV cameras to OpenGL cameras. " https://amytabb.com/tips/tutorials/2019/06/28/OpenCV-to-OpenGL-tutorial-essentials/, 訪問於2022/07/29.
[37] M. Iorga, L. Feldman, R. Barton, M. J. Martin, N. Goren, and C. Mahmoudi, "Fog Computing Conceptual Model: Recommendations of the National Institute of Standards and Technology," National Institute of Standards and Technology, Gaithersburg, Maryland , United States, 2018.
[38] M. Peter and G. Timothy, "The NIST Definition of Cloud Computing: Recommendations of the National Institute of Standards and Technology," National Institute of Standards and Technology, Gaithersburg, Maryland , United States, 2011.
[39] L. Zhu and C. Liu, "Recent progress of chatter prediction, detection and suppression in milling," Mechanical Systems and Signal Processing, vol. 143, 2020.
[40] 經濟部標準檢驗局, "安全顏色－一般性事項", 中華民國國家標準, 臺北市, 中華民國, 2018.
[41] M. Quandt and M. Freitag, "A Systematic Review of User Acceptance in Industrial Augmented Reality," Frontiers in Education, vol. 6, 2021.
[42] CISCO, " 5 things to know about Wi-Fi 6/6E and Private 5G ", https://www.cisco.com/c/m/en_us/solutions/enterprise-networks/802-11ax-solution/nb-06-5-things-WiFi6-5G-infograph-cte-en.html, 訪問於2022/07/16.
[43] 林韋丞, "以OPC UA為基之智慧防碰撞機械手臂協作系統實現", 國立成功大學製造資訊與系統研究所碩士論文, 臺南市, 中華民國, 2020.
[44] 朱宏國, 姚智原, 賴祐吉, "OpenGL 3D 繪圖互動程式設計", 旗標科技股份有限公司, 臺北市, 中華民國, 2018.
[45] 曹曉明, "Unity 3D 遊戲設計與開發", 清華大學出版社, 北京市, 中華人民共和國, 2019.