當前製造業中加工機台之運動控制程式編輯仍以文字程式為大宗，文字程式不僅不易於修改及除錯，對使用者也有相當高的專業門檻。為改善上述問題，本研究於PC-Based控制器中，以自行研發之視覺化程式設計語言(Visual Programming Language, VPL)及創新之人機互動(Human Machine Interaction, HMI)方式，實現可應用於智慧製造場域之人機協作(Human Robot Collaboration, HRC)控制系統。
本系統之VPL以流程圖的概念進行設計，將智慧製造場域中機械設備控制所需功能(如六軸機械手臂運動控制、生產線周邊設備控制、巨集程式等)包裝為不同功能圖示，以其作為編輯生產流程之主要元件，並透過易於理解的連結線表達程式執行之順序。藉由上述設計，讓使用者不必費時學習困難的程式語法或牢記繁雜的運動控制指令，只需透過拖曳、排列、連結及設定不同功能圖示，即可完成製造場域設備控制程式之編輯，不僅有效降低編輯製程之程式能力需求，亦同時提升控制程式之可讀性及可維護性。
VPL以混合式編譯器作為執行核心，此混合式編譯器之架構由VPL編譯器與直譯器組成，不僅可執行本研究自訂之Python運動控制程式API，未來亦可引用各種以Python語法撰寫的運動函式庫及製程輔助功能(如物聯網應用、影像識別應用、AI應用等)，使系統功能具備良好的可擴充性，有助於將其導入不同應用情境中。
此外，為了進一步降低本系統之操作門檻，本系統以深度學習、電腦視覺、語音識別、眼動追蹤(eye tracking)等技術，開發基於手勢、語音、視線等人機互動功能，並結合上述功能提出創新人機互動VPL編程機制以及智慧人機協作機械手臂軌跡教導機制。藉由上述機制之輔助，本系統讓使用者可透過較為直覺的方式實現生產線控制程式之編輯，或者完成機械手臂運動路徑之教導。為了驗證此人機協作控制系統之功能，本研究透過實驗模擬完整的智慧製造場域製程設計，將VPL與上述HMI、HRC兩項機制妥善結合，不僅驗證本系統架構之擴充性，亦證明本系統具備導入業界之實用性及發展性。

At present, the editing of motion control programs for processing machines in the manufacturing industry is still text-based programming. Text program is difficult to modify and debug, and has a high professional threshold for users. To resolve the above problems, the self-developed Visual Programming Language (VPL) and the innovative Human-Machine Interaction (HMI) method is adopted in the PC-Based controller to realize the application of Human-Robot Collaboration (HRC) control system in the smart manufacturing field.
The VPL of this system is designed with the concept of the flowchart. The functions required for equipment control in the smart manufacturing field (such as six-axis robot arm motion control, peripheral equipment in the production line control, macros, etc.) are packaged into different functional icons. Functional icons are the main components of editing the production process, and easy-to-understand lines show the sequence of program execution. With the above design users don't need to learn difficult programming technique or remember complicated motion control commands. Users only need to drag, arrange, link, and set different functional icons to complete the program. The requirement of programming ability to edit the program is effectively reduced, and the readability and maintainability of this system are improved with the proposed VPL.
A hybrid compiler is used as the core of VPL's execution, which is composed of a VPL compiler and an interpreter. This hybrid compiler can execute the Python motion control API customized for this study and reference various motion function libraries and process assistance functions written in Python syntax (e.g., Internet of Things applications, image recognition applications, cloud applications, etc.) in the future.
In addition, to decrease the operating threshold of the system, the system adopts deep learning, computer vision, speech recognition, eye tracking, and other technologies to develop human-computer interaction functions based on hand gestures, voice, and sight. With the functions above, the "innovative human-machine interaction VPL programming mechanism" and the "smart human-machine collaboration robot arm trajectory teaching mechanism" are proposed. With the above mechanisms, users can edit the production line control program or complete the motion path teaching of the robot arm in a more intuitive way.
Two experiments are taken to simulate the process planning of smart manufacturing to verify the functions of this human-robot collaboration control system. The experiments verify that the system architecture is suitable for expanding various innovative applications and prove that the system has practicability to be applied into the industry.

[1] M. Erwig, K. Smeltzer, and X. Wang, "What is a visual language?," Journal of Visual Languages & Computing, vol. 38, pp. 9-17, 2017.
[2] T. Booth and S. Stumpf, "End-user experiences of visual and textual programming environments for Arduino," in International symposium on end user development, 2013: Springer, pp. 25-39.
[3] D. Weintrop and U. Wilensky, "Comparing block-based and text-based programming in high school computer science classrooms," ACM Transactions on Computing Education (TOCE), vol. 18, no. 1, pp. 1-25, 2017.
[4] T. W. Price and T. Barnes, "Comparing textual and block interfaces in a novice programming environment," in Proceedings of the eleventh annual international conference on international computing education research, 2015, pp. 91-99.
[5] D. Weintrop, D. C. Shepherd, P. Francis, and D. Franklin, "Blockly goes to work: Block-based programming for industrial robots," in 2017 IEEE Blocks and Beyond Workshop (B&B), 2017: IEEE, pp. 29-36.
[6] F. Riedo, M. Chevalier, S. Magnenat, and F. Mondada, "Thymio II, a robot that grows wiser with children," in 2013 IEEE Workshop on Advanced Robotics and its Social Impacts, 2013: IEEE, pp. 187-193.
[7] J. M. R. Corral, I. Ruiz-Rube, A. C. Balcells, J. M. Mota-Macías, A. Morgado-Estévez, and J. M. Dodero, "A study on the suitability of visual languages for non-expert robot programmers," IEEE access, vol. 7, pp. 17535-17550, 2019.
[8] S. Dragule, T. Berger, C. Menghi, and P. Pelliccione, "A survey on the design space of end-user-oriented languages for specifying robotic missions," Software and Systems Modeling, pp. 1-36, 2021.
[9] E. Pasternak, R. Fenichel, and A. N. Marshall, "Tips for creating a block language with blockly," in 2017 IEEE Blocks and Beyond Workshop (B&B), 2017: IEEE, pp. 21-24.
[10] D. Weintrop, D. C. Shepherd, P. Francis, and D. Franklin, "Blockly goes to work: Block-based programming for industrial robots," in 2017 IEEE Blocks and Beyond Workshop (B&B), 2017: IEEE, pp. 29-36.
[11] 范智翔. "機械手臂之圖形化運動控制介面及編譯器發展與實現." 成功大學製造資訊與系統研究所學位論文 (2020): 1-81.
[12] A. V. Aho, "Compilers principles," Techniques, and Tools, 1986.
[13] 陳響亮, 蔣承儒, 陳文泉, 李峰吉, and 李桂銘, "工具機數控程式編譯器與應用," 機械工業雜誌, no. 422, pp. 79-88, 2018.
[14] I. El Makrini et al., "Working with walt: How a cobot was developed and inserted on an auto assembly line," IEEE Robotics & Automation Magazine, vol. 25, no. 2, pp. 51-58, 2018.
[15] Y. R. Oh, J. S. Yoon, J. H. Park, M. Kim, and H. K. Kim, "A name recognition based call-and-come service for home robots," IEEE Transactions on Consumer Electronics, vol. 54, no. 2, pp. 247-253, 2008.
[16] M. Jain et al., "Object Detection and Gesture Control of Four-Wheel Mobile Robot," in 2019 International Conference on Communication and Electronics Systems (ICCES), 2019: IEEE, pp. 303-308.
[17] M. Yasen and S. Jusoh, "A systematic review on hand gesture recognition techniques, challenges and applications," PeerJ Computer Science, vol. 5, p. e218, 2019.
[18] P. P. Sarker, F. Abedin, and F. N. Shimim, "R3Arm: Gesture controlled robotic arm for remote rescue operation," in 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), 2017: IEEE, pp. 428-431.
[19] R. Mardiyanto, M. F. R. Utomo, D. Purwanto, and H. Suryoatmojo, "Development of hand gesture recognition sensor based on accelerometer and gyroscope for controlling arm of underwater remotely operated robot," in 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA), 2017: IEEE, pp. 329-333.
[20] M. Ababneh, H. Sha'ban, D. AlShalabe, D. Khader, H. Mahameed, and M. AlQudimat, "Gesture Controlled Mobile Robotic Arm for Elderly and Wheelchair People Assistance Using Kinect Sensor," in 2018 15th International Multi-Conference on Systems, Signals & Devices (SSD), 2018: IEEE, pp. 636-641.
[21] M. M. F. M. Fareed, Q. I. Akram, S. B. A. Anees, and A. H. Fakih, "Gesture based wireless single-armed robot in cartesian 3D space using Kinect," in 2015 Fifth International Conference on Communication Systems and Network Technologies, 2015: IEEE, pp. 1210-1215.
[22] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[23] Y. Zhu, S. Jiang, and P. B. Shull, "Wrist-worn hand gesture recognition based on barometric pressure sensing," in 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), 2018: IEEE, pp. 181-184.
[24] J. O. Pinzón-Arenas, R. Jiménez-Moreno, and J. E. Herrera-Benavides, "Convolutional neural network for hand gesture recognition using 8 different emg signals," in 2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA), 2019: IEEE, pp. 1-5.
[25] E. A. Chung and M. E. Benalcázar, "Real-time hand gesture recognition model using deep learning techniques and EMG signals," in 2019 27th European Signal Processing Conference (EUSIPCO), 2019: IEEE, pp. 1-5.
[26] K. Luan and T. Matsumaru, "Dynamic Hand Gesture Recognition for Robot Arm Teaching based on Improved LRCN Model," in 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2019: IEEE, pp. 1269-1274.
[27] Y.-T. Tsai and K.-Y. Chen, "The autonomous control of the Service-Oriented robot arms based on the artificial intelligence object identifying technology," in 2019 International Conference on Fuzzy Theory and Its Applications (iFUZZY), 2019: IEEE, pp. 110-113.
[28] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
[29] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: Optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
[30] R. Mitkov, The Oxford handbook of computational linguistics. Oxford University Press, 2004.
[31] D. Isaacs, "A comparison of the network speech recognition and distributed speech recognition systems and their effect on speech enabling mobile devices," University of Cape Town, 2010.
[32] R. Gomez, K. Nakamura, T. Mizumoto, and K. Nakadai, "Compensating changes in speaker position for improved voice-based human-robot communication," in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), 2015: IEEE, pp. 977-982.
[33] P. X. Liu, A. D. Chan, R. Chen, K. Wang, and Y. Zhu, "Voice based robot control," in 2005 IEEE International Conference on Information Acquisition, 2005: IEEE, p. 5 pp.
[34] S. P. Rasal, "Voice controlled robotic vehicle," International Journal of New Trends in Electronics and Communication, vol. 2, no. 1, pp. 28-30, 2014.
[35] Microsoft Docs, "評估和改善自訂語音精確度." [Online]. Available: https://docs.microsoft.com/zh-tw/azure/cognitive-services/speech-service/how-to-custom-speech-evaluate-data
[36] V. Këpuska and G. Bohouta, "Comparing speech recognition systems (Microsoft API, Google API and CMU Sphinx)," Int. J. Eng. Res. Appl, vol. 7, no. 03, pp. 20-24, 2017.
[37] Z. Zhu and Q. Ji, "Eye gaze tracking under natural head movements," in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005, vol. 1: IEEE, pp. 918-923.
[38] Tobii pro, "眼動儀是怎樣工作的?" [Online]. Available: https://www.tobiipro.cn/learn-and-support/learn/eye-tracking-essentials/how-do-tobii-eye-trackers-work/
[39] K. Fujii, A. Salerno, K. Sriskandarajah, K.-W. Kwok, K. Shetty, and G.-Z. Yang, "Gaze contingent cartesian control of a robotic arm for laparoscopic surgery," in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013: IEEE, pp. 3582-3589.
[40] S. Dziemian, W. W. Abbott, and A. A. Faisal, "Gaze-based teleprosthetic enables intuitive continuous control of complex robot arm use: Writing & drawing," in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), 2016: IEEE, pp. 1277-1282.
[41] ezLabel, [online] Available: https://www.aicreda.com.
[42] C.-K. Tseng, C.-C. Liao, P.-C. Shen, and J.-I. Guo, "Using C3D to Detect Rear Overtaking Behavior," in 2019 IEEE International Conference on Image Processing (ICIP), 2019: IEEE, pp. 151-154.
[43] iThome, "標註AI訓練用影片資料有新方法，1千種內嵌物件都能半自動化標記，比傳統流程快15倍." [Online]. Available: https://www.ithome.com.tw/news/131088
[44] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, "The pascal visual object classes (voc) challenge," International journal of computer vision, vol. 88, no. 2, pp. 303-338, 2010.
[45] D. M. Powers, "Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation," arXiv preprint arXiv:2010.16061, 2020.
[46] R. Padilla, S. L. Netto, and E. A. da Silva, "A survey on performance metrics for object-detection algorithms," in 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), 2020: IEEE, pp. 237-242.
[47] M. D. Dunn, Pro Microsoft Speech Server 2007: developing speech enabled applications with. NET. Springer, 2007.
[48] W. Kaczmarek, J. Panasiuk, S. Borys, and P. Banach, "Industrial robot control by means of gestures and voice commands in off-line and on-line mode," Sensors, vol. 20, no. 21, p. 6358, 2020.
[49] Tobii Technology, "This is eye-tracking." [Online]. Available: https://www.tobii.com/group/about/this-is-eye-tracking/
[50] Tobii pro, "眼動儀校準過程中發生了什麼?" [Online]. Available: https://www.tobiipro.cn/learn-and-support/learn/eye-tracking-essentials/what-happens-during-the-eye-tracker-calibration/