本論文介紹了一個整合 ROS2 進行資料傳遞的數位孿生系統，此系統利用了 Unity 3D 遊戲引擎來模擬和控制虛擬環境中的機器人，並精心設計了一套多功能控制面板，實現了車輛和機械手臂的多模式操作，允許使用者在手動操作和程式自動控制之間自由切換，此外，系統整合了一系列視覺化的工具，例如利用光達進行環境的掃描、應用 YOLO 進行即時的物體偵測以及 QR code 的辨識，這些功能顯著增強了機器人對其操作環境的感知和互動能力，而通過 FABRIK 演算法的應用，還能控制機械手臂使其流暢地移動到指定的目標地點，系統還特別添加了地圖物件的生成、儲存與讀取功能，讓使用者能夠根據具體需求設計和重用地圖，極大地提升了系統的適用性與靈活性。整體而言，本論文不僅展示了 ROS2 與 Unity 3D 的技術整合，也為使用者提供了一個可根據特定需求進行擴展的框架。

This paper introduces a digital twin system that uses ROS2 for data transmission, alongside the Unity 3D game engine to simulate and control robots within a virtual environment. We have developed a multifunctional dashboard that supports different operations for vehicles and robotic arms, making it easy to switch between manual and automated controls. Moreover, the system integrates a series of visualization tools, including LiDAR for scanning environments, real-time object detection using YOLO, and QR code recognition—enhancements that significantly improve the robots’ ability to perceive and interact with their surroundings. We also implemented the FABRIK algorithm, which facilitates smooth and precise movements of the robotic arms to destination. Additionally, the system features the ability to generate, save, and load map objects, thus permitting users to customize and reuse maps based on specific needs, thereby greatly increasing the system’s versatility and adaptability. Overall, this study not only shows the integration of ROS2 and Unity 3D technologies but also offers a flexible framework that can be tailored to meet specific user requirements.

[1] Jessica Van Brummelen, Marie O’Brien, Dominique Gruyer, and Homayoun Najjaran. "Autonomous vehicle perception: The technology of today and tomorrow." Transportation research part C: emerging technologies 89 (2018): 384-406.
[2] Michele D. Simoni, Erhan Kutanoglu, and Christian G. Claudel. "Optimization and analysis of a robot-assisted last mile delivery system." Transportation Research Part E: Logistics and Transportation Review 142 (2020): 102049.
[3] Paolo Rocco. "Stability of PID control for industrial robot arms." IEEE transactions on robotics and automation 12.4 (1996): 606-614.
[4] Robin R Murphy. Introduction to AI robotics. MIT press, 2019.
[5] I. Buyuksalih, S. Bayburt, G. Buyuksalih, A. P. Baskaraca, H. Karim, and A. A. Rahman. "3D modelling and visualization based on the unity game engine–advantages and challenges." ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 4 (2017): 161-166.
[6] Marco Guidali, et al. "A robotic system to train activities of daily living in a virtual environment." Medical & biological engineering & computing 49 (2011): 1213-1223.
[7] Andrea Bonci, Francesco Gaudeni, Maria Cristina Giannini, and Sauro Longhi. "Robot Operating System 2 (ROS2)-Based Frameworks for Increasing Robot Autonomy: A Survey." Applied Sciences 13.23 (2023): 12796.
[8] Lennart Puck, et al. "Performance evaluation of real-time ros2 robotic control in a time-synchronized distributed network." 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE). IEEE, 2021.
[9] Christopher Crick, Graylin Jay, Sarah Osentoski, Benjamin Pitzer, and Odest Chadwickle Jenkins. "Rosbridge: Ros for non-ros users." Robotics Research: The 15th International Symposium ISRR. Springer International Publishing, 2017.
[10] Yuya Maruyama, Shinpei Kato, and Takuya Azumi. "Exploring the performance of ROS2." Proceedings of the 13th international conference on embedded software. 2016.
[11] Ian Fette, and Alexey Melnikov. The websocket protocol. No. rfc6455. 2011.
[12] Zhaolin Chen. Performance analysis of ros 2 networks using variable quality of service and security constraints for autonomous systems. Diss. Monterey, CA; Naval Postgraduate School, 2019.
[13] Aaron D'Souza, Sethu Vijayakumar, and Stefan Schaal. "Learning inverse kinematics." Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01CH37180). Vol. 1. IEEE, 2001.
[14] ReachUI. https://docs.michsky.com/docs/reach-ui/
[15] Andreas Aristidou, and Joan Lasenby. "FABRIK: A fast, iterative solver for the Inverse Kinematics problem." Graphical Models 73.5 (2011): 243-260.
[16] Peiyuan Jiang, et al. "A Review of Yolo algorithm developments." Procedia computer science 199 (2022): 1066-1073.
[17] Sumit Tiwari. "An introduction to QR code technology." 2016 international conference on information technology (ICIT). IEEE, 2016.
[18] Erik Pasternak, Rachel Fenichel, and Andrew N. Marshall. "Tips for creating a block language with blockly." 2017 IEEE blocks and beyond workshop (B&B). IEEE, 2017.
[19] Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. "Reinforcement learning: A survey." Journal of artificial intelligence research 4 (1996): 237-285.
[20] Photon Fusion. https://www.photonengine.com/zh-tw/fusion