本論文實作一套建構於 ROS 2 架構下的多機器人協作與數位孿生系統，目標是實現多台自主移動機器人間的協同作業與模擬驗證︒系統整合四大模組：中央控制︑車體導航控制︑機械手臂控制與視覺辨識，並透過 ROS 2 平台進行即時資料傳輸與任務協調︒在開發過程中，本論文因只使用機械手臂上的單一深度相機做物件定位，也就是使用第一人稱視角（First-Person View, FPV）操控機器人︒相較於傳統的第三人稱視角（third-person view）系統，FPV 操作在導航與物體夾取任務上具更高複雜度︒本論文開發基於路徑導航與深度相機資訊的自動化流程如下：首先，機器人透過導航模組移動至目標物附近；當深度相機偵測到物體進入視野，系統便將控制權由導航模組移交給視覺模組︒此階段，透過物件辨識技術判斷物體是否位於畫面中心，並依據物體與畫面中心的像素距離，持續微調機器人位置，以穩定接近目標︒當機器人進入指定距離內，開始調整機械手臂，使手臂上的深度相機對準物體中心，並執行向前伸出的夾取動作︒夾取成功後，會將自身的座標資訊以 ROS topic 形式傳送給其他機器人︒第二台機器人收到座標後，依相同流程搜尋並微調定位目標，夾取第一台機器人手中的物體，達成多機器人間的協作傳遞任務。
本系統先於自行開發的基於 Unity 的數位孿生環境中測試與驗證，此一數位孿生環境完全複製實體環境的 ROS 為基礎的控制系統，除了可使虛實環境同步運作外，也可獨立進行模擬︒實驗結果顯示，本系統可先於 Unity 的數位孿生環境中完成任務模擬與測試，確認系統穩定性與機構安全後可以直接部署至實體機器人，有效提升開發效率與操作可靠性。

This thesis implements a multi-robot collaboration and digital twin system based on the ROS 2 framework, aiming to enable coordinated operation and simulation validation among multiple autonomous mobile robots. The system integrates four main modules: centralized control, mobile robot navigation control, robotic arm manipulation, and visual recognition, with real-time data transmission and task coordination handled via the ROS 2 platform.
During development, this study utilizes only a single depth camera mounted on the robotic arm for object localization, thus adopting a first-person view (FPV) approach for robot operation. Compared to traditional third-person view systems, FPV-based operation introduces higher complexity in both navigation and object grasping tasks.
An automated workflow is developed based on path planning and depth camera data: first, the robot navigates near the target using the navigation module. Once the object is detected within the depth camera’s field of view, control is transferred from the navigation module to the vision module. At this stage, object recognition is used to determine whether the object is centered in the camera image, and the robot continuously fine-tunes its position based on the pixel offset between the object and the image center to steadily approach the target.
When the robot reaches a predefined distance, it begins adjusting the robotic arm so that the depth camera mounted on the arm aligns with the object’s center, and then performs a forward-reaching grasp. Upon a successful grasp, the robot publishes its coordinate information via a ROS topic to other robots.
The second robot receives the coordinates and follows the same procedure to search for and precisely locate the target, eventually grasping the object from the first robot—thereby accomplishing collaborative object transfer between multiple robots.
The entire system is first tested and validated in a custom-developed Unity-based digital twin environment. This digital twin replicates the real robot system based on ROS, enabling synchronized operation between the virtual and physical environments or allowing standalone simulation. Experimental results demonstrate that the system can complete task simulations and tests in the Unity digital twin before deployment to physical robots, improving development efficiency and operational reliability.

[1] Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng. Ros: an open-source robot operating system. In ICRA Workshop on Open Source Software, 2009.
[2] Open Source Robotics Foundation. Understanding ros 2 topics — ros 2 documentation. https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html, 2022.
[3] Open Source Robotics Foundation. Understanding ros 2 services — ros 2 documentation. https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Services/Understanding-ROS2-Services.html, 2024.
[4] Open Source Robotics Foundation. Understanding ros 2 actions — ros 2 documentation. https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Actions/Understanding-ROS2-Actions.html, 2023.
[5] Steve Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operating system 2: Design, architecture, and uses in the wild. arXiv preprint arXiv:2211.07752, 2022.
[6] Tully Foote. urdf - ros wiki. https://wiki.ros.org/urdf, 2010.
[7] Open Source Robotics Foundation. Urdf - joint element. http://wiki.ros.org/urdf/XML/joint, 2023.
[8] Toshinori Kitamura. fusion2urdf: Fusion 360 add-in for exporting urdf files. https://github.com/syuntoku14/fusion2urdf, 2018. GitHub repository.
[9] Steve Macenski, Ruffin Martin, Lincoln Slater, and Xander Liu. The marathon 2: A navigation system. arXiv preprint arXiv:2203.11819, 2022.
[10] Steve Macenski, Ruffin Martin, Lincoln Slater, and Xander Liu. From the desks of ros maintainers: A survey of modern & capable mobile robotics algorithms in the robot operating system 2. arXiv preprint arXiv:2305.09933, 2023.
[11] Steve Macenski, Ruffin Martin, Lincoln Slater, and Xander Liu. Open-source, cost-aware kinematically feasible planning for mobile and surface robotics. arXiv preprint arXiv:2305.09934, 2023.
[12] Steve Macenski, Ruffin Martin, Lincoln Slater, and Xander Liu. Regulated pure pursuit for robot path tracking. arXiv preprint arXiv:2305.09935, 2023.
[13] Steve Macenski and Ivan Jambrecic. Slam toolbox: Slam for the dynamic world. Journal of Open Source Software, 6(61):2783, 2021.
[14] Kaiwalya Belsare, Antonio Cuadros Rodriguez, Pablo Garrido Sánchez, Jan Staschulat, Ralph Lange, Jaime Martín, and Ingo Lütkebohle. micro-ros. In Anis Koubaa, editor, Robot Operating System (ROS): The Complete Reference (Volume 7), pages 3–55. Springer International Publishing, 2023.
[15] micro-ROS Contributors. micro-ROS Features Overview. https://micro.ros.org/docs/overview/features/, n.d. Accessed: 2025-05-16.
[16] Foxglove Technologies Inc. Foxglove studio. https://foxglove.dev/, 2024.
[17] AWS Open Source Blog. The open source robot operating system (ros) and aws robomaker. https://aws.amazon.com/tw/blogs/opensource/open-source-robot-operating-system-ros-aws-robomaker/, 2018.
[18] Open Source Robotics Foundation. 2021 eclipse cyclone dds ros middleware evaluation report with iceoryx and zenoh. https://osrf.github.io/TSC-RMW-Reports/humble/eclipse-cyclonedds-report.html, 2021.
[19] Toyota Blog. Toyota human support robot: What is it and how can it be used? Toyota UK Magazine, https://mag.toyota.co.uk/toyota-human-support-robot/, March 2021.
[20] NVIDIA Developer. Nvidia isaac ros (robot operating system). NVIDIA Developer, https://developer.nvidia.com/isaac/ros. Accessed: 2025-05-10.
[21] NVIDIA Developer Blog. Supercharge robotics workflows with ai and simulation using nvidia isaac sim 4.0 and nvidia isaac lab. https://developer.nvidia.com/blog/supercharge-robotics-workflows-with-ai-and-simulation-using-nvidia-isaac-sim-4-0-and-nvidia-isaac-lab/, 2024.
[22] NVIDIA. Jetson orin nano developer kit user guide: Hardware specification. https://developer.nvidia.com/embedded/learn/jetson-orin-nano-devkit-user-guide/hardware_spec.html, 2024.
[23] Siemens AG. Ros# - installation and configuration. https://github.com/siemens/ros-sharp/wiki/User_Inst_InstallationAndConfiguration, 2020.
[24] Siemens PLM Components Team. Robotic automation in packaging and distribution plants: Kineoworks. https://blogs.sw.siemens.com/plm-components/robotic-automation-in-packaging-and-distribution-plants-kineoworks/, 2024.
[25] Michael Grieves. Digital twin: Manufacturing excellence through virtual factory replication. In White Paper. Florida Institute of Technology, 2015.
[26] Michael Grieves and John Vickers. Digital twin: Mitigating unpredictable, undesirable emergent behavior in complex systems. In F.-J. Kahlen et al., editor, Transdisciplinary Perspectives on Complex Systems, pages 85–112. Springer International Publishing, 2017.
[27] E. H. Glaessgen and D. S. Stargel. The digital twin paradigm for future nasa and u.s. air force vehicles. In 53rd AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Honolulu, Hawaii, 2012. American Institute of Aeronautics and Astronautics.
[28] Eric J. Tuegel, Anthony R. Ingraffea, Thomas G. Eason, and S. Michael Spottswood. Reengineering aircraft structural life prediction using a digital twin. International Journal of Aerospace Engineering, 2011:1–14, 2011.
[29] Fei Tao, He Zhang, Ang Liu, and A. Y. C. Nee. Digital twin in industry: State-of-the-art. IEEE Transactions on Industrial Informatics, 15(4):2405–2415, April 2019.
[30] Gary White, Anna Zink, Lara Codecà, and Siobhán Clarke. A digital twin smart city for citizen feedback. Cities, 110:103064, 2021.
[31] Lei Zhou, Yi Ding, Yanning Zhang, and Keqiang Li. Digital twin simulation of connected and automated vehicles with vehicle-to-everything (v2x) communication environment. IEEE Transactions on Industrial Informatics, 18(2):1379–1389, 2022.
[32] Unity Technologies. Articulations. https://docs.unity3d.com/6000.0/Documentation/Manual/articulations-section.html, 2024. Accessed: 2025-05-09.
[33] Nathan Koenig and Andrew Howard. Design and use paradigms for gazebo, an open-source multi-robot simulator. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2149–2154, Sendai, Japan, September 2004. IEEE.
[34] Olivier Michel. Webotstm: Professional mobile robot simulation. International Journal of Advanced Robotic Systems, 1(1):39–42, 2004.
[35] Cyberbotics Ltd. Aibo ers7 — webots user guide (r2019a). https://www.cyberbotics.com/doc/guide/aibo-ers7?version=cyberbotics:R2019a, 2019.
[36] Cyberbotics Ltd. Mindstorms — webots user guide (r2021a). https://www.cyberbotics.com/doc/guide/mindstorms?version=R2021a, 2021.
[37] Cyberbotics Ltd. Khepera iv — webots user guide (r2019a). https://www.cyberbotics.com/doc/guide/khepera4?version=cyberbotics:R2019a, 2019.
[38] Cyberbotics Ltd. Koala — webots user guide (r2020a). https://cyberbotics.com/doc/guide/koala?version=R2020a, 2020.
[39] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108.10470, 2021.
[40] NVIDIA Corporation. Nvidia isaac sim: Robotics simulation and synthetic data generation. https://developer.nvidia.com/isaac/sim. Accessed: 2025-05-09.
[41] Cheng-Chieh Liu. A ros2-based digital twin system for robot control and interaction in unity. Master’s thesis, National Cheng Kung University, Taiwan, 2024.
[42] Shital Shah, Debadeepta Dey, Chris Lovett, and Ashish Kapoor. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. arXiv preprint arXiv:1705.05065, 2017.
[43] Microsoft Research. Microsoft is accelerating the future of aerial autonomy. https://www.microsoft.com/en-us/research/video/microsoft-is-accelerating-the-future-of-aerial-autonomy/?locale=zh-cn, 2024.
[44] Hao Zhou, Yongping Pan, Xiaokang Zhou, Wei Zhang, and Yimin Song. Navigation simulation of a mecanum wheel mobile robot based on an improved a* algorithm in unity3d. Sensors, 19(13):2976, 2019.
[45] Booz Allen Hamilton and Unity. Building dod’s largest-ever digital twin of its kind. https://www.boozallen.com/insights/digital-twin/building-dods-largest-ever-digital-twin-of-its-kind.html.
[46] Jihye Bae, Christian Gütl, Dietrich Albert, Klaus Schöffmann, and Peter Simon. Mobile delivery robots: Mixed reality-based simulation relying on ros and unity 3d. arXiv preprint arXiv:2006.09002, 2020.
[47] Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. CoRR, abs/1506.02640, 2015.
[48] Erwin Coumans. Pybullet physics engine - forum. https://pybullet.org/wordpress/index.php/forum-2/, 2025.
[49] Xinchen Yan, Jasmine Hsu, Mohi Khansari, Yunfei Bai, Arkanath Pathak, Abhinav Gupta, James Davidson, and Honglak Lee. Learning 6-dof grasping interaction via deep geometry-aware 3d representations, 2018.
[50] Xintong Yang, Ze Ji, Jing Wu, and Yu-Kun Lai. An Open-Source Multi-goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet, page 14–24. Springer International Publishing, 2021.
[51] Zhengyou Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000.
[52] Baichuan Huang, Jingjin Yu, and Siddarth Jain. Earl: Eye-on-hand reinforcement learner for dynamic grasping with active pose estimation, 2023.
[53] Microsoft. Windows subsystem for linux documentation. https://learn.microsoft.com/en-us/windows/wsl/, 2024.
[54] Microsoft. Compare wsl 1 and wsl 2. https://learn.microsoft.com/en-us/windows/wsl/compare-versions, 2024.