隨著智慧製造技術的蓬勃發展，自動化工廠中對機器人系統的依賴顯著提升，而機器人插件技術為其中重要的一環。有鑑於此，本論文提出一完整插件架構，其中融合視覺遮擋階段的運動規劃、深度神經網路之物件位姿估計、基於位置的視覺伺服控制，以及插入階段之順應控制等技術，建立從視覺感知定位到機械手臂控制的完整流程。最終於實際場景中，成功實現該架構並完成插件任務。其中，視覺遮擋階段的運動規劃方法，採用具備空間與時間縮放特性的改良式動態運動原語結合最小急跳度姿態規劃，提升機械手臂初始姿態的靈活性與規劃自由度。為提高視覺定位的精準度與泛用性，本論文使用基於深度神經網路的演算法來獲取物件位姿。所使用之模型結合PSPNet與PointNet++進行特徵萃取，並透過Shared MLPs所構成之多任務學習架構進行關鍵點預測，在獲得關鍵點後即可進行位姿的計算。而在機械手臂的控制中，使用本論文所提出之基於速率規劃調整之PBVS以進行視覺特徵的對齊，並透過速度融合策略實現前期運動規劃至視覺伺服之平滑切換。最終，導入順應控制於此架構中，以提升整體架構對位置不確定性的容錯能力與力量安全性。

With the rapid advancement of smart manufacturing technologies, reliance on robotic systems in automated factories has significantly increased. One such operation, robotic peg-in-hole insertion, is crucial for achieving high-precision and automated assembly. In this paper, a comprehensive insertion framework is proposed that, integrates motion planning under visual occlusion, deep neural network-based object pose estimation, position-based visual servoing (PBVS), and compliance control during the insertion phase. The proposed framework establishes a complete pipeline from visual perception and localization to robot control, and is successfully validated in a real-world scenario to accomplish the insertion task. During the visual occlusion stage, the motion planning approach utilizes a modified version of Dynamic Movement Primitives (MDMP) with spatial and temporal scaling, combined with minimum-jerk orientation planning, to enhance the flexibility of the robot's initial posture and increase planning freedom. For object pose estimation, a deep learning-based method is employed. The network architecture incorporates PSPNet and PointNet++ for feature extraction, and a multi-task learning structure based on Shared MLPs is used to predict keypoints; these are subsequently used to compute the 6-DoF object pose. In the control stage, a rate-scheduled PBVS method is proposed to align visual features, and a velocity blending strategy is adopted to ensure smooth transitions from motion planning to visual servoing. Finally, compliance control is integrated into the system to improve robustness against positional uncertainties and to ensure force safety during the insertion process.

[1] 2024/2025產業技術白皮書，經濟部技術處，2024
[2] International Federation of Robotics, “Global Robotics market: How can Europe step up its game,” Jun. 2025. [Online] Available: https://ifr.org/downloads/press_docs/Market_presentation_on_global_industrial_robot_installations_-_preliminary_figures_2024.pdf
[3] A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y. Zhu, and R. Martín-Martín, “What matters in learning from offline human demonstrations for robot manipulation,” in Proceedings of the 5th Conference on Robot Learning, London, U.K., 2022, pp. 1678–1690.
[4] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. Cambridge, Ma, USA: MIT Press, 2018.
[5] M. Kyrarini, M. A. Haseeb, D. Ristić-Durrant, and A. Gräser, “Robot learning of industrial assembly task via human demonstrations,” Autonomous Robots, vol. 43, no. 1, pp. 239–257, Apr. 2018.
[6] J. Sheng, Y. Tang, S. Xu, F. Tan, R. Hou, and T. Xu, “A Stable Learning-Based Method for Robotic Assembly with Motion and Force Measurements,” IEEE Transactions on Industrial Electronics, vol. 71, no. 9, pp. 11093–11103, Dec. 2023.
[7] C. Chang, K. Haninger, Y. Shi, C. Yuan, Z. Chen and J. Zhang, “Impedance Adaptation by Reinforcement Learning with Contact Dynamic Movement Primitives,” in Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Sapporo, Japan, 2022, pp. 1185–1191.
[8] T. Tang, H. -C. Lin, Y. Zhao, Y. Fan, W. Chen and M. Tomizuka, “Teach industrial robots peg-hole-insertion by human demonstration,” in Proceedings of IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Banff, AB, Canada, 2016, pp. 488–494.
[9] D. A. Reynolds, “Gaussian mixture models,” Encyclopedia Biometrics, vol. 741, pp. 659–663, Jul. 2009.
[10] H. G. Sung, “Gaussian Mixture Regression and Classification,” 2004. [Online] Available: http://www.stat.rice.edu/~hgsung/thesis.pdf
[11] G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, “Extreme learning machine: Theory and applications,” Neurocomputing, vol. 70, no. 1–3, pp. 489–501, May. 2006.
[12] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal, “Dynamical Movement Primitives: learning attractor models for motor behaviors,” Neural Computation, vol. 25, no. 2, pp. 328–373, Nov. 2012.
[13] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018, pp. 1861–1870.
[14] J. Xu, Z. Hou, W. Wang, B. Xu, K. Zhang, and K. Chen, “Feedback Deep Deterministic Policy Gradient With Fuzzy Reward for Robotic Multiple Peg-in-Hole Assembly Tasks,” IEEE Transactions on Industrial Informatics, vol. 15, no. 3, pp. 1658–1667, Mar. 2019.
[15] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” 2015, arXiv:1509.02971.
[16] C. C. Beltran-Hernandez, D. Petit, I. G. Ramirez-Alpizar, and K. Harada, “Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep-Reinforcement-Learning Approach,” Applied Sciences, vol. 10, no. 19, p. 6923, Oct. 2020.
[17] C. C. Beltran-Hernandez, D. Petit, I. G. Ramirez-Alpizar, T. Nishi, S. Kikuchi, T. Matsubara, and K. Harada, “Learning Force Control for Contact-Rich Manipulation Tasks With Rigid Position-Controlled Robots,” IEEE robotics and automation letters, vol. 5, no. 4, pp. 5709–5716, Mar. 2020.
[18] T. Inoue, G. De Magistris, A. Munawar, T. Yokoya and R. Tachibana, “Deep reinforcement learning for high precision assembly tasks,” in Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 2017, pp. 819–825.
[19] S. Hochreiter and J. Schmidhuber, “Long Short-Term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, Nov. 1997.
[20] C. Ku, C. Winge, R. Diaz, W. Yuan and K. Desingh, “Evaluating Robustness of Visual Representations for Object Assembly Task Requiring Spatio-Geometrical Reasoning,” in Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Yokohama, Japan, 2024, pp. 831–837.
[21] Y. Shen, Q. Jia, R. Wang, Z. Huang, and G. Chen, “Learning-Based Visual Servoing for High-Precision Peg-in-Hole Assembly,” Actuators, vol. 12, no. 4, pp. 144–144, Mar. 2023.
[22] S. Huang, Y. Yamakawa, T. Senoo and M. Ishikawa, “Realizing peg-and-hole alignment with one eye-in-hand high-speed camera,” in Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Wollongong, NSW, Australia, 2013, pp. 1127–1132.
[23] R. J. Chang, C. Y. Lin, and P. S. Lin, “Visual-based automation of peg-in-hole microassembly process,” Journal of Manufacturing Science and Engineering, vol. 133, no. 4, Aug. 2011, Art. no. 041015.
[24] W.-C. Chang, “Robotic assembly of smartphone back shells with eye-in-hand visual servoing,” Robotics and Computer-Integrated Manufacturing, vol. 50, pp. 102–113, Apr. 2018.
[25] D. E. Whitney, “Quasi-Static assembly of compliantly supported rigid parts,” Journal of Dynamic Systems Measurement and Control, vol. 104, no. 1, pp. 65–77, Mar. 1982.
[26] K. Hara, R. Yokogawa and Y. Kai, “Evaluation of task-performance of a manipulator for a peg-in-hole task,” in Proceedings of International Conference on Robotics and Automation, Albuquerque, NM, USA, 1997, pp. 600–605.
[27] I. F. Jasim, P. W. Plapper, and H. Voos, “Contact-state modelling in force-controlled robotic peg-in-hole assembly processes of flexible objects using optimised Gaussian mixtures,” in Proceedings of the Institution of Mechanical Engineers. Part B, Journal of engineering manufacture, vol. 231, no. 8, pp. 1448–1463, 2017.
[28] Hikaru Unten, Sho Sakaino, and T. Tsuji, “Peg-in-Hole Using Transient Information of Force Response,” IEEE/ASME Transactions on Mechatronics, vol. 28, no. 3, pp. 1674–1682, Dec. 2022.
[29] O. Spector and D. D. Castro, “InsertionNet - A Scalable Solution for Insertion,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5509–5516, Jul. 2021.
[30] H. Park, J. Park, D.-H. Lee, J.-H. Park, M.-H. Baeg, and J.-H. Bae, “Compliance-Based Robotic Peg-in-Hole Assembly Strategy Without Force Feedback,” IEEE Transactions on Industrial Electronics, vol. 64, no. 8, pp. 6299–6309, Aug. 2017.
[31] H. Qiao, M. Wang, J. Su, S. Jia, and R. Li, “The Concept of ‘Attractive Region in Environment’ and its Application in High-Precision Tasks With Low-Precision Systems,” IEEE/ASME Transactions on Mechatronics, vol. 20, no. 5, pp. 2311–2327, Oct. 2015.
[32] A. Salem and Yiannis Karayiannidis, “Robotic Assembly of Rounded Parts With and Without Threads,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2467–2474, Feb. 2020.
[33] M. Ginesi, N. Sansonetto, and P. Fiorini, “Overcoming some drawbacks of Dynamic Movement Primitives,” Robotics and Autonomous Systems, vol. 144, Oct. 2021, Art. no. 103844.
[34] T. Flash and N. Hogan, “The coordination of arm movements: an experimentally confirmed mathematical model,” Journal of Neuroscience, vol. 5, no. 7, pp. 1688–1703, Jul. 1985.
[35] A. C. Reddy, “Difference between Denavit-Hartenberg (DH) classical and modified conventions for forward kinematics of robots with case study,” in Proceedings of the International Conference on Advanced Materials and Manufacturing Technologies, Hyderabad, India, 2014, pp. 267–286.
[36] P. M. Kebria, S. Al-wais, H. Abdi and S. Nahavandi, "Kinematic and dynamic modelling of UR5 manipulator," in Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 2016, pp. 4229–4234.
[37] R. Keating and N. J. Cowan, “UR5 Inverse Kinematics,” unpublished, Johns Hopkins University, 2016. [Online] Available: https://tianyusong.com/wp-content/uploads/2017/12/ur5_inverse_kinematics.pdf.
[38] L. Sciavicco and B. Siciliano, Modelling and Control of Robot Manipulators. Vienna, Austria: Springer, 2012.
[39] F. Chaumette and S. Hutchinson, “Visual servo control part I basic approaches,” IEEE Robotics & Automation Magazine, vol. 13, no. 4, pp. 82–90, Dec. 2006.
[40] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330–1334, Nov. 2000.
[41] Y. He, W. Sun, H. Huang, J. Liu, H. Fan and J. Sun, “PVN3D: A Deep Point-Wise 3D Keypoints Voting Network for 6DoF Pose Estimation,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 11629–11638.
[42] E. Malis, “Contributions à la modélisation et à la commande en asservissement visuel,” Ph.D. dissertation, IRISA, Rennes I Univ., Paris, France, 1998.
[43] H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid Scene Parsing Network,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6230–6239.
[44] C. R. Qi, L. Yi, H. Su, and L. J. Guibas. ”Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” in Proceedings of Advances in neural information processing systems, 2017, pp. 5099–5108.
[45] C. Wang, D. Xu, Y. Zhu, R. Martin-Martin, C. Lu, L. Fei-Fei, and S. Savarese, “DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion,” in Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019, pp. 3338–3347.
[46] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-Squares Fitting of Two 3-D Point Sets,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 698–700, Sep. 1987.
[47] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770–778.
[48] R. Q. Charles, H. Su, M. Kaichun and L. J. Guibas, “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 77–85.
[49] D. Comaniciu and P. Meer, “Mean shift: a robust approach toward feature space analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603–619, May. 2002.
[50] N. Hogan, “Impedance control: An approach to manipulation: Part II—Implementation,” Journal of Dynamic Systems, Measurement and Control, vol. 107, no. 1, pp. 8–16, Mar. 1985.
[51] N. Hogan, “Impedance Control: An Approach to Manipulation,” in Proceedings of American Control Conference, San Diego, CA, USA, 1984, pp. 304–313.
[52] D. Reyes-Uquillas and T. Hsiao, “Safe and intuitive manual guidance of a robot manipulator using adaptive admittance control towards robot agility,” Robotics and Computer-Integrated Manufacturing, vol. 70, Aug. 2021, Art. no. 102127.
[53] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, “PoseCNN: A convolutional neural network for 6d object pose estimation in cluttered scenes,” 2017, arXiv:1711.00199.
[54] E. Olson, “AprilTag: A robust and flexible visual fiducial system,” in Proceedings of IEEE International Conference on Robotics and Automation, Shanghai, China, 2011, pp. 3400–3407.
[55] M. Oikawa, T. Kusakabe, K. Kutsuzawa, S. Sakaino, and T. Tsuji, “Reinforcement Learning for Robotic Assembly Using Non-Diagonal Stiffness Matrix,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2737–2744, Apr. 2021.