機器人透過視覺拾取物件的技術廣泛地應用於日常生活之中。除了傳統簡單、重複性高的任務之外，如何更彈性地面對多元的場景，例如讓機器人透過演算法快速地判斷合適的抓取姿態、拾取未知甚至是堆疊的物件等，成為物件拾取重要的發展方向，也因此本論文聚焦在使用單一視角點雲的抓取姿態生成演算法研究。在論文中我們實現S4G演算法，並透過電腦模擬以及上機實驗，測試演算法所生成的抓取姿態在實際抓取任務中的效果。除此之外，論文中針對點雲的完整性對於S4G演算法的抓取效果影響，設計了點雲疊合演算法，並在模擬環境中進行抓取成功率比較。為了執行實際的物件抓取實驗，本論文也介紹了抓取流程中運用到的順逆向運動學，並且加入考量真實世界限制的抓取篩選機制。實驗顯示所用的演算法在球體、長方體、圓柱體等形狀物件的單物件及多物件場景有良好的抓取姿態生成效果。然而當場景中的物件擺放更為密集、堆疊程度提高及加入未知物件，所訓練的網路對於抓取生成的穩定性會降低。

Robotic grasping technologies based on vision are widely used these days. Beyond performing traditional repetitive tasks, adapting to various scenarios such as, quickly deciding optimal grasp poses and the grasping of unknown objects has become an important issue in the field of robotic grasping. Therefore, this thesis focuses on studying the algorithm for grasp pose estimation based on a single-view point cloud. The S4G algorithm for generating grasp poses is implemented in this thesis. The effectiveness of the S4G algorithm is verified through simulations and real-world experiments. Both single object and multi-object grasping experiments were performed. Furthermore, in order to investgate the impact of point cloud completeness on algorithm performance, the network model was trained with complete point clouds which were merged by single-view point clouds and compared it to the one trained by single-view point clouds. To perform actual object grasping experiments, this thesis also introduces both forward and inverse kinematics which are essential in grasping processes. After the grasp poses are estimated, three criteria considering real-world limitations are added for filtering grasp poses. Through the experiment, the algorithm demonstrated good performance in estimating the grasp poses of both single object and multi-object scenarios composed of spheres, cuboids and cylinders. However, when scanarios became dense and cluttered with unknown objects, the performance of grasp pose estimation became worse.

[1] M. Christopher. “World Robotics - Industrial Robots.” International Federation of Robotics. [Online] Available: International Federation of Robotics (ifr.org). (accessed Jun. 11, 2024).
[2] G. Du, K. Wang, S. Lian, and K. Zhao, “Vision-based Robotic Grasping from Object Localization, Object Pose Estimation to Grasp Estimation for Parallel Grippers: a Review,” Artificial Intelligence Review, vol. 54, no. 3, pp. 1677-1734, Mar. 2021.
[3] H. Cao, G. Chen, Z. Li, J. Lin, and A. Knoll, “Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection,” 2021, arXiv:2101.10226.
[4] 陳亞伶，基於電腦視覺與深度增強式學習之工業用機械手臂物件取放任務研究，國立成功大學，電機工程學系，2021。
[5] Y. Li, R. Bu, M. Sun, W. Wu, X. Di, and B. Chen, “PointCNN: Convolution on χ-transformed Points,” in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 826–836.
[6] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep Learning on Point Sets for 3d Classification and Segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2017, pp. 652-660.
[7] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space,” in Proc. Adv. Neural Inf. Process. Syst., 2017.
[8] P. J. Besl and N. D. McKay, “A Method for Registration of 3-D Shapes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239–256, Feb. 1992.
[9] P. Biber and W. Strasser, “The Normal Distributions Transform: A New Approach to Laser Scan Matching,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., vol. 3, Nov. 2003, pp. 2743–2748.
[10] D. Yang, T. Tosun, B. Eisner, V. Isler, and D. Lee, “Robotic Grasping through Combined Image-based Grasp Proposal and 3D Reconstruction,” in Proc. IEEE Int. Conf. Robot. Automat., 2021, pp. 6350–6356.
[11] X. Yan, M. Khansari, J. Hsu, Y. Gong, Y. Bai, S. Pirk and H. Lee, “Data-efficient Learning for Sim-to-real Robotic Grasping Using Deep Point Cloud Prediction Networks,” 2019, arXiv:1906.08989.
[12] P. R. Florence, L. Manuelli, and R. Tedrake, “Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation,” in Conference on Robot Learning. PMLR. 2018, pp. 373-385.
[13] Y. Lin, C. Tang, F.-J. Chu, and P. A. Vela, “Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping,” Sep. 2019, vol. 1, pp. 1–8, arXiv:1909.08508.
[14] A. ten Pas and R. Platt. “Using Geometry to Detect Grasp Poses in 3d Point Clouds,” Robotics Research, pages 307–324. Springer, 2018.
[15] A. ten Pas, M. Gualtieri, K. Saenko, and R. Platt, “Grasp Pose Detection in Point Clouds,” The International Journal of Robotics Research, vol. 36, no. 13-14, pp. 1455-1473, 2017.
[16] H. Liang, X. Ma, S. Li, M. Görner, S. Tang, B. Fang, F. Sun and J. Zhang, “PointnetGPD: Detecting Grasp Configurations from Point Sets,” in Proc. IEEE Int. Conf. Robot. Autom., 2019, pp. 3629–3635.
[17] P. Ni, W. Zhang, X. Zhu, and Q. Cao, “PointNet++ Grasping: Learning an End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds,” in Proc. IEEE Int. Conf. Robot. Autom., 2020, pp. 3619–3625
[18] B. Zhao, H. Zhang, X. Lan, H. Wang, Z. Tian, and N. Zheng, “RegNet: Region-based Grasp Network for End-to-end Grasp Detection in Point Clouds,” in Proc. IEEE Int. Conf. Robot. Autom., 2021, pp. 13474–13480.
[19] Y. Qin, R. Chen, H. Zhu, M. Song, J. Xu, and H. Su, “S4g: Amodal Single-view Single-shot SE (3) Grasp Detection in Cluttered Scenes,” in Proc. Conf. Robot Learning. PMLR, 2020, pp. 53–65.
[20] D. Prattichizzo and J. C. Trinkle, Grasping, 2008, pp. 671-700.
[21] R. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction to Robotic Manipulation. Boca Raton, FL, USA: CRC Press, 1994
[22] B. Calli, A. Singh, A. Walsman, S. Srinivasa, P. Abbeel, and A. M. Dollar, “The YCB Object and Model Set: Towards Common Benchmarks for Manipulation Research,” in Proc. Int. Conf. Adv. Robot., 2015, pp. 510-517.
[23] H.-S. Fang, C. Wang, M. Gou, and C. Lu, “GraspNet: A Large-Scale Clustered and Densely Annotated Dataset for Object Grasping,” 2019, arXiv:1912.13470.
[24] T. Patten, K. Park, and M. Vincze, “DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-based Robotic Grasping,” Front. Robot. AI, vol. 7, 2020, Art. no. 120.
[25] E. Todorov, T. Erez, and Y. Tassa, “MuJoCo: A Physics Engine for Model-based Control,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2012, pp. 5026-5033.
[26] E. Fix and J. L. Hodges, “Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties,” International Statistical Review / Revue Internationale de Statistique, vol. 57, no. 3, p. 238, Dec. 1989.
[27] J. L. Bentley, “Multidimensional Binary Search Trees Used for Associative Searching,” Communications of the ACM, vol. 18, no. 9, pp. 509–517, Sep. 1975.
[28] Y. Zhou, C. Barnes, J. Lu, J. Yang, and H. Li, “On the Continuity of Rotation Representations in Neural Networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2019.
[29] J. J. Craig, Introduction to Robotics: Mechanics and Control, Pearson Prentice Hall, 2005.
[30] “Franka Control Interface Documentation — Franka Control Interface (FCI) documentation,” [Online] Available: https://frankaemika.github.io/docs/index.html. (accessed Sep. 11, 2023).
[31] S. Akram and Q. U. Ann, “Newton Raphson Method,” International Journal of Scientific & Engineering Research, vol. 6, no. 7, pp. 1748-1752, 2015.
[32] R. W. Wedderburn, “Quasi-likelihood Functions, Generalized Linear Models, and the Gauss—Newton Method,” Biometrika, vol. 61, no. 3, pp. 439-447, 1974.
[33] N. Yamashita and M. Fukushima, “On the Rate of Convergence of the Levenberg-Marquardt Method,” in Topics in Numerical Analysis: With Special Emphasis on Nonlinear Problems, 2001: Springer, pp. 239-249.
[34] P. Corke and J. Haviland, “Not Your Grandmother’s Toolbox – the Robotics Toolbox Reinvented for Python,” in IEEE Int. Conf. Robot. Autom., 30 May-5 June 2021, pp. 11357-11363.
[35] “UR5 Technical Specifications,” Universal Robots A/S, Denmark. [Online] Available: https://www.universal-robots.com/media/50588/ur5_en.pdf (accessed Apr. 17, 2024).
[36] D. F. Huber and M. Hebert, “Fully Automatic Registration of Multiple 3D Data Sets,” Image and Vision Computing, vol. 21, no. 7, pp. 637–650, Jul. 2003.
[37] S. Fantoni, U. Castellani, and A. Fusiello, “Accurate and Automatic Alignment of Range Surfaces,” in Proc. Int. Conf. 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), Oct. 2012, pp.73-80.
[38] Z. Gojcic, C. Zhou, J. D. Wegner, L. J. Guibas, and T. Birdal, “Learning Multiview 3D Point Cloud Registration,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 1759–1769.
[39] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-Squares Fitting of Two 3-D Point Sets,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-9, no. 5, pp. 698–700, May 1987.