本文之目標在於將兩台RGB-D裝置透過NiTE SDK所得到的同一個人體骨架位置及方向資訊進行對齊，讓第二台裝置的坐標系能對齊到第一台裝置的坐標系，如此便能在同一坐標系下進行兩台裝置資料的分析與運用。本文除了達到全自動進行兩個裝置的坐標系對齊外，在兩台相機間隔距離較長的寬基線（Wide Baseline）的情況下也能良好適應。
本文以兩台RGB-D裝置與一個全身人體的應用環境下，提出兩階段方法求得兩台RGB-D裝置對人的最佳轉換矩陣，第一階段：透過兩台裝置對同一個人體骨架的位置與方向資訊，求得裝置2對裝置1的初始轉換矩陣，第二階段：透過迭代鄰近點（Iterative Closest Point，ICP）最佳化最終相機轉換矩陣。本文在變因僅有單一裝置NiTE 估測的骨架變異下，人體點雲平均誤差為3mm，在變因為兩個裝置各自的NiTE估測骨架變異及寬基線下，人體點雲最小平均誤差為39.3 mm，且誤差與裝置間的距離成正比。與其他文獻相比，現有兩台相機以上的對齊方法都非全自動對齊，都需要人為介入操作對齊過程，目前提出可應用於寬基線最新最好的方法，則需要一個外部球體進行參考校正。本文提出之方法不需要人介入，但是必須在兩台相機視野中的可看到一個人正面全身的環境下。

This work is aim to align two sets of position and orientation data of a skeleton estimated from two RGB-D devices via NiTE SDK. The coordinate of second device was aligned with the coordinate of the first one, so that we could utilize those aligned data for further use. Our approach not only achieve fully automatic alignment but also can adapted to wide-baseline situation. However, the result of our alignment approach was not as good as state-of-the-art surveys.
This work proposed two stages of approach for finding the finest transformation matrix with two RGB-D devices on a person to make two coordinates aligned. In the first stage, we derived the initial transformation matrix by using the position and orientation of two skeletal data from same person with two RGB-D devices. In the second stage, we optimize the final transformation matrix with Iterative Closest Point(ICP) algorithm. Our experiment results are as follows: 3mm for average error of human point cloud subset error metric with only skeletal estimation error from NiTE. A minimal error of human point cloud subset error metric for 39.3 mm and this error is proportional to the distance between two RGB-D devices. Compare with other works, all the alignment method with two or more cameras are not fully automatic and all need manual assistance. The best and latest method that can be applied to wide baseline alignment, however, using a calibration ball as reference point. Our approach is free from human’s intervention and this approach is fully automatic under the restriction that the person’s full body is presented in the FOV(Field of View) of both RGB-D devices.

[1] P. C. Su, J. Shen, and M. U. Rafique, "RGB-D Camera Network Calibration and Streaming for 3D Telepresence in Large Environment," 2017 IEEE Third International Conference on Multimedia Big Data (BigMM), Laguna Hills, CA, USA, pp. 362-369, 19-21 April 2017.
[2] W. Xu, P. c. Su, and S. c. S. Cheung, "Human Pose Estimation using Two RGB-D Sensors," 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 1279-1283, 25-28 September 2016.
[3] S. Baek and M. Kim, "Real-Time Performance Capture using Multiple Kinects," 2014 International Conference on Information and Communication Technology Convergence (ICTC), Busan, South Korea, pp. 647-648, 22-24 October 2014.
[4] W. Lemkens, P. Kaur, K. Buys, P. Slaets, T. Tuytelaars, and J. D. Schutter, "Multi RGB-D Camera Setup for Generating Large 3D Point Clouds," 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, pp. 1092-1099, 3-7 November 2013.
[5] S. Rogge and C. Hentschel, "A Multi-Depth Camera Capture System for Point Cloud Library," 2014 IEEE Fourth International Conference on Consumer Electronics Berlin (ICCE-Berlin), Berlin, Germany, pp. 50-54, 7-10 September 2014.
[6] S. Jeon, M. C. Roh, J. H. Lee, and J. Lee, "Accurate Fusing Multiple RGBD Sensors Using Geometry," 2015 12th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Goyang, South Kroea, pp. 18-21, 28-30 October 2015.
[7] N. A. Azis, H. J. Choi, and Y. Iraqi, "Substitutive Skeleton Fusion for Human Action Recognition," 2015 International Conference on Big Data and Smart Computing (BIGCOMP,) Jeju, South Korea, pp. 170-177, 9-11 February 2015.
[8] M. Caon, Y. Yue, J. Tscherrig, E. Mugellini, and O. A. Khaled, "Context-Aware 3D Gesture Interaction Based on Multiple Kinects," AMBIENT 2011, The First International Conference on Ambient Computing, Applications, Services and Technologies, Barcelona, Spain, pp. 7-12, 23-28 October 2011.
[9] B. Williamson, J. L. Viola, T. Roberts, and P. Garrity, "Multi-Kinect Tracking for Dismounted Soldier Training," Proceedings of the Interservice/Industry Training, Simulation and Education Conference(I/ITSEC), Orlando, Florida, pp. 3-5, 3-6 December 2012.
[10] R. A. Newcombe, S. Izadi, O. Hilliges, "KinectFusion: Real-time Dense Surface Mapping and Tracking," 2011 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, pp. 127-136, Octbor 2011.
[11] I. Culjak, D. Abram, T. Pribanic, H. Dzapo, and M. Cifrek, "A Brief Introduction to OpenCV," 2012 Proceedings of the 35th International Convention MIPRO, Opatija, Croatia, pp. 1725-1730, 21-25 May 2012.
[12] Heresy, "OpenNI2 Introduction.," [Online]. Available: https://kheresy.wordpress.com/2012/12/21/basic-openni-2/. [Accessed 15 July 2017].
[13] F. Endres, J. Hess, J. Sturm, D. Cremers, and W. Burgard, "3-D Mapping With an RGB-D Camera," IEEE Transactions on Robotics, vol. 30, no. 1, pp. 177-187, February 2014.
[14] C.-C. Chen, Markless Exercise Training Toolkit Using Multiple RGB-D based Sensors, Master Thesis, Department of Engineering Science, National Cheng Kung University, Tainan, Taiwan, 2015.
[15] T.-L. Cheng, An IoT-Based Exercise Training System, Master Thesis, Department of Engineering Science, National Cheng Kung University, Tainan, Taiwan, 2016.
[16] H. Haggag, M. Hossny, D. Filippidis, D. Creighton, S. Nahavandi, and V. Puri, "Measuring Depth Accuracy in RGBD Cameras," 7th International Conference on Signal Processing and Communication Systems (ICSPCS), Carrara, VIC, Australia, pp. 1-7, 16-18 December 2013.
[17] R. M. Martín, M. Lorbach, and O. Brock, "Deterioration of Depth Measurements due to Interference of Multiple RGB-D Sensors," 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, pp. 4205-4212, 14-18 September 2014.
[18] L. Shao, Z. Zhang, P. Kohli, and J. Han, Computer Vision and Machine Learning with RGB-D Sensors, Springer, 2014.
[19] H. Gonzalez-Jorge, B. Riveiro, E. Vazquez-Fernandez, J. Martínez-Sánchez, and P. Arias, "Metrological Evaluation of Microsoft Kinect and Asus Xtion Sensors," Measurement, vol. 46, no. 6, pp. 1800-1806, July 2013.
[20] Z. Zhengyou, "Flexible Camera Calibration by Viewing a Plane from Unknown Orientations," Seventh IEEE International Conference on Computer Vision, vol. 1, no. 1, Kerkyra, Greece, Greece, pp. 666-673, 20-27 September 1999.
[21] P. J. Besl and N. D. McKay, "A Method for Registration of 3-D Shapes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, February 1992.
[22] Y. Chen, and G. Medioni, "Object Modelling by Registration of Multiple Range Images," Image Vision Computing, vol. 10, no. 3, Butterworth-Heinemann Newton, MA, USA, pp. 145-155, April 1992.
[23] C. Yuan, X. Yu, and Z. Luo, "3D Point Cloud Matching Based on Principal Component Analysis and Iterative Closest Point Algorithm," 2016 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China, pp. 404-408, 11-12 July 2016.
[24] S. Rusinkiewicz and M. Levoy, "Efficient Variants of The ICP Algorithm," Proceedings Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, Quebec, Canada, Canada, pp. 145-152, 28 May-1 June 2001.
[25] A. Segal, D. Haehnel, and S. Thrun., "Generalized-ICP," In Proceedings of Robotics: Science and Systems, Seattle, USA, 28 June-1 July 2009.
[26] Y. Ge, B. Wang, J. Nie, and B. Sun, "A Point Cloud Registration Method Combining Enhanced Particle Swarm Optimization and Iterative Closest Point Method," 2016 Chinese Control and Decision Conference (CCDC), Yinchuan, China, pp. 2810-2815, 28-30 May 2016.
[27] R. Kidson, D. Pangercic, D. Stanimirovic, and M. Beetz, "Elaborative Evaluation of RGB-D Based Point Cloud Registration for Personal Robots," CRA 2012 Workshop on Semantic Perceptionand Mapping for Knowledge-enabled Service Robotics, St. Paul, MN, USA, 14-18 May 2012.
[28] S. Bouaziz, A. Tagliasacchi, and M. Pauly, "Sparse Iterative Closest point," Proceeding SGP '13 Proceedings of the Eleventh Eurographics/ACMSIGGRAPH Symposium on Geometry Processing, Gnova, Italy, pp. 113-123, 3-5 July 2013.
[29] D. Holz, A. E. Ichim, F. Tombari, R. B. Rusu, and S. Behnke, "Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D," IEEE Robotics & Automation Magazine, vol. 22, no. 4, pp. 110-124, 17 September 2015.
[30] "The Point Cloud Library (PCL)," [Online]. Available: https://github.com/PointCloudLibrary/pcl. [Accessed 15 July 2017].
[31] R. Lagamoere, OpenCV 2 Computer Vision Application Programming Cookbook, Packt Publishing Ltd, 2011.
[32] "Open Graphics Library (OpenGL)," [Online]. Available: https://www.khronos.org/opengl/wiki/Main_Page. [Accessed 15 July 2017].
[33] "Free OpenGL Utility Toolkit Library (GLUT)," [Online]. Available: http://freeglut.sourceforge.net/. [Accessed 7 2017].
[34] "OpenGL Extension Wrangler Library (GLEW)," [Online]. Available: http://glew.sourceforge.net/. [Accessed 15 July 2017].
[35] "Eigen," [Online]. Available: http://eigen.tuxfamily.org/index.php?title=Main_Page. [Accessed 15 July 2017].
[36] "The Open Toolkit Library (OpenTK)," [Online]. Available: https://github.com/opentk/opentk. [Accessed 15 July 2017].
[37] M. Sigalas, M. Pateraki, I. Oikonomidis, and P. Trahanias, "Robust Model-Based 3D Torso Pose Estimation in RGB-D Sequences," 2013 IEEE International Conference on Computer Vision Workshops, Sydney, NSW, Australiz, pp. 315-322, 2-8 December 2013.