自動駕駛車輛，是即將改變人們生活習慣的科技。2025年，世界將有數百萬輛自駕車在路上行駛，安全無疑是最重要的議題。本論文開發於多計算機平台，整合於自駕車的多感知融合追蹤系統，其中感知是確保安全的基礎，而障礙物偵測和追蹤是關鍵技術。自動駕駛系統包括車輛，感測器，通信設備和車載電腦。為提高系統性能，本論文提出了完整的感測器數據處理和整合架構。系統感測器相機鏡頭(Camera)、光達(Lidar)和雷達(Radar)在內的多個感測器，為此研究目標提供了可行性和互補性。感測器數據預處理，數據格式定義和融合算法開發，建置在基於機器人作業系統（ROS）的自駛車系統，該系統提供良好的模組化開發環境。
本論文應用深度學習、卡爾曼濾波器等演算法，實現前方障礙物的追蹤以及偵測，基於多感測器之訊號特性與資料差異性，提出整合的系統架構開發，實驗車輛與行人之追蹤及偵測，增加訊息可靠度，以及更完整的資訊內容，提供自動駕駛與先進駕駛輔助系統(ADAS)等其他應用，更可靠之感知技術。為提高安全性以減少交通事故，已在校內封閉道路測試。未來，將努力開發更完整的系統及更有效率的演算法，將功能從校園道路及測試場地拓展至城市道路。

Autonomous driving vehicle is the technology that will appear in daily life in the future. By 2025, the world will see millions of autonomous vehicles on the road, and its safety is undoubtedly the most important topic in this field. In this thesis, a multi-sensor data fusion system for multiple computers autonomous vehicle is developed. Environ-ment perception is the foundation for ensuring safety, so accurate obstacles detection and tracking are critical for autonomous driving. Autonomous driving system contains vehi-cle, sensors, communication equipment and on-board computers. To improve system performance, this thesis proposes a parallel computing mechanism on two computers and an integration architecture for sensor data processing and decision-making.
Multiple sensors including camera, lidar, and radar provide various data for this goal. Raw data preprocessing, data format definition, and fusion algorithm development are integrated in the NCKU autonomous vehicle system based on Robot Operating System (ROS), which provides a good modular development environment.
Camera has good classification ability; Lidar performs well in bad lighting envi-ronment; Radar provides long-range coverage and good velocity information. Deep Learning and Kalman Filter are used to implement detection-level and tracking-level fu-sion algorithm. Sensor fusion mechanism of autonomous vehicle is experimented on the campus road with vehicles and pedestrians. In the thesis, pedestrians tracking perfor-mance is in focus. In order to increase safety and decrease road accidents of autonomous vehicles, tests in a controlled environment and/or open fields are essential. In the future, we will make efforts to adapt the system from suiting the campus road to suiting the ur-ban road.

[1] Shinpei Kato, Eijiro Takeuchi, Yoshio Ishiguro, Yoshiki Ninomiya, Kazuya Takeda, and Tsuyoshi Hamada, "An Open Approach to Autonomous Vehicles," IEEE Micro, vol. 35, no. 6, pp. 60 - 68, 29 Dec. 2015.
[2] Xiaohui Li, Zhenping Sun, Dongpu Cao, Zhen He, and Qi Zhu, "Real-Time Trajectory Planning for Autonomous Urban Driving: Framework, Algorithms, and Verifications," IEEE/ASME Transactions on Mechatronics, vol. 21, no. 2, pp. 740 - 753, 26 Oct. 2015.
[3] Sung-Hyun Kim, Dong-Kyoung Kye, Sang-Hyun Lee, Soomok Lee, Myung-Ok Shin, and Seung-Woo Seo, "Autonomous Campus Mobility Services Using Driverless Taxi," IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 12, pp. 3513 - 3526, 19 Sep. 2017.
[4] Lin Zhao, Mohammed Quddus, Washington Ochieng, and Robert B. Noland, "An Extended Kalman Filter Algorithm for Integrating GPS and Low Cost Dead Reckoning System Data for Vehicle Performance and Emissions Monitoring," The Journal of Navigation, vol. 56, 2003.
[5] Hugh Durrant-Whyte and Tim Bailey, "Simultaneous Localization and Mapping: Part I," IEEE Robotics & Automation Magazine, vol. 13, no. 2, pp. 99 - 110, June 2006.
[6] Junqing Wei, Jarrod M. Snider, Tianyu Gu, John M. Dolan and Bakhtiar Litkouhi, "A Behavioral Planning Framework for Autonomous Driving," in Proceedings of IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, 2014.
[7] Enric Galceran, Alexander G. Cunningham, Ryan M. Eustice, and Edwin Olson, "Multipolicy Decision-Making for Autonomous Driving via Changepoint-based Behavior Prediction," Robotics: Science and Systems 2015, July 2015.
[8] Pan Zhao, Jiajia Chen, Yan Song, Xiang Tao, Tiejuan Xu, and Tao Mei, "Design of a Control System for an Autonomous Vehicle Based on Adaptive-PID," International Journal of Advanced Robotic Systems, vol. 9, no. 2, July 2012.
[9] Jaebum Choi, Simon Ulbrich, Bernd Lichte, and Markus Maurer, "Multi-Target Tracking Using a 3D-Lidar Sensor for Autonomous vehicles," in Proceedings of 16th International IEEE Conference on Intelligent Transportation Systems, 2013.
[10] Raul Dominguez, Enrique Onieva, J. Alonso, Jorge Villagra, and Christine Gonzalez, "LIDAR Based Perception Solution for Autonomous Vehicles," in Proceedings of 11th International Conference on Intelligent Systems Design and Applications, Cordoba, Spain, 2011.
[11] Michael Himmelsbach, Armin Müller, Thorsten Luettel, and Hans-Joachim Wünsche , "LIDAR-based 3D Object Perception," in Proceedings of 1st International Workshop on Cognition for Technical Systems, Munich, Germany, 2008.
[12] Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun, "Monocular 3D Object Detection for Autonomous Driving," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016.
[13] Zhaowei Cai, Mohammad Saberian, and Nuno Vasconcelos, "Learning Complexity-Aware Cascades for Deep Pedestrian Detection," in Proceedings of IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 2015.
[14] Juergen Dickmann, Jens Klappstein, Markus Hahn, Nils Appenrodt, Hans-Ludwig Bloecher, and Klaudius, "Automotive Radar the Key Technology for Autonomous Driving: From Detection and Ranging to Environmental Understanding," in Proceedings of IEEE Radar Conference, Philadelphia, PA, USA, 2016.
[15] R. Omar Chavez-Garcia, Julien Burlet, Trung-Dung Vu, and Olivier Aycard, "Frontal Object Perception using Radar and Mono-Vision," in Proceedings of IEEE Intelligent Vehicles Symposium, Alcala de Henares, Spain, 2012.
[16] Ricardo Omar Chavez-Garcia, and Olivier Aycard, "Multiple Sensor Fusion and Classification for Moving Object Detection and Tracking," IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 2, pp. 525 - 534, Feb. 2016.
[17] Hyunggi Cho, Young-Woo Seo, B.V.K. Vijaya Kumar, and Ragunathan (Raj) Rajkumar, "A Multi-Sensor Fusion System for Moving Object Detection and Tracking in Urban Driving Environments," in Proceedings of IEEE International Conference on Robotics and Automation (ICRA-2014), 2014.
[18] Min-Ki Park, Sang-Yeob Lee, Chan-Keun Kwon, and Soo-Won Kim, "Design of Pedestrian Target Selection With Funnel Map for Pedestrian AEB System," vol. 66, no. 5, pp. 3597 - 3609, May 2017.
[19] Wang Jun, Tao Wu, and Zhongyang Zheng, "LIDAR and Vision Based Pedestrian Detection and Tracking System," in Proceedings of IEEE International Conference on Progress in Informatics and Computing (PIC), Nanjing, China, 2015.
[20] Nico Kaempchen, Bruno Schiele, and Klaus Dietmayer, "Situation Assessment of an Autonomous Emergency Brake for Arbitrary Vehicle-to-Vehicle Collision Scenarios," IEEE Transactions on Intelligent Transportation Systems , vol. 10, no. 4, pp. 678 - 687, Dec. 2009.
[21] "SAE Standards News: J3016 Automated-Driving Graphic Update," SAE, [Online]. Available: https://www.sae.org/news/2019/01/sae-updates-j3016-automated-driving-graphic. [Accessed June 2019].
[22] "Apollo 自動駕駛感知技術," [Online]. Available: https://mp.weixin.qq.com/s/IIRQoAnEVgTbcmpTI2jo4g. [Accessed June 2019].
[23] Diebel, James, "Representing Attitude: Euler Angles, Unit Quaternions, and Rotation Vectors," Matrix, vol. 58, no. 15, Jan. 2006.
[24] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed , Cheng-Yang Fu, and Alexander C. Berg, "SSD: Single Shot MultiBox Detector," in Proceedings of ECCV, 2016.
[25] Bruce D. Lucas and Takeo Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," in Proceedings of DARPA Image Understanding Workshop, 1981.
[26] Jianbo Shi and Carlo Tomasi, "Good Features to Track," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, USA, 1994.
[27] Simon Baker and Iain Matthews, "Lucas-Kanade 20 Years On: A Unifying Framework," International Journal of Computer Vision, vol. 56, no. 3, p. 221–255, 2004.
[28] Stan Birchfield, "Derivation of Kanade-Lucas-Tomasi Tracking Equation," 1997.
[29] Rudolf E. Kálmán, "A New Approach to Linear Filtering and Prediction Problems," ASME–Journal of Basic Engineering, vol. 82, no. 1, pp. 34-45, 1 Mar. 1960..
[30] Greg Welch and Gary Bishop, "An Introduction to the Kalman Filter," SIGGRAPH, pp. 1-16, 2001.
[31] Nikos Floudas, Aris Polychronopoulos, Olivier Aycard, Julien Burlet, and Malte Ahrholdt, "High Level Sensor Data Fusion Approaches For Object Recognition In Road Environment," in Proceedings of IEEE Intelligent Vehicles Symposium, 2007.
[32] Michael Aeberhard and Nico Kaempchen, "High-Level Sensor Data Fusion Architecture for Vehicle Surround Environment Perception," in Proc. 8th Int. Workshop Intell. Transp, Hamburg, Germany, 2011.
[33] Huimin Chen and Yaakov Bar-Shalom, "Track Fusion with Legacy Track Sources," in Proceedings of 9th International Conference on Information Fusion, Florence, Italy, 2006.
[34] Yaakov Bar-Shalom, X. Rong Li, and Thiagalingam Kirubarajan, Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software, New York: Wiley: Wiley, 2004.
[35] Seong-Geun Shin, Dae-Ryong Ahn, and Hyuck-Kee Lee, "Occlusion Handling and Track Management Method of High-Level Sensor Fusion for Robust Pedestrian Tracking," in Proceedings of International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Daegu, South Korea, 2017.
[36] Samuel S. Blackman and Robert Popoli, Design and Analysis of Modern Tracking Systems, Artech House, 1999.
[37] SAMUEL S. BLACKMAN, "Multiple Hypothesis Tracking for Multiple Target Tracking," IEEE Aerospace and Electronic Systems Magazine, vol. 19, no. 1, pp. 5 - 18, Jan. 2004.
[38] Carine Hue, Jean-Pierre Le Cadre, and Patrick Pérez, "Sequential Monte Carlo Methods for Multiple Target Tracking and Data Fusion," IEEE Transactions on Signal Processing, vol. 50, no. 2, pp. 309 - 325, Feb. 2002.
[39] Quigley, Morgan., Conley, Ken., Gerkey, Brian P.., Faust, Josh., Foote, Tully., Leibs, Jeremy., Wheeler, Rob., and Ng, Andrew Y., "ROS: An Open-Source Robot Operating System," in Proceedings of ICRA Workshop on Open Source Software, 2009.
[40] "ROS.org," Open Source Robotics Foundation, [Online]. Available: http://wiki.ros.org/. [Accessed June 2019].
[41] "Point Cloud Library," [Online]. Available: http://pointclouds.org/. [Accessed June 2019].
[42] "OpenCV @ CVPR 2019," OpenCV team, [Online]. Available: https://opencv.org/. [Accessed June 2019].
[43] Everingham, M. and Van~Gool, L. and Williams, C. K. I. and Winn, J. and Zisserman, A., "pascal-voc-2007," [Online]. Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2007/. [Accessed June 2019].
[44] Everingham, M. , Van~Gool, L., Williams, C. K. I. , Winn, J. and Zisserman, A., "pascal-voc-2012," [Online]. Available: http://host.robots.ox.ac.uk/pascal/VOC/voc2012/. [Accessed June 2019].
[45] "Autoware Camera-LiDAR Calibration Package," Autoware, [Online]. Available: https://github.com/autowarefoundation/autoware/tree/master/ros/src/sensing/fusion/packages/autoware_camera_lidar_calibrator. [Accessed June 2019].
[46] Martin Velas, Michal Spanel, Zdenek Materna, and Adam Herout, "Calibration of RGB Camera With Velodyne LiDAR," in Proceedings of International Conference on Computer Graphics, 2014.
[47] "What Is Camera Calibration?," The MathWorks, Inc., [Online]. Available: https://www.mathworks.com/help/vision/ug/camera-calibration.html. [Accessed June 2019].
[48] "Delphi ESR Startup Guide," AutonomouStuff, 2015.
[49] Autoware, "Autoware," [Online]. Available: https://www.autoware.ai/. [Accessed June 2019].
[50] Wang Jun, Tao Wu, and Zhongyang Zheng, "LIDAR and Vision Based Pedestrian Detection and Tracking System," in Proceedings of IEEE International Conference on Progress in Informatics and Computing (PIC), Nanjing, China, 2015.