本文旨在解決自動駕駛車輛的交通號誌辨識任務。我們採用基於視覺的方法中的深度學習方法來執行交通號誌辨識。但是自動駕駛車輛上的邊緣設備計算資源有限，如果將整幅圖像作為模型輸入，推理時間成本會很高。因此，我們利用高精地圖來提取感興趣區域作為模型輸入。這個方法的另一個優點是可以通過高精地圖的先驗知識來去除圖片中不相關的背景減少物體檢測器產生的誤報。此外，我們提出了一種具有數據關聯的感興趣區域提取策略，即使在定位失敗時也能繼續執行交通號誌辨識任務。我們還構建了狀態機以提供平滑的紅綠燈狀態。實驗證明本文提出的方法可以在保持預測性能的同時降低推理時間的成本。

In this thesis, we focus on solving the task of traffic light recognition which is one of
the important perception tasks for an automated driving vehicle. We adopt the deep learning methods among vision-based methods to perform the traffic light recognition. The computing resources of edge devices on an automated driving vehicle are limited so that the inference time cost will be high if the entire image is used as model input. Therefore, we leverage the high-definition maps (HD maps) to extract the region of interest (ROI) as model input. Another advantage is that the false positives generated by object detectors can be effectively reduced by prior knowledge. In addition, we propose an ROI extraction strategy with data association to provide the ROI even when localization fails. We also build the state
machine to provide smooth traffic light states. The experiments show that the proposed method maintains the prediction performance and reduces inference time costs at the same time.

[1] A. Bochkovskiy, C. Wang and H. Liao, “YOLOv4: optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020
[2] A. Almagambetov, S. Velipasalar and A. Baitassova, “Mobile standards-based traffic light detection in assistive devices for individuals with color-vision deficiency,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, issue 3, 2015
[3] Z. Chen, Q. Shi and X. Huang, “Automatic detection of traffic lights using support vector machine,” in IEEE Intelligent Vehicles Symposium (IV), 2015
[4] K. Behrendt, L. Novak and R. Botros, “A deep learning approach to traffic lights: detection, tracking, and classification,” in IEEE International Conference on Robotics and Automation (ICRA), 2017
[5] N. Fairfield, C. Urmson and R. Botros, “Traffic light mapping and detection,” in IEEE International Conference on Robotics and Automation, 2011
[6] C. Janga, S. Choa, S. Jeong et al, “Traffic light recognition exploiting map and localization at every stage,” Expert Systems with Applications, vol. 88, pp. 290–304, 2017
[7] M. Hirabayashi, A. Sujiwo, A. Monrroy et al, “Traffic light recognition using high-definition map features,” Robotics and Autonomous Systems, vol. 111, pp. 62–72, 2019
[8] S. Han, J. Pool, J. Tran et al, “Learning both weights and connections for efficient neural network,” Advances in neural information processing systems, pp. 1135–1143, 2015
[9] M. Zhu and S. Gupta. “To prune, or not to prune: exploring the efficacy of pruning for model compression,” in International Conference on Learning Representations, 2018
[10] S. Gao, F. Huang, W. Cai et al, “Network pruning via performance maximization,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021
[11] N. Jouppi, C. Young, N. Patil et al, “In-datacenter performance analysis of a tensor processing unit,” in ACM/IEEE 44th Annual International Symposium on Computer Architecture, 2017
[12] NVIDIA, “NVIDIA Tesla V100 GPU architecture,” Accessed: Jul. 10, 2022. [Online]. Available: http://images.nvidia.com/content/volta-architecture/pdf/volta-architecture-whitepaper.pdf
[13] C. Zhu, K. Huang, S. Yang et al, “An efficient hardware accelerator for structured sparse convolutional neural networks on FPGAs,” IEEE Transactions on Very Large Scale Integration Systems, vol. 28, issue 9, 2020
[14] M. O. TAŞ, H. S. YAVUZ and A. YAZICI, “Updating HD-Maps for autonomous transfer vehicles in smart factories,” in International Conference on Control Engineering & Information Technology (CEIT), 2018
[15] Y. Kang and A. Magdy, “HiDaM: A unified data model for high-definition (HD) map data,” in International Conference on Data Engineering Workshops (ICDEW), 2020
[16] R. Liu, J. Wang and B. Zhang, “High definition map for automated driving: overview and analysis,” The Journal of Navigation, vol. 73, no. 2, pp. 324–341, 2020
[17] W. N. Tun, S. Kim, J.-W. Lee et al, “Open-source tool of vector map for path planning in autoware autonomous driving software,” in IEEE International Conference on Big Data and Smart Computing, 2019
[18] T. Foote, "tf: The transform library," in IEEE Conference on Technologies for Practical Robot Applications (TePRA), 2013
[19] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Transactions on pattern analysis and machine intelligence, vol. 22, no. 11, pp. 1330–1334, 2000.
[20] Opencv, “Camera calibration and 3D reconstruction,” Accessed: Jul. 3, 2022. [Online]. Available: https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html?highlight=calibratecamera
[21] MathWorks, “What is camera calibration,” Accessed: Jul. 3, 2022. [Online]. Available: https://www.mathworks.com/help/vision/ug/camera-calibration.html#bu0niag
[22] Kuhn HW, “The hungarian method for the assignment problem,” Princeton University, Naval Research Logistics Quarterly 2, pp. 83–97,1955
[23] R. Girshick, J. Donahue, T. Darrell et al, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2014
[24] J. Redmon, S. Divvala, R. Girshick et al, “You only look once: unified, real-time object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2016
[25] ROS, “ROS.org | Wiki: Documentation,” Accessed: Jun. 3, 2022. [Online]. Available: http://wiki.ros.org/
[26] Autoware, “What is Autoware,” Accessed: Jun. 4, 2022. [Online]. Available: https://github.com/autowarefoundation/autoware/tree/autoware-ai
[27] F. Munir, S. Azam, A. Sheri et al, “Where am I: localization and 3D maps for autonomous vehicles,” VEHITS, 2019