成功大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	黃宗穎 Huang, Zong-Ying
論文名稱：	使用電腦視覺與深度學習於無人機空中防撞應用 Implementation of Computer Vision and Deep Learning on Mid-Air Collision Avoidance of UAVs
指導教授：	賴盈誌 Lai, Ying-Chih
學位類別：	碩士 Master
系所名稱：	工學院 - 航空太空工程學系 Department of Aeronautics & Astronautics
論文出版年：	2020
畢業學年度：	108
語文別：	中文
論文頁數：	58
中文關鍵詞：	Deep learning 、Sense and avoid 、Distance estimation.
外文關鍵詞：	深度學習, 感知與避障, 距離估算
相關次數：	點閱：163 下載：87
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

在各領域中，距離偵測在障礙物避讓的議題中扮演著重要的角色，例如在自駕車系統上，感測與前車的距離。只要能夠計算出與障礙物之間的距離，就可以決定是否有碰撞的風險存在。本論文使用深度學習估算出自己擁有的無人機(與其他目標無人機的距離，並且只使用一個相機作為感測器。使用YOLO偵測器來偵測目標物，並通過一個基於深度學習之距離模型估測與目標無人機的距離。運算流程如下，首先將影像中YOLO所偵測到的無人機區域，以無人機為中心點分割出一個200乘200的圖片，並將其作為距離模型的輸入，輸出則為自己與目標物無人機的距離。在本論文中，將距離估測視為一個捲積神經網路(Convolutional Neural Network, CNN)的迴歸問題。CNN距離模型由兩個網路架構所組成，分別為特徵提取與距離迴歸。首先，分割出的圖片會先通過特徵提取網路，提取出無人機的特徵，再將所得到的特徵當作距離網路之輸入迴歸出一個估測的距離。本論文同時卡爾曼濾波器來平滑CNN距離模型所估測的距離，作為目標物之追蹤器，以避免YOLO失效時丟失目標物的情況發生。CNN距離模型的訓練採用使用人造的圖片訓練，並且在人造影片及實際飛行影片作CNN距離模型的驗證。

Distance estimation of the target object is an important information for obstacle avoidance in many areas, such as autonomous cars. When the distance of an obstacle is calculated, one can determine the potential risk of object collision. In this paper, a monocular camera was utilized to obtain the distance from an incoming unmanned aerial vehicle (UAV) using a deep learning approach. The object detection of a UAV in this work is based on the you only look once (YOLO) object detector, and distance estimation using a deep learning approach is proposed in this study. The region containing the detected UAV was enclosed in a region of 200 by 200 pixels and was input into the proposed model to estimate the distance between the target object and determine the status of the drone. In the proposed model, a convolutional neural network (CNN) was adopted to solve the regression problem. First, the feature extraction network based on VGG16 was performed, and then the results were applied to the distance network to estimate distance. In addition, the Kalman filter was used to achieve object tracking when the YOLO detector was not able to detect the UAV, as well as to smooth the estimated distance. The proposed model was trained only by using synthetic images from Blender animation software and was validated using both synthetic and real flight tests.

中文摘要................................................	I
Abstract.................................................II
Acknowledgements......................................III
Contents...............................................IV
List of Tables.........................................VI
List of Figures.........................................1
Introduction....................................3
1	Research Background.............................3
2	Motivation and Objectives.......................5
3	Literature Review...............................5
4	Thesis Outline..................................7
Research Overview...............................8
1	Research Process................................8
2	Architecture of Vision-Based Sense and Avoid....9
Object detection...............................11
1	Object at a Long Distance......................11
1.1	Background Subtraction.........................12
1.2	Optical Flow Algorithm.........................12
1.3	Estimation of a Homogeneous Matrix and Image 
        Perspective Transform..........................14
1.4	Get Background Subtracted Image and Target.....14
2	Object at a short distance.....................18
2.1	YOLO Detector..................................18
2.2	Data Collection................................19
2.3	Detection Results..............................20
Distance Estimation............................22
1	Distance estimation using CNN..................22
1.1	Convolutional Neural Network...................23
1.2	Model Architecture.............................25
1.3	Data Collection................................27
1.4	Training Results...............................31
2	Distance Estimation using DNN..................32
2.1	Attitude Estimation via CNN....................33
2.2	Bounding Box Rectification.....................34
2.3	DNN Architecture...............................35
3	Comparison Between the CNN and DNN Distance 
        Regression.....................................36
Model Evaluation and Real Flight Experiments...38
1	Model Evaluation in the Synthetic Videos.......38
2	Model Evaluation in Real Flight Videos.........43
3	Result and Discussion..........................52
Conclusion and Future Work.....................54
References.....................................56
                                    

[1] F. Nex and F. Remondino, "UAV for 3D mapping applications: a review," Applied geomatics, vol. 6, no. 1, pp. 1-15, 2014.
[2] S. Xu, A. Savvaris, S. He, H.-s. Shin, and A. Tsourdos, "Real-time implementation of YOLO+ JPDA for small scale UAV multiple object tracking," in 2018 International Conference on Unmanned Aircraft Systems (ICUAS), 2018: IEEE, pp. 1336-1341.
[3] J. Li, D. H. Ye, T. Chung, M. Kolsch, J. Wachs, and C. Bouman, "Multi-target detection and tracking from a single camera in Unmanned Aerial Vehicles (UAVs)," in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016: IEEE, pp. 4992-4997.
[4] N. Ammour, H. Alhichri, Y. Bazi, B. Benjdira, N. Alajlan, and M. Zuair, "Deep learning approach for car detection in UAV imagery," Remote Sensing, vol. 9, no. 4, p. 312, 2017.
[5] K. Uto, H. Seki, G. Saito, and Y. Kosugi, "Characterization of rice paddies by a UAV-mounted miniature hyperspectral sensor system," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, no. 2, pp. 851-860, 2013.
[6] J. James, J. J. Ford, and T. L. Molloy, "Learning to Detect Aircraft for Long-Range Vision-Based Sense-and-Avoid Systems," IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 4383-4390, 2018.
[7] X. Yu and Y. Zhang, "Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects," Progress in Aerospace Sciences, vol. 74, pp. 152-166, 2015.
[8] R. Carnie, R. Walker, and P. Corke, "Image processing algorithms for UAV" sense and avoid"," in Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., 2006: IEEE, pp. 2848-2853.
[9] J. Lai, J. J. Ford, L. Mejias, and P. O'Shea, "Characterization of Sky‐region Morphological‐temporal Airborne Collision Detection," Journal of Field Robotics, vol. 30, no. 2, pp. 171-193, 2013.
[10] A. Nussberger, H. Grabner, and L. Van Gool, "Aerial object tracking from an airborne platform," in 2014 international conference on unmanned aircraft systems (ICUAS), 2014: IEEE, pp. 1284-1293.
[11] Q. Zhu, M.-C. Yeh, K.-T. Cheng, and S. Avidan, "Fast human detection using a cascade of histograms of oriented gradients," in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), 2006, vol. 2: IEEE, pp. 1491-1498.
[12] D. G. Lowe, "Object recognition from local scale-invariant features," in Proceedings of the seventh IEEE international conference on computer vision, 1999, vol. 2: Ieee, pp. 1150-1157.
[13] P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, 2001, vol. 1: IEEE, pp. I-I.
[14] C. Liu, F. Chang, and C. Liu, "Cascaded split-level colour Haar-like features for object detection," Electronics Letters, vol. 51, no. 25, pp. 2106-2107, 2015.
[15] D. H. Ye, J. Li, Q. Chen, J. Wachs, and C. Bouman, "Deep Learning for Moving Object Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (UAVs)," Electronic Imaging, vol. 2018, no. 10, pp. 466-1-466-6, 2018.
[16] Z.-Q. Zhao, P. Zheng, S.-t. Xu, and X. Wu, "Object detection with deep learning: A review," IEEE transactions on neural networks and learning systems, vol. 30, no. 11, pp. 3212-3232, 2019.
[17] S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," in Advances in neural information processing systems, 2015, pp. 91-99.
[18] K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969.
[19] W. Liu et al., "Ssd: Single shot multibox detector," in European conference on computer vision, 2016: Springer, pp. 21-37.
[20] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779-788.
[21] M. Rezaei, M. Terauchi, and R. Klette, "Robust vehicle detection and distance estimation under challenging lighting conditions," IEEE transactions on intelligent transportation systems, vol. 16, no. 5, pp. 2723-2743, 2015.
[22] M. Monajjemi, S. Mohaimenianpour, and R. Vaughan, "UAV, come to me: End-to-end, multi-scale situated HRI with an uninstrumented human and a distant UAV," in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016: IEEE, pp. 4410-4417.
[23] H. Ponce, J. Brieva, and E. Moya-Albor, "Distance estimation using a bio-inspired optical flow strategy applied to neuro-robotics," in 2018 International Joint Conference on Neural Networks (IJCNN), 2018: IEEE, pp. 1-7.
[24] M. A. Haseeb, J. Guan, D. Ristić-Durrant, and A. Gräser, "DisNet: a novel method for distance estimation from monocular camera," 10th Planning, Perception and Navigation for Intelligent Vehicles (PPNIV18), IROS, 2018.
[25] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXiv preprint arXiv:1804.02767, 2018.
[26] T.-Y. Lin et al., "Microsoft coco: Common objects in context," in European conference on computer vision, 2014: Springer, pp. 740-755.
[27] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.

校內：2022-08-31公開
校外：2022-08-31公開

簡易檢索 / 詳目顯示

相關論文