本論文主旨基於深度學習物件偵測技術運用在航空和飛機維修安全。在20世紀1980年代「噴射發動機」與「線傳飛控」技術被開發並安裝到飛機上，以提升飛行燃油效率和飛行安全。一直以來航空安全發展變化巨大，最前沿的深度學習技術不斷維新，在飛行安全方面的需求也繼續超越極限。藉此，國際民航組織在2020年代末期大力推動深度學習技術發展，被航空產業界視為能預測「機場跑道入侵」和「飛機機械故障」創新系統的契機。深度學習技術的進步被認為是簡化和自動化分析飛機維修，以及許多內部的流程、任務方法和程序。發展航空產業的深度學習系統著重在以人為本的核心，例如自動駕駛系統、跑道起降、乘客舒適度等，應用深度學習技術改善航空和飛機維修安全在自動維修和保養工廠。本論文實驗基於ASMIS和AE-RTIS模型的深度學習技術開發，以神經網路演算法，進行了600多次迭代訓練、測試和檢測取得成果，開發具備現代標準的檢測、識別和分類的系統，其平均精度達到了0.9，多核心平行運算時間只需要大約 100 微秒。因此，發表了「機場標誌及標記檢測系統網路」與「航空發動機放射線照相檢測系統網路」模型，基於更快速區域的卷積神經網路和旋轉識別技術為基礎，強化驗證且有效率的提高圖像檢測和識別的精度。本論文的主要貢獻是為深度學習技術在航空和飛機維修安全方面的創意與應用提出研究成果。

The dissertation applies deep learning (DL)-based object detection methods for the improvement of aircraft maintenance and aviation safety (AVS), which has a significant impact on personal safety and consequential economic benefits. In the 1980s, jet engines and fly-by-wire technologies were developed and installed in aircraft to enhance fuel efficiency and operational safety during flight. Aviation safety is changing dramatically and DL technologies are at the forefront of innovative improvements to meet the needs of passengers in terms of flight safety. As such, the International Civil Aviation Organization’s efforts to promote the revival of DL technological developments in the late 2020s were deemed by the aviation industry as an opportunity to develop innovative systems for predicting runway incursions and aircraft mechanical failures. Advances in DL technologies are thereby regarded as methods and procedures to streamline and automate the analysis of aircraft maintenance, AVS, as well as many other internal processes and tasks. Furthermore, DL systems for the aviation community are focused on human-centric developments, such as automatic pilot systems, runway take-off and landing procedures, passenger comfort, and automated repair and maintenance plants, that apply DL technologies to improve aviation and aircraft maintenance safety. To this end, the experiment in this thesis is based on DL layers of airport signs and markings inspection system (ASMIS) and aeronautics engine radiographic testing inspection system net (AE-RTISNet) models, in which an optimized convolutional neural network (CNN) algorithm is trained using more than 600 iterations for training, testing, and validation to develop a modern standard of object detection, identification, and classification. The results achieved a mean average precision of 0.9, while parallel computing only required approximately 100 microseconds. As a result, ASMIS and AE-RTISNet models were published based on a faster region-based CNN, and rotation-based identification technology was used to augment and effectively improve the accuracy of image detection and recognition. The proposed networks were verified using experimental data in practice. The main contribution of this dissertation is the innovation and application of DL techniques for aviation and aircraft maintenance safety by providing useful reference information.

[1] S. Akcay, M. E. Kundegorski, C. G. Willcocks and T. P. Breckon, “Using Deep Convolutional Neural Network Architectures for Object Classification and Detection within X-ray Baggage Security Imagery,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 9, pp. 2203-2215, 2018.
[2] E. D. S. Alves, J. B. O. S. Filho, R. M. Galliez and A. Kritski, “Specialized MLP Classifiers to Support the Isolation of Patients Suspected of Pulmonary Tuberculosis,” in Proceedings of the BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence, pp. 40-45, 2014.
[3] R. Amalberti and L. Wioland, “Human Error in Aviation,” Aviation Safety, pp. 91-108, 2020.
[4] J. Barker, S. Sarathy and A. Tao, “DetectNet: Deep Neural Network for Object Detectionin DIGITS,” Deep Learning for Object Detection with DIGITS by Nvidia, pp.1-3, 2016.
[5] P. Aswathy, Siddhartha and D. Mishra, “Deep GoogLeNet Features for Visual Object Tracking,” in Proceedings of the IEEE 13th International Conference on Industrial and Information Systems (ICIIS), pp. 60-66, 2019.
[6] E. M. Beth pasztor, “Statistical Summary of Commercial Jet Airplane Accidents worldwide operationals Worldwide Operations 1959-2019,” Airplane Safety Boeing Commerical Airplanes, pp. 1-28, 2020.
[7] D. Bhargava, and K. Marais, “Archetypal Models of Runway Incursions,” in Proceedings of the 17th AIAA Aviation Technology, Integration, and Operations Conference, pp. 4392-4396, 2017.
[8] N. T. S. Board, “DCA18MA142 SWA1380 Investigative Update,” in Proceedings of the National Transportation Safety Board Investigative Update Southwest Airlines Flight 1380 Engine Failure, pp. 1-6, 2018.
[9] I. Buck, “GPU Computing: Programming a Massively Parallel Processor,” in Proceedings of the International Symposium on Code Generation and Optimization (CGO'07), pp. 17-21, 2007.
[10] A. T. S. Bureau, “Taxiway Design, Flight Crew Distraction and Incorrect Mental model, Lead to Perth Airport Runway Incursion,” Australia’s National Transport Safety Investigator, 2020.
[11] K. Button, “Containing a Blade-out,” Aerospace America (AIAA), pp. 1-3, 2018.
[12] Z. Chen, Y. Zheng, B. R. Abidi, D. L. Page, and M. A. Abidi, “A Combinational Approach to the Fusion, De-noising and Enhancement of Dual-energy X-ray Luggage Images,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05)-Workshops, pp. 2-6, 2005.
[13] Z. H. Chen and J. C. Juang, “Using Deep Learning Edge Detection to Improve Non-Destructive Radiographic Tests of Engine,” in Proceedings of the International Symposium on Computer, Consumer and Control (IS3C), Taichung, Taiwan, pp. 1227-157, 2020.
[14] Z. H. Chen and J. C. Juang, “COVID-19 Chest Radiography Employing via Deep Convolutional Stacked Autoencoders,” in Proceedings of the International Automatic Control Conference (CACS), pp. 1-7, 2020.
[15] Z. H. Chen and J. C. Juang, “Neural Network Learning to Identify Airport Runway Taxiway Numbers,” in Proceedings of the International Symposium on Computer, Consumer and Control (IS3C), Taichung, Taiwan, pp. 153-157, 2019.
[16] A. Cho, J. Kim, S. Lee, B. Kim, N. Park, D. Kim, and C. Kee, “Fully Automatic Taxiing, Takeoff and Landing of a UAV Based on a Single-antenna GNSS Receiver,” in Proceedings of the 17th World CongressThe International Federation of Automatic ControlSeoul (IFAC), Korea, vol. 41, no. 2, pp. 4719-4724, 2008.
[17] H. Cui, and N. Dahnoun, “Real-Time Stereo Vision Implementation on Nvidia Jetson TX2,” in Proceedings of the 8th Mediterranean Conference on Embedded Computing (MECO), pp. 1-5, 2019.
[18] X. Dong, C. J. Taylor, and T. F. Cootes, “Automatic Inspection of Aerospace Welds Using X-Ray Images,” in Proceedings of the 24th International Conference on Pattern Recognition (ICPR), pp. 2002-2007, 2018.
[19] A. Fuentes, S. Yoon, S. C. Kim, and D. S. Park, “A Robust Deep-Learning-Based Detector for Real-Time Tomato Plant Diseases and Pests Recognition,” Sensors (Basel), vol. 17, no. 9, 2017.
[20] S. Gholizadeh, “A Review of Non-destructive Testing Methods of Composite Materials,” Procedia Structural Integrity, vol. 1, pp. 50-57, 2016.
[21] R. Girshick, “Fast R-CNN,” Computer Vision and Pattern Recognition, arXiv:1504.08083, pp. 1440-1448, 2015.
[22] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.
[23] Y. Gong, H. Shao, J. Luo, and Z. Li, “A Deep Transfer Learning Model for Inclusion Defect Detection of Aeronautics Composite Materials,” Composite Structures, vol. 252, pp. 1-11, 2020.
[24] G. Henry, “Howard Chu on Lightning Memory-Mapped Database,” IEEE Software, vol. 36, no. 6, pp. 83-87, 2019.
[25] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, “Large-Scale Video Classification with Convolutional Neural Networks,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 1725-1732, 2014.
[26] D. R. Kelley, and J. G. Steinbacher, “An Analysis of Pilot Deviations in Runway Incidents,” Air Traffic Control Quarterly, vol. 1, no. 2, pp. 197-221, 2016.
[27] M. Komar, P. Yakobchuk, V. Golovko, V. Dorosh, and A. Sachenko, “Deep Neural Network for Image Recognition Based on the Caffe Framework,” in Proceedings of the IEEE Second International Conference on Data Stream Mining & Processing (DSMP), pp. 102-106, 2018.
[28] P. Krus, R. Braun, P. Nordin, and B. Eriksson, “Aircraft System Simulation for Preliminary Design,” in Proceedings of the 28th International Congress of the Aeronautical Sciences, Brisbane Australia, pp. 1-10, 2018.
[29] F. Kurtulmus, and I. Kavdir, “Detecting Corn Tassels Using Computer Vision and Support Vector Machines,” Expert Systems with Applications, vol. 41, no. 16, pp. 7390-7397, 2014.
[30] Y. Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Handwritten Digit Recognition with a Back-propagation Network,” in Proceedings of the Proceedings of the 2nd International Conference on Neural Information Processing Systems, pp. 396-404, 1990.
[31] Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature, vol. 521, no. 7553, pp. 436-44, 2015.
[32] J. E. Little, X. Yuan, and M. I. Jones, “Characterisation of Voids in Fibre Reinforced Composite Materials,” NDT & E International, vol. 46, pp. 122-127, 2012.
[33] X. Y. Liu, D. A. Zhao, W. K. Jia, W. Li, C. Z. Ruan, and Y. P. Sun, “Cucumber Fruits Detection in Greenhouses Based on Instance Segmentation,” IEEE Access, vol. 7, pp. 139635-139642, 2019.
[34] D. F. Mathaisel, “A Lean Architecture for Transforming the Aerospace Maintenance, Repair and Overhaul (MRO) Enterprise,” International Journal of Productivity and Performance Management, vol. 54 no. 8, pp. 623-644, 2005.
[35] D. Mery, “Computer Vision Technology for X-ray Testing,” Insight Non-destructive Testing and Condition Monitoring, vol. 56, no. 3, pp. 147-155, 2014.
[36] D. Mery, V. Riffo, U. Zscherpel, G. Mondragon, I. Lillo, I. Zuccar, H. Lobel, and M. Carrasco, “GDXray: The Database of X-ray Images for Nondestructive Testing,” Journal of Nondestructive Evaluation, vol. 34, no. 4, pp. 42-50, 2015.
[37] News posted by J.T. Burn, “Two Passenger Planes (a Korean Airlines B777 and an Asiana Airlines A330) at Seoul’s Gimpo International Airport in South Korea collided on the ground near a boarding gate at about 8:00 a.m.,” in Proceedings of the Airline Routes and Ground Services, 2018.
[38] K. Y. Patrick, “EASA Artificial Intelligence Roadmap Published a Human-centric Approach to AI in Aviation,” in Proceedings of the European Aviation Safety Agency, pp. 1-30, 2020.
[39] J. A. Pontecorvo, “MSG-3–A Method For Maintenance Program Planning,” SAE Technical Paper 841485, pp.1-250, 1984.
[40] S. Rahman, and D. A. Robertson, “Classification of Drones and Birds Using Convolutional Neural Networks Applied to Radar Micro-Doppler Spectrogram Images,” Iet Radar Sonar and Navigation, vol. 14, no. 5, pp. 653-661, 2020.
[41] J. Redmon, and A. Farhadi, “Yolov3: An Incremental Improvement,” arXiv preprint:1804.02767, pp.1-6, 2018.
[42] S. Q. Ren, K. M. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2016.
[43] G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez, “The Synthia Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3234-3243, 2016.
[44] P. J. Schilling, B. R. Karedla, A. K. Tatiparthi, M. A. Verges, and P. D. Herrington, “X-ray Computed Microtomography of Internal Damage in Fiber Reinforced Polymer Matrix Composites,” Composites Science and Technology, vol. 65, no. 14, pp. 2071-2078, 2005.
[45] R. Sikora, P. Baniukiewicz, T. Chady, P. Łopato, B. Piekarczyk, G. Psuj, B. Grzywacz, and L. Misztal, “Detection and Classification of Weld Defects in Industrial Radiography with Use of Advanced AI Methods,” in Proceedings of the Far East Forum on Nondestructive Evaluation/Testing: New Technology and Application, pp. 12-17, 2013.
[46] M. Singh, and S. Singh, “Image Segmentation Optimisation for X-ray Images of Airline Luggage,” in Proceedings of the IEEE International Canfcrenee on Computational Intelligence for Homeland Security and Personal Safety Venice, Italy, pp. 10-17, 2004.
[47] S. Soderman, A. Kola, M. Podkorytov, M. Geyer, and M. Gubanov, “Hybrid. ai: A Learning Search Engine for Large-scale Structured Data,” in Proceedings of the Companion Proceedings of the The Web Conference, pp. 1507-1514, 2018.
[48] V. Turchenko, E. Chalmers, and A. Luczak, “A Deep Convolutional Auto-encoder with Pooling-unpooling Layers in Caffe,” arXiv preprint:1701.04949, 2017.
[49] S. H. Wang, K. Muhammad, J. Hong, A. K. Sangaiah, and Y. D. Zhang, “Alcoholism Identification via Convolutional Neural Network Based on Parametric ReLU, Dropout, and Batch Normalization,” Neural Computing & Applications, vol. 32, no. 3, pp. 665-680, 2018.
[50] F. Yang, H. Fan, P. Chu, E. Blasch, and H. Ling, “Clustered Object Detection in Aerial Images,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8310-8319, 2019.
[51] Q. Zhang, X. Chang, and S. B. Bian, “Vehicle-Damage-Detection Segmentation Algorithm Based on Improved Mask R-CNN,” IEEE Access, vol. 8, pp. 6997-7004, 2020.
[52] S. Shi. Sa, “Taiwan AI Federated Learning Alliance,” in Proceedings of the Global Partnership on Artificial Intelligence (GPAI) Taiwan AILabs Economy Daily News, pp.1-2, 2020.