本研究欲開發一套能夠根據環境而做學習的四旋翼機避障系統，吾人提出一種增強式學習與模糊控制理論結合的方法。四旋翼機避障系統的核心演算法為模糊控制演算法，使用人工智慧來對避障系統做修正；人工智慧有很多分支，藉由機器學習中的增強式學習，可以對整個控制過程中的變數做學習，來完成對環境要求的系統。在避障系統中，模糊控制演算法裡的歸屬函數會對整個系統的輸出有影響，本研究使用增強式學習的Q-Learning對控制系統做優化，透過更動歸屬函數及模糊控制的明確輸出大小，來做為增強式學習的動作。每次更動後測試的過程為狀態，在確實的經過整個流程後，可以完成對四旋翼機避障系統的學習。在避障的過程中，會限制使用者權限的控制來確保避障過程不會受其干擾，進而安全達成避障，並且設計無線傳輸模組來將實驗數據傳輸到地面資料站，可與原有地面站做資料比對及互補。最後藉由飛行測試，對實驗環境做學習的實際飛行測試，驗證所開發的避障系統可行性。

This thesis is to develop a collision avoidance system that can smartly learn from the environment used for quadcopters. This research proposed a method for tuning and learning a fuzzy logic controller based on reinforcements from a collision avoidance system. A collision avoidance system is built on the fuzzy control system with intense algorithm. There are several branches of artificial intelligence in this decade. Through the reinforcement learning, one of machine learning techniques, the variables in the collision avoidance system can be effectively tuned for the requirements of various environments. In this development of collision avoidance system, the membership functions in the fuzzy control algorithm affect the output of the system. Q-Learning algorithm is a reinforcement learning technique used in such machine learning. Q-Learning algorithm action can effectively tune the membership functions and crisp output. The machine state can be immediately responded after every observation of change. Therefore, the collision avoidance system can be completely optimized after the reinforcement learning process. The most importance of avoiding obstacles for this system, the system should partially restrict the signals from undefined control to ensure that the obstacle avoidance process does not suffer interference from them. The wireless transmission module is also designed to deliver the experimental data to the data receiving station on ground. The obtained experimental data is compared with the data from open source ground station. This processing is beneficial for the confirmation of the authenticity and complement for each other. Moreover, It is also verified the feasibility of the developed collision avoidance system through a couple of real flights. The contribution of the fuzzy control for quadcopter collision avoidance system demonstrates the potential capability of industrial application in the future.

[1] E. Dourado, R. Hagemann, and A. Thierer, “Operation and Certification of Small Unmanned Aircraft Systems,” Public Interest Comment, Mercatus Center at George Mason University, Arlington, VA, 2015.
[2] X. Li and B.-J. Choi, “Design of obstacle avoidance system for mobile robot using fuzzy logic systems,” International Journal of Smart Home, vol.7, no.3, pp.321-328, 2013.
[3] Z. Xu, R. Wei, Q. Zhang, K. Zhou, and R. He, “Obstacle avoidance algorithm for UAVs in unknown environment based on distributional perception and decision making,” Guidance, Navigation and Control Conference (CGNCC), 2016 IEEE Chinese, pp.1072-1075, 2016.
[4] P. K. Mohanty, A. K. Sah, V. Kumar, and S. Kundu, “Application of Deep Q-Learning for Wheel Mobile Robot Navigation,” Computational Intelligence and Networks (CINE), 3rd International Conference on, pp.88-93, 2017.
[5] 許時準, “以機器學習法自動產生模糊控制規則,” 國立臺灣大學資訊工程研究所碩士論文, 1994.
[6] 吳秉勳, “四旋翼防撞系統設計與驗證研究,” 國立成功大學航空太空工程研究所碩士論文, 2015.
[7] 陳俊維, “應用決策樹演算法於四旋翼防撞系統設計與驗證,” 國立成功大學航空太空工程研究所碩士論文, 2017.
[8] 林宜陞, “智慧型機器人應用基因演算法結合增強式學習於模糊控制歸屬函數之最佳化,” 國立高雄大學電機工程研究所碩士論文, 2012.
[9] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction (no.1). MIT press Cambridge, 1998.
[10] S. F. Desouky and H. M. Schwartz, “A novel technique to design a fuzzy logic controller using Q (λ)-learning and genetic algorithms in the pursuit-evasion game,” Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on, pp.2609-2615, 2009.
[11] L. A. Zadeh, “Fuzzy Sets,” Information and control, vol.8, pp.338-353, 1965.
[12] 賴耿陽, 紅外線工學基礎應用, 台灣復文興業股份有限公司, 2001.
[13] Ardupilot Autopilot Suite, http://ardupilot.com/
[14] TeraRanger One User Manual, https://www.terabee.com/
[15] Mission Planner Overview, http://ardupilot.org/planner/