簡易檢索 / 詳目顯示

回結果列表
研究生： 田智輝
Tien, Chih-Hui
論文名稱： 混合人工蜂群演算法及增強式學習法於模糊控制器設計之研究
Study of Hybrid Artificial Bee Colony Algorithm with Reinforcement Learning to Fuzzy Controller Design
指導教授： 李祖聖
Li, Tzuu-Hseng S.
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系碩士在職專班
Department of Electrical Engineering (on the job class)
論文出版年： 2018
畢業學年度： 106
語文別： 英文
論文頁數： 65
中文關鍵詞： 人工蜂群演算法 、模糊控制器 、增強式學習法
外文關鍵詞： Artificial bee colony algorithm, Fuzzy controller, Reinforcement learning
相關次數： 點閱：100  下載：2 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 本論文提出了混合人工蜂群演算法及增強式學習法中的Q-learning於模糊控制器設計。由於最佳化演算法有探索與開發難以得到平衡情形，為了加快收斂速度以及避免落入區域最佳解，本論文利用增強式學習法中的獎勵與懲罰概念混合了人工蜂群演算法，使人工蜂群演算法可改善落入區域最佳解以及有提早收斂的情形。本論文將所提出的混合人工蜂群演算法及Q-learning進行模擬實驗最佳化常用的23個測試函數，其中包含了單峰值及多峰值測試函數，經過性能測試，從模擬實驗結果顯示混合人工蜂群演算法及Q-learning得到最佳解的次數較多。混合人工蜂群演算法及Q-learning與人工蜂群演算法比較，其結果顯示混合人工蜂群演算法及Q-learning的收斂速度較快，再利用混合人工蜂群演算法及Q-learning於非線性系統的模糊控制器參數最佳化，將設計模糊控制器內所需調整參數，使其參數最佳化，經由模擬結果證明此方法之有效性與可行性。

    The present study utilizes a hybrid artificial bee colony (ABC) algorithm with reinforcement learning Q-learning to design a fuzzy controller. The optimization method demonstrates a poor balance between exploration and exploitation. In order to accelerate the rate of convergence and avoids convergence to a local optimum. Therefore we use the concepts of reward and penalty from reinforcement learning methods to train a hybrid ABC algorithm. Our study works on the principle of reward and penalty to assist the ABC algorithm to avoid premature convergence and escape the local optimum. The proposed hybrid ABC algorithm with Q-learning is successfully applied to solving 23 benchmark problems of global numerical optimization which have unimodal and multimodal functions. The simulation results show that the proposed hybrid ABC algorithm with Q-learning offers more opportunities to find the optimal solutions. Simultaneously, the ABC algorithm with Q-learning is compared with the original ABC algorithm. The results show that the hybrid ABC algorithm with Q-learning has a superior performance in convergence rate. Finally, this study aims to tune the parameters of the fuzzy logic controller for nonlinear systems. The simulation results demonstrate the effectiveness and feasibility of the proposed method.

    Contents Abstract I Acknowledgement Ⅲ Contents Ⅳ List of Figures VI List of Tables VIII Chapter 1. Introduction 1.1 Motivation 1 1.2 Thesis Organization 4 Chapter 2. Overview of the Artificial Bee Colony Algorithm 2.1 Introduction of Artificial Bee Colony Algorithm 5 2.1.1 Initialization 7 2.1.2 Employed bees 7 2.1.3 Onlooker bee 7 2.1.4 Scout bees 8 2.1.5 Termination condition 8 2.2 Pseudo code of artificial bee colony algorithm 8 2.3 Flowchart of artificial bee colony algorithm 10 Chapter 3. Overview of the Reinforcement Learning 3.1 Introduction of reinforcement learning 12 3.2 SARSA 14 3.3 Q-learning 14 3.4 Summary 15 Chapter 4. Hybrid Artificial Bee Colony Algorithm with Reinforcement Learning 4.1 Introduction 16 4.1.1 Initialization 17 4.1.2 Q-tables assisting employed bees 21 4.1.3 Q-tables assisting onlooker bees 21 4.2.4 Scout bees and reset Q-tables 22 4.2.5 Termination condition 22 4.2 Pseudo code of artificial bee colony algorithm with Q-learning 22 4.3 Flowchart of hybrid artificial bee colony algorithm with Q-learning 25 4.4 Summary 27 Chapter 5. Simulation Results 5.1 Introduction 28 5.2 Benchmark functions 29 5.2.1 Unimodal benchmark function 29 5.2.2 Multimodal benchmark function 31 5.2.3 Fixed-dimension benchmark function 33 5.2.4 Algorithm control parameter setting 35 5.2.5 Benchmark function experiments results 37 5.3 Fuzzy controller design 43 5.3.1 Inverted pendulum system 45 5.3.2 Truck backer upper control system 48 5.3.3 Boat problem 51 5.3.4 Cart–pole inverted pendulum system 55 Chapter 6. Conclusion and Future Work 6.1 Conclusion 60 6.2 Future Work 61 References

    [1] L. A. Zadeh, “Fuzzy sets,” Information Control, vol. 8, pp. 338-352, Jun. 1965.
    [2] E. H. Mamdani, “Applications of fuzzy algorithms for simple dynamic plant,” in Proceedings of the Institution of Electric Engineers, vol. 121, no. 12, pp. 1585-1588, Dec.1974.
    [3] C.-Y. Chen, T.-H. S. Li, Y. -C. Yeh, and C.-C. Chang, “Design and implementation of an adaptive sliding-mode dynamic controller for wheeled mobile robots,” Mechatronics, vol. 19, no. 2, pp. 155-166, Mar. 2009.
    [4] T.-H. S. Li, S.-H. Tsai, and M.-Y. Hsiao, “Robust H-infinity fuzzy control for a class of time-delay fuzzy bilinear systems with an additive disturbance,” International Journal of Nonlinear Sciences and Numerical Simulation, vol. 10, no. 3, pp. 315-322, Mar. 2009.
    [5] C. -H. Chou, “Genetic algorithm-based optimal fuzzy controller design in the linguistic space,” IEEE Transactions on Fuzzy Systems, vol. 14, no. 3, pp. 372-385, Jun. 2006.
    [6] C. -F. Juang and P. -H. Chang, “Designing fuzzy-rule-based systems using continuous ant-colony optimization,” IEEE Transactions on Fuzzy Systems, vol. 18, no. 1, pp. 138-149, Feb. 2010.
    [7] J. -T. Tsai, T. -K. Liu, and J. -H. Chou, “Hybrid Taguchi-genetic algorithm for global numerical optimization,” IEEE Transactions on Evolutionary Computation, vol. 8, no. 4, pp. 365-377, Aug. 2004.
    [8] S. Mirjalili, S. M. Mirjalili, and A. Lewis, “Grey wolf optimizer,” Advances in Engineering Software, vol. 69, no. 13, pp. 46-61, Mar.2014.
    [9] E. Rashedi, H. Nezamabadi-pour, and S. Saryazdi, “GSA: a gravitational search algorithm,” Information Sciences, vol. 179, no. 13, pp. 2232-2248, Mar.2009.
    [10] A. H. Gandomi, X. -S. Yang, S. Talatahari, and A. H. Alavid, “Firefly algorithm with chaos,” Commun Nonlinear Sci Numer Simulat, vol. 18, pp. 89-98, Jun.2013.
    [11] E. Cuevas, M. Cienfuegos, D. Zaldívar, and M. Pérez-Cisneros, “A swarm optimization algorithm inspired in the behavior of the social-spider,” Expert Systems with Applications, vol. 40, no. 16, pp. 6374-6384, Nov. 2013.
    [12] R. Rajabioun, “Cuckoo optimization algorithm,” Applied Soft Computing, vol. 11, no. 8, pp. 5508-5518, Dec. 2011.
    [13] O. B. Haddad, “Honey-bees mating optimization (HBMO) algorithm: a new heuristic approach for water resources optimization,” Water Resources Management, vol. 20, no. 5, pp. 661-680, Oct. 2006.
    [14] A. H. Gandomi and A. H. Alavi, “Krill herd: A new bio-inspired optimization algorithm,” Communications in Nonlinear Science and Numerical Simulation, vol. 17, no. 12, pp. 4831-4835, Dec. 2012.
    [15] S. Das, A. Biswas, S. Dasgupta, and A. Abraham, “Bacterial foraging optimization algorithm: theoretical foundations, analysis, and applications,” Foundations of Computational Intelligence, vol. 3, pp. 23-55, 2009.
    [16] M. Neshat, G. Sepidnam, M. Sargolzaei, and A. N. Toosi, “Artificial fish swarm algorithm: a survey of the state-of-the-art, hybridization, combinatorial and indicative applications,” Communications in Nonlinear Science and Numerical Simulation, vol. 42, no. 4, pp. 965-997, Dec. 2014.
    [17] D. Karaboga and B. Basturk, “A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm,” Journal of Global Optimization, vol. 39, no. 3, pp. 459-471, Apr. 2007.
    [18] D. Karaboga, “An idea based on honey bee swarm for numerical optimization,” Tech.Rep.TR06, Computer Engineering Department, Engineering Faculty, Erciyes University, Oct. 2005.
    [19] R. S. Sutton and A. G. Barto, “Reinforcement learning: an introduction,” The MIT Press Cambridge, Massachusetts London, England, Oct. 2012.
    [20] D. Karaboga and B. Basturk, “On the performance of artificial bee colony (ABC) algorithm,” Applied Soft Computing, vol. 8, pp. 687-697, May. 2007.
    [21] D. Karaboga and B. Gorkemli, “A quick artificial bee colony (qABC) algorithm and its performance on optimization problems,” Applied Soft Computing, vol. 23, pp. 227-238, Jun. 2014.
    [22] A. Banharnsakun, T. Achalakul, and B. Sirinaovakul, “The best-so-far selection in artificial bee colony algorithm,” Applied Soft Computing, vol. 11, pp. 2888-2901, Dec. 2010.
    [23] W. -F. Gao, L. -L. Huang, S. -Y. Liu, and C. Dai, “Artificial bee colony algorithm based on information learning,” IEEE Transactions on Cybernetics, vol. 45, no. 12, pp. 2827-2839, Dec. 2015.
    [24] H. Chen, L. Ma, M. He, X. Wang, X. Liang, L. Sun, and M. Huang, “Artificial bee colony optimizer based on bee life-cycle for stationary and dynamic optimization,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 47, no. 2, pp. 327-346, Feb. 2017.
    [25] X. Yan, Y. Zhu, and W. Zou, “A hybrid artificial bee colony algorithm for numerical function optimization,” 2011 11th International Conference on Hybrid Intelligent Systems, pp. 27-132, Dec. 2011.
    [26] S. Kumar, V. K. Sharma, and R. Kumari, “A novel hybrid crossover based artificial bee colony algorithm for optimization problem,” International Journal of Computer Applications, vol. 82, no. 8, pp. 327-346, Nov. 2013.
    [27] T. -H. S. Li, P.-H. Kuo, Y. -F. Ho, M. -C. Kao, and L. -H. Tai, “A biped gait learning algorithm for humanoid robots based on environmental impact assessed artificial bee colony,” IEEE Access, vol. 3, pp. 13-26, Jan. 2015.
    [28] Y. -H. Wang, T. -H. S. Li, and C. -J. Lin, “Backward Q-learning: the combination of sarsa algorithm and Q-learning,” Engineering Applications of Artificial Intelligence, vol. 26, pp. 2184-2193, Jun. 2013.
    [29] G. Dhiman, and V. Kumar, “Spotted hyena optimizer: a novel bio-inspired based metaheuristic technique for engineering applications,” Advances in Engineering Software, vol. 114, pp. 48-70, Dec. 2017.
    [30] T. -H. S Li and M. -Y. Shieh, "Switching-type fuzzy sliding mode control of a cart-pole system," Mechatronics, vol. 10, no. 1-2, pp. 91-109, Feb. 2000.
    [31] S. -G. Kong and B. Kosko, “Comparison of fuzzy and neural truck backer upper control systems,” International Joint Conference on, pp. 349-358, Jun. 1990.
    [32] L. -X. Wang and J. M. Mendel, “Generating fuzzy rules by learning from examples,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no. 6, pp. 1414-1427, Nov./Dec. 1992.
    [33] C. -F. Juang and C. -M. Lu, “Ant colony optimization incorporated with fuzzy q-learning for reinforcement fuzzy control,” IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, vol. 39, no. 3, pp. 597-608, Feb. 2009.
    [34] L. Jouffe, “Fuzzy inference system learning by reinforcement methods,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 28, no. 3, pp. 338-355, Aug. 1998.
    [35] L. Jouffe, “Fuzzy sarsa learning and the proof of existence of its Stationary points,” Asian Journal of Control, vol. 10, no. 5, pp. 535-549, Sep. 1998.
    [36] [Online]. https://abc.erciyes.edu.tr/

    下載圖示 校內：2023-06-01公開
    校外：2023-06-01公開
    QR CODE