簡易檢索 / 詳目顯示

回結果列表
研究生： 張家揚
Chong, Kiah-Yang
論文名稱： 人形機器人模糊策略梯度步態學習法之設計與實現
Design and Implementation of Fuzzy Policy Gradient Gait Learning Method for Humanoid Robot
指導教授： 李祖聖
Li, Tzuu-Hseng S.
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2010
畢業學年度： 98
語文別： 英文
論文頁數： 78
外文關鍵詞： Humanoid, Gait generation, Machine learning
相關次數： 點閱：91  下載：4 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 本論文以模糊策略梯度步態學習法去設計並實現小型人形機器人的步態訓練。除了闡述此命名為aiRobots-V的人形機器人之機構設計與系統架構之外，也改善過去人形機器人步態的缺點並將其參數化。此機器人的步態也加入了手臂擺動的動作，以降低在步行過程中機器人軀幹的晃動程度。模糊策略梯度步態學習法以策略梯度增強式學習法之訓練行走步態參數的能力為基礎，並以模糊邏輯的分析與決策能力去增強步態訓練的速度與效能。人形機器人利用模糊策略梯度步態學習法，並以機器人在固定步伐數內所步行的距離為學習法的獎懲回授，去自動尋找可能的參數以得到更快的步伐。機器人的晃動角度也被選擇為學習手臂擺動動作的獎懲回授。經實驗證明，模糊策略梯度步態學習法讓機器人在約60分鐘內從每秒9.26公厘的行走速度學習至每秒162.27公厘。實驗也證明此學習法比基本策略梯度增強式學習法的學習效能增加了約13%。手臂擺動動作也確實降低了步行過程中機器人軀幹的晃動程度。此機器人也成功地應用在RoboCup 2010的邊線投球挑戰賽中。

    The design and implementation of Fuzzy Policy Gradient Learning (FPGL) method for small-sized humanoid robot is proposed in this thesis. This thesis not only introduces the mechanism structure of the humanoid robot and the hardware system adapted on the robot, which is named as aiRobots-V, but also improves and parameterizes the gait pattern of the robot. The movement of arms is added to the gait pattern to reduce the tilt of trunk while walking. FPGL method is an integrated machine learning method based on Policy Gradient Reinforcement Learning (PGRL) method and fuzzy logic concept in order to improve the efficiency and speed of gait learning computation. The humanoid robot is trained with FPGL method which is using the walking distance in constant walking cycles as the reward to learn faster and stable gait automatically. The tilt degree of trunk is chosen as the reward to learn the movement of arms in the walking cycle. The result of the experiment shows that FPGL method could train the gait pattern from 9.26 mm/s walking speed to 162.27 mm/s in about an hour. The training data of experiments also shows that this method could improve the efficiency of basic PGRL method up to 13%. The effect of arm movement to reduce the tilt degree of trunk is also proved by the experimental results. This robot is also applied to participate in the throw-in technical challenge of RoboCup 2010.

    Abstract I Acknowledgment III Contents IV List of Figures VII List of Tables X Chapter 1. Introduction 1 1.1 Motivation 1 1.2 Thesis Organization 3 Chapter 2. Mechanism and Hardware of the Humanoid 5 2.1 Introduction 5 2.2 Design of Mechanism 6 2.3 Hardware of aiRobots-V 11 2.3.1 Actuators 12 2.3.2 Motion Controller 13 2.3.3 Zigbee Module 16 2.3.4 Accelerometer 18 2.3.5 Li-po Batteries 18 2.4 Summary 20 Chapter 3. Concept and Design of Gait Pattern 21 3.1 Introduction 21 3.2 Concept of Gait Pattern 22 3.3 Gait Pattern for Previous aiRobots 24 3.4 Design and Parameters of Gait Pattern for aiRobots-V 25 3.4.1 Design of Gait Pattern for aiRobots-V 25 3.4.2 Parameters of Gait Pattern for aiRobots-V 28 3.5 Human-machine Interface for aiRobots-V 31 3.6 Summary 35 Chapter 4. Fuzzy Policy Gradient Gait Learning 36 4.1 Introduction 36 4.2 Concept of Policy Gradient Reinforcement Learning Method 37 4.3 Concept of Fuzzy Logic Control 42 4.4 Fuzzy Policy Gradient Gait Learning 44 4.5 Configuration of Gait Learning for aiRobots-V 46 4.5.1 Configuration of FPGL Part 47 4.5.2 Configuration of PGPR Part 52 4.6 Gait Learning Page of Human-machine Interface 54 4.7 Summary 55 Chapter 5. Experimental Results 56 5.1 Introduction 56 5.2 Experimental Results 57 5.2.1 Gait Learning with PGPR and FPGL Methods 57 5.2.2 Tilt of Trunk with and without Arm Movement 63 5.2.3 Throw-in Technical Challenge 65 Chapter 6. Conclusions and Future Works 71 6.1 Conclusions 71 6.2 Future Works 73 References 74 Biography 77

    References
    [1] K. Harada, S. Kajita, K. Kaneko, and H. Hirukawa, “Dynamics and balance of a humanoid robot during manipulation tasks,” IEEE Trans. Robotics, vol. 22, no. 3, pp. 568-575, Jun. 2006.
    [2] S. Kajita, T. Nagasaki, K. Kaneko and H. Hirukawa, “ZMP-based biped running control,” IEEE Trans. Robotics, vol. 14, no. 2, pp. 63-72, Jun. 2007.
    [3] E. Ohashi, T. Aiko, T. Tsuji, H. Nishi, and K. Ohnishi, “Collision avoidance method of humanoid robot with arm force,” IEEE Trans. Industrial Electronics, vol. 54, no. 3, pp. 1632-1641, Jun. 2007.
    [4] C. Fu and K. Chen, “Gait synthesis and sensory control of stair climbing for a humanoid robot,” IEEE Trans. Industrial Electronics, vol. 55, no. 5, pp. 2111-2120, May. 2008.
    [5] K. Erbatur and O. Kurt, “Natural ZMP trajectories for biped robot reference generation,” IEEE Trans. Industrial Electronics, vol. 56, no. 3, pp. 835-845, Mar. 2009.
    [6] P. Sardain and G. Bessonnet, “Zero moment point-measurements from a human walker wearing robot feet as shoes,” IEEE Trans. Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 34, no. 5, pp. 638-648, Sep. 2004.
    [7] Q. Huang, K. Kaneko, K. Yokoi, S. Kajita, T. Kotoku, N. Koyachi, H. Arai, N. Imamura, K. Komoriya and K. Tanie, “Balance control of a biped robot combining off-line pattern with real-time modification,” in Proc. IEEE Int. Conf. Robotics and Automation, 2000, pp. 3346-3352.
    [8] P. Sardain and G. Bessonnet, “Forces acting on a biped robot. Center of pressure-zero moment point,” IEEE Trans. Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 34, pp. 630–372, Sep. 2004.
    [9] F. Asano, M. Yamakita, N. Kamamichi and Z. W. Luo, “A novel gait generation for biped walking robots based on mechanical energy constraint,” IEEE Trans. Robotics and Automation, vol. 20, no. 3, pp. 565-573, Jun. 2004.
    [10] Q. Huang and Y. Nakamura, “Sensory reflex control for humanoid walking,” IEEE Trans. Robotics, vol. 21, no. 5, pp. 977-984, Oct. 2005.
    [11] K. Harada, S. Kajita, F. Kanehiro, K. Fujiwara, K. Kanedo, K. Yokoi and H. Hirukawa, “Real-time planning of humanoid robot’s gait for force-controlled manipulation,” IEEE/ASME Trans. Mechatronics, vol. 12, no. 1, pp. 53-62, Feb. 2007.
    [12] P. Sardain, M. Rostami and G. Bessonnet, “An anthropomorphic biped robot: dynamic concepts and technological design,” IEEE Trans. Systems, Man and Cybernetics, Part A: Systems and Humans, vol. 28, no. 6, pp.823-838, Nov. 1998.
    [13] Y. D. Kim, B. J. Lee, J. H. Ryu and J. H. Kim, “Landing force control for humanoid robot by time-domain passivity approach,” IEEE Trans. Robotics, vol. 23, no. 6, pp. 1294-1301, Dec. 2007.
    [14] H. Minakata, H. Seki and S. Tadakuma, “A study of energy-saving shoes for robot considering lateral plane motion,” IEEE Trans. Industrial Electronics, vol. 55, no. 3, pp. 1271-1276, Mar. 2008.
    [15] J. Morimoto, G. Endo, J. Nakanishi and G. Cheng, “A biologically inspired biped locomotion strategy for humanoid robots: Modulation of sinusoidal patterns by a coupled oscillator model,” IEEE Trans. Robotics, vol. 24, no. 1, pp. 185-191, Feb. 2008.
    [16] T. Zielinska, “Biological inspiration used for robots motion synthesis,” Journal of Physiology - Paris, vol. 103, pp. 133-140, Jul. 2009.
    [17] N. Kohl and P. Stone, “Policy gradient reinforcement learning for fast quadrupedal locomotion,” in Proc. IEEE Int. Conf. Robotics and Automation, 2004, vol. 3, pp. 2619-2624.
    [18] A. Cherubini, F. Giannone, L. Iocchi and P.F. Palamara, “An extended policy gradient algorithm for robot task learning,” in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and System, 2007, pp. 4121-4126.
    [19] A. Cherubini, F. Giannone, L. Iocchi, M. Lombardo and G. Oriolo, “Policy gradient learning for a humanoid soccer robot,” Robotics and Autonomous Systems, vol. 57, pp. 808-818, Apr. 2009.
    [20] FIRA, http://www.fira.net/
    [21] RoboCup, http://www.robocup.org/
    [22] ROBOTIS, http://www.robotis.com/
    [23] Arduino, http://www.arduino.cc/
    [24] ZIG-100, http://www.nodna.com/download/ROBOTIS/ZIG-100(english).pdf
    [25] Hitachi H48C 3-axis Accelerometer, http://www.parallax.com/dl/docs/prod/acc/HitachiH48C3AxisAccelerometer.pdf
    [26] S.-H. Liu, Design and Implementation of a Gait Pattern Generator based on Genetic Algorithms and Fuzzy Control for Small-sized Humanoid Robot by using SOPC, Master Thesis, National Cheng Kung University, Jul. 2008.
    [27] S.-W. Lai, Design and Implementation of Reinforce Learning Based Fuzzy Gait Controller for Humanoid Robot, Master Thesis, National Cheng Kung University, Jul. 2009.
    [28] L.A. Zadeh, “Fuzzy sets,” Information and Control, vol. 8, no. 3, pp. 338-353, 1965.
    [29] L.A. Zadeh, “Fuzzy algorithms,” Information and Control, vol. 12, no. 2, pp. 94–102,1968.
    [30] C. C. Lee, “Fuzzy logic in control system: Fuzzy logic controller-part I,” IEEE Trans. Systems, Man and Cybernetics, vol. 20, no. 2, pp. 404-418, 1990.
    [31] C. C. Lee, “Fuzzy logic in control system: Fuzzy logic controller-part II,” IEEE Trans. Systems, Man and Cybernetics, vol. 20, no. 2, pp. 419-435, 1990.
    [32] T. Takagi and M. Sugeno, “Fuzzy identification of systems and its application to modeling and control,” IEEE Trans. Systems, Man and Cybernetics, vol. 15, no. 1, pp. 116-132, 1985.
    [33] R.E. Kalman, “A new approach to linear filtering and prediction problems,” ASME Jour. Basic Engineering, vol. 82, no. 1, pp. 35-45, 1960.
    [34] R.E. Kalman and R.S. Bucy, “New results in linear filtering and prediction theory,” ASME Jour. Basic Engineering, Series D, Vol. 83, pp. 95-107, 1961.

    下載圖示 校內：2012-08-16公開
    校外：2015-08-16公開
    QR CODE