簡易檢索 / 詳目顯示

回結果列表
研究生： 許顥屏
Hsu, Hao-Ping
論文名稱： 整合SAC演算法及可改變質心高度LIPM於雙足機器人上階梯步態之設計與實踐
Design and Implementation of Stair-Climbing Gait Pattern Generator for Biped Robot by Integrating SAC with Altered Height CoM LIPM
指導教授： 李祖聖
Li, Tzuu-Hseng S
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 71
中文關鍵詞： SAC深度加強式學習 、線性倒單擺模型 、質心高度 、雙足機器人
外文關鍵詞： SAC Deep Reinforcement Learning, Linear Inverted Pendulum Model, CoM Height, Biped Robot
相關次數： 點閱：391  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 本論文探討SAC深度加強式學習網路結合可改變質心高度之線性倒單擺回授控制器於雙足機器人上階梯之應用，所提控制器能即時修正上階梯步態，並且實踐於雙足機器人進行測試及驗證。本論文之回授控制器會根據當前腳底板壓力感測器所計算出之壓力中心以及利用視覺所觀測到階梯高度作為輸入資料，並且利用慣性測量單位所測量之X-Y方向之加速度誤差平方和當作獎懲依據以修正機器人於上階梯時之晃動。接著利用輸入資料訓練SAC深度神經網路進而預測線性倒單擺模型之參數，SAC深度神經網路採用最大化熵值，能夠適當地表現訓練資料間的非線性關係，避免過擬合現象，並預測出線性倒單擺之十二個參數，調整機器人於上階梯之質心高度、擺幅幅度、跨腳高度和步距大小等運動特徵。本論文利用PyBullet模擬軟體進行模擬以及驗證，接著轉移至實體雙足機器人進行實踐。本論文使用了基於Linux之控制系統，確保能夠在機器人上達到即時運算，最後在模擬軟體和實體雙足機器人實驗結果顯示，所提出之方法能夠讓雙足機器人在增加動作穩定性的狀況下完成不同高度之上階梯之任務，並具備動態回授之效果。

    This thesis focuses on feedback controller that integrates Soft Actor-Critic (SAC) deep reinforcement learning and altered height Center of Mass (CoM) Linear Inverted Pendulum Model (LIPM) on stair-climbing gait pattern generator for biped robot. This controller can modify stair-climbing gait pattern in real time and this method is identified by experiments on biped robot in real world. The feedback controller in this thesis will use sensor information like height of the stairs from depth camera and Center of Pressure (CoP), which is measured from pressure sensors placed on the sole of the biped robot as input data. Also, the reciprocal of square error of acceleration in x and y direction will be taken into consideration in the reward function to avoid shaking during stair climbing. The input data will be used to train SAC and then the network will predict LIPM parameters for climbing stair. SAC adopts maximum entropy that can deal with the training data properly and avoid over-fitting. The predicted twelve parameters for LIPM can adjust biped robot’s CoM height, swing of legs, step height, and moving distance. This thesis adopts PyBullet simulator to simulate and identify at first, and then the result is implemented on the biped robot in real world. This thesis uses control system based on Linux operating system to reach real-time calculation. The results in both simulator and real world demonstrate that the method proposes in this thesis can make the biped robot achieve stair climbing stably and reach dynamic feedback.

    Abstract I Acknowledge Content III List of Figures VII List of Tables IX Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Related Works 3 1.3 Thesis Organization 5 Chapter 2 Robot Hardware and Control System 7 2.1 Introduction 7 2.2 Hardware Specification and Sensors 8 2.2.1 The Mechanism and Configuration of Robot 8 2.2.2 Motor 12 2.2.3 IMU 14 2.2.4 Pressure Sensor 15 2.2.5 Camera 16 2.2.6 Power Supply 17 2.3 Control System 18 2.3.1 Computer 19 2.3.2 U2D2 21 2.3.3 CAN 21 2.4 Summary 22 Chapter 3 Altered Height CoM LIPM Gait Pattern Generator for Biped Robot Stair Climbing 23 3.1 Introduction 23 3.2 Linear Inverted Pendulum Model 24 3.2.1 Conventional LIPM 24 3.2.2 Double-Link LIPM 28 3.2.3 Zero Moment Point 31 3.3 Inverse Kinematics of the Biped Robot 33 3.3.1 Forward Kinematics 34 3.3.2 Inverse Kinematics 37 3.4 Altered Height CoM 39 3.5 Stair-Climbing Pattern 41 3.6 Summary 42 Chapter 4 Soft Actor-Critic Based Feedback System for Stair Climbing 43 4.1 Introduction 43 4.2 Soft Actor-Critic Feedback System 45 4.2.1 Traditional Q-learning 45 4.2.2 Soft Q-learning 46 4.2.3 Soft Actor-Critic 49 4.3 Parameterized Stair-Climbing Gait Pattern Generator 51 4.4 Implementation of Soft Actor-Critic Based Feedback System for Stair Climbing 52 4.5 Summary 53 Chapter 5 Simulation and Experimental Results 54 5.1 Introduction 54 5.2 Collecting and Analyzing Sensor Data 55 5.3 Reward Defining for Soft Actor-Critic Based Feedback System 57 5.4 Training Result 58 5.5 Simulations and Experiments 58 5.5.1 Experiment 1：Stair Climbing under Various Heights 59 5.5.2 Experiment 2：Stair Climbing under Different Mass of Trunks 62 5.6 Summary 64 Chapter 6 Conclusions and Future Work 65 6.1 Conclusions 65 6.2 Future Work 66 References 68

    [1]“Atlas Boston Dynamics.” https://www.bostondynamics.com/atlas (accessed Jun. 16, 2021).
    [2]“Cassie - ROBOTS: Your guide to the world of robotics. https://robots.ieee.org/robots/cassie/ (accessed Jun. 16, 2021).
    [3]“HRP-4 - ROBOTS: Your guide to the world of robotics.” https://robots.ieee.org/robots/hrp4/ (accessed Jun. 16, 2021).
    [4]“ASIMO by Honda | The World’s most advanced humanoid robot.” https://asimo.honda.com/ (accessed Jun. 16, 2021).
    [5] C. Fu and K. Chen, “Gait synthesis and sensory control of stair climbing for a humanoid robot,” IEEE Trans. Ind. Electron., vol. 55, no. 5, pp. 2111–2120, May 2008, doi: 10.1109/TIE.2008.921205.
    [6] C. L. Shih, “Ascending and descending stairs for a biped robot,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans., vol. 29, no. 3, pp. 255–268, 1999, doi: 10.1109/3468.759271.
    [7] T.-H. S.Li, Y. T. Su, C. H. Kuo, C. Y. Chen, C. L. Hsu, and M. F. Lu, “Stair-climbing control of humanoid robot using force and accelerometer sensors,” in Proceedings of the SICE Annual Conference, 2007, pp. 2115–2120, doi: 10.1109/SICE.2007.4421336.
    [8] S. Osswald, A. Gorog, A. Hornung, and M. Bennewitz, “Autonomous climbing of spiral staircases with humanoids,” in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Dec. 2011, pp. 4844–4849, doi: 10.1109/iros.2011.6094533.
    [9] Z. Yu, X. Chen, Q. Huang, “Gait planning of omnidirectional walk on inclined ground for biped robots,” IEEE Trans. Syst. Man, Cybern. Syst., vol. 46, no. 7, pp. 888–897, July 2016, doi: 10.1109/TSMC.2015.2487240.
    [10] B. J. Lee, D. Stonier, Y. D. Kim, J. K. Yoo, and J. H. Kim, “Modifiable walking pattern of a humanoid robot by using allowable ZMP variation,” IEEE Trans. Robot., vol. 24, no. 4, pp. 917–925, Aug. 2008, doi: 10.1109/TRO.2008.926859.
    [11] T. Sato, S. Sakaino, E. Ohashi, and K. Ohnishi, “Walking trajectory planning on stairs using virtual slope for biped robots,” IEEE Trans. Ind. Electron., vol. 58, no. 4, pp. 1385–1396, Apr. 2011, doi: 10.1109/TIE.2010.2050753.
    [12] P. Sardain and G. Bessonnet, “Zero moment point - measurements from a human walker wearing robot feet as shoes,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans., vol. 34, no. 5, pp. 638–648, Sep.2004, doi: 10.1109/TSMCA.2004.832833.
    [13] T.-H. S. Li, Y. T. Su, S. H. Liu, J. J. Hu, and C. C. Chen, “Dynamic balance control for biped robot walking using sensor fusion, Kalman filter, and fuzzy logic,” IEEE Trans. Ind. Electron., vol. 59, no. 11, pp. 4394–4408, Nov. 2012, doi: 10.1109/TIE.2011.2175671.
    [14] K. Hu, C. Ott, and D. Lee, “Learning and generalization of compensative zero-moment point trajectory for biped walking,” IEEE Trans. Robot., vol. 32, no. 3, pp. 717–725, Jun. 2016, doi: 10.1109/TRO.2016.2553677.
    [15] Y. D. Hong and B. Lee, “Real-time feasible footstep planning for bipedal robots in three-dimensional environments using particle swarm optimization,” IEEE/ASME Trans. Mechatronics, vol. 25, no. 1, pp. 429–437, Feb. 2020, doi: 10.1109/TMECH.2019.2955701.
    [16] S. Caron, A. Kheddar, and O. Tempier, “Stair climbing stabilization of the HRP-4 humanoid robot using whole-body admittance control,” in Proceedings of the IEEE Inter. Conf. on Robotics and Automation, May 2019, vol. 2019-May, pp. 277–283, doi: 10.1109/ICRA.2019.8794348.
    [17] T.-H. S. Li, Y. F. Ho, P. H. Kuo, Y. T. Ye, and L. F. Wu, “Natural walking reference generation based on double-link LIPM gait planning algorithm,” IEEE Access, vol. 5, pp. 2459–2469, 2017, doi: 10.1109/ACCESS.2017.2669209.
    [18] Y. D. Hong, B. J. Lee, and J. H. Kim, “Command state-based modifiable walking pattern generation on an inclined plane in pitch and roll directions for humanoid robots,” IEEE/ASME Trans. Mechatronics, vol. 16, no. 4, pp. 783–789, Aug. 2011, doi: 10.1109/TMECH.2010.2089530.
    [19] T. Komura, A. Nagano, H. Leung, and Y. Shinagawa, “Simulating pathological gait using the enhanced linear inverted pendulum model,” IEEE Trans. Biomed. Eng., vol. 52, no. 9, pp. 1502–1513, Sep. 2005, doi: 10.1109/TBME.2005.851530.
    [20] K. S. Hwang, J. L. Lin, and K. H. Yeh, “Learning to adjust and refine gait patterns for a biped robot,” IEEE Trans. Syst. Man, Cybern. Syst., vol. 45, no. 12, pp. 1481–1490, Dec.2015, doi: 10.1109/TSMC.2015.2418321.
    [21] T.-H. S. Li, P.-H. Kuo, L.-H. Chen, C.-C. Hung, P.-C. Luan, H.-P. Hsu, C.-H. Chang, Y.-T. Hsieh, and W.-H. Lin, “Fuzzy double deep Q-network-based gait pattern controller for humanoid robots,” IEEE Trans. Fuzzy Syst., pp. 1–1, Oct. 2020, doi: 10.1109/tfuzz.2020.3033141.
    [22] H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-learning,” Sep.2015, Accessed: Jun.16, 2021. [Online]. Available: http://arxiv.org/abs/1509.06461.
    [23] J. Schulman, F. Wolski, P. Dhariwal, A.R adford, and O. Klimov, “Proximal policy optimization algorithms,” Jul.2017, Accessed: Jun.16, 2021. [Online]. Available: http://arxiv.org/abs/1707.06347.
    [24] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor,” Jan. 2018, Accessed: Jun.14, 2021. [Online]. Available: http://arxiv.org/abs/1801.01290.
    [25] “ROBOTIS.” https://www.robotis.com/#firstPage (accessed Jun. 19, 2021).
    [26] Intel, “RealSense depth camera D435i.” .
    [27] K. Erbatur and O. Kurt, “Natural ZMP trajectories for biped robot reference generation,” IEEE Trans. Ind. Electron., vol. 56, no. 3, pp. 835–845, 2009, doi: 10.1109/TIE.2008.2005150.
    [28] M. H. P. Dekker, “Zero-moment point method for stable biped walking,” Internsh. Rep. DCT Nr, no. July, p. 62, 2009, doi: 10.1109/IROS.2011.6048045.
    [29] Q. Huang, S. Sugano, and K. Tanie, “Stability compensation of a mobile manipulator by manipulator motion: feasibility and planning,” in Proceedings of the 1997 IEEE/RSJ Int. Conf. Intelligent Robot and Syst. Innovative Robotics for Real-World Applications. IROS ’97, 2002, pp. 1285–1292, doi: 10.1109/iros.1997.656417.
    [30] J. P. Ferreira, M. M. Crisostomo, A. P. Coimbra, and B. Ribeiro, “Control of a biped robot with support vector regression in sagittal plane,” IEEE Trans. Instrum. Meas., vol. 58, no. 9, pp. 3167–3176, 2009, doi: 10.1109/TIM.2009.2017148.
    [31] D. J. Braun, J. E. Mitchell, and M. Goldfarb, “Actuated dynamic walking in a seven-link biped robot,” IEEE/ASME Trans. Mechatronics, vol. 17, no. 1, pp. 147–156, 2012, doi: 10.1109/TMECH.2010.2090891.
    [32] P.S ardain and G. Bessonnet, “Forces acting on a biped robot. Center of pressure - Zero moment point,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans., vol. 34, no. 5, pp. 630–637, 2004, doi: 10.1109/TSMCA.2004.832811.
    [33] C. Liu, D. Wang, and Q. Chen, “Central pattern generator inspired control for adaptive walking of biped robots,” IEEE Trans. Syst. Man, Cybern. Part ASystems Humans, vol. 43, no. 5, pp. 1206–1215, 2013, doi: 10.1109/TSMC.2012.2235426.
    [34] P. H. Kuo, Y. F. Ho, K. F. Lee, L. H. Tai, and T. H. S. Li, “Development of humanoid robot simulator for gait learning by using particle swarm optimization,” in Proceedings of the 2013 IEEE Int. Conf. Syst. Man, Cybern. SMC 2013, pp. 2683–2688, 2013, doi: 10.1109/SMC.2013.457.
    [35] M. Azarkaman, M. Aghaabbasloo, and M. E. Salehi, “Evaluating GA and -PSO evolutionary algorithms for humanoid walk pattern planning,” in Proceedings of the 2014 ICEE Int. Conf. Electr, ICEE 2014, pp. 868–873, doi: 10.1109/IranianCEE.2014.6999658.
    [36] Y. D. Hong, C. S. Park, and J. H.K im, “Stable bipedal walking with a vertical center-of-mass motion by an evolutionary optimized central pattern generator,” IEEE Trans. Ind. Electron., vol. 61, no. 5, pp. 2346–2355, 2014, doi: 10.1109/TIE.2013.2267691.
    [37] T. P. Lillicrap, J. J. Hunt, A. Pritzel, “Continuous control with deep reinforcement learning,” Sep.2015, Accessed: Jun.16, 2021. [Online]. Available: http://arxiv.org/abs/1509.02971.
    [38] C. J. C. H. Watkins and P. Dayan, “Q-learning,” Mach. Learn., vol. 8, no. 3–4, pp. 279–292, 1992, doi: 10.1007/BF00992698.
    [39] T. Haarnoja, H. Tang, P. Abbeel, and S. Levine, “Reinforcement learning with deep energy-based policies,” Feb.2017, Accessed: Jun. 16, 2021. [Online]. Available: http://arxiv.org/abs/1702.08165.
    [40] S. H. Hyon, “Compliant terrain adaptation for biped humanoids without measuring ground surface and contact forces,” IEEE Trans. Robot., vol. 25, no. 1, pp. 171–178, 2009, doi: 10.1109/TRO.2008.2006870.

    下載圖示
    2026-08-10公開
    QR CODE