面向运输代价最小化的鲁棒多步态四足机器人高效学习

王志成; 赵欣; Meng Yee (Michael) CHUAH; Zhibin LI; 吴俊; 朱秋国

doi:10.1631/FITEE.2401070

Your Location：

Home >

Browse articles >

面向运输代价最小化的鲁棒多步态四足机器人高效学习

常规文章 | Updated：2025-11-05

- 面向运输代价最小化的鲁棒多步态四足机器人高效学习
- Efficient learning of robust multigait quadruped locomotion for minimizing the cost of transport
- 信息与电子工程前沿（英文） 2025年26卷第9期页码：1679-1691
- Affiliations：
  
  1.Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou 310027, China
  2.Institute for Infocomm Research, A*STAR, Singapore 138632, Singapore
  3.Chinese Scholartree Ridge State Key Laboratory, China North Vehicle Research Institute, Beijing 100072, China
  4.Department of Computer Science, University College London, London WC1E, UK
  5.State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou 310027, China
- Author bio：
  
  E-mail: willw317@zju.edu.cn
  ‡ Corresponding author
- Funds：
  
  "Leading Goose" R&D Program of Zhejiang Province(2023C01177);Research Project on the Motion Control of Bipedal Maneuver(8KD006(2024)-2);National Key R&D Program of China(2022YFB4701502);2035 Key Technological Innovation Program of Ningbo City(2024Z300)
- DOI：10.1631/FITEE.2401070
  中图分类号： TP242.6;TP18
- 收稿：2024-12-16，
  
  修回：2025-05-12，
  
  纸质出版：2025-09
- Accepted：
Scan QR Code
王志成, 赵欣, Meng Yee (Michael) CHUAH, 等. 面向运输代价最小化的鲁棒多步态四足机器人高效学习[J]. 信息与电子工程前沿（英文）, 2025,26(9):1679-1691.

Zhicheng WANG, Xin ZHAO, Meng Yee (Michael) CHUAH, et al. Efficient learning of robust multigait quadruped locomotion for minimizing the cost of transport[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(9): 1679-1691.
王志成, 赵欣, Meng Yee (Michael) CHUAH, 等. 面向运输代价最小化的鲁棒多步态四足机器人高效学习[J]. 信息与电子工程前沿（英文）, 2025,26(9):1679-1691. DOI： 10.1631/FITEE.2401070.

Zhicheng WANG, Xin ZHAO, Meng Yee (Michael) CHUAH, et al. Efficient learning of robust multigait quadruped locomotion for minimizing the cost of transport[J]. Frontiers of Information Technology & Electronic Engineering, 2025, 26(9): 1679-1691. DOI： 10.1631/FITEE.2401070.

摘要

四足机器人能够使用多种步态节律进行移动，每种步态在地形通过性和能量效率方面具有不同特点。通过在不同环境下主动切换调整步态，四足机器人可以实现节能且适应性强的运动策略。本文探讨不同步态参数下基于强化学习的四足机器人运动策略的能量效率和地形通过能力，提出一种训练—整合框架，将习得的单步态运动策略整合为一个高效的多步态运动策略。所得到的控制策略实现了低成本的步态切换和可控的步态。实验结果表明，该多技能策略在保持能量最优的同时，能够实现平滑安全的步态过渡。

Abstract

Quadruped robots are able to exhibit a range of gaits

each with its own traversability and energy efficiency characteristics. By actively coordinating between gaits in different scenarios

energy-efficient and adaptive locomotion can be achieved. This study investigates the performances of learned energy-efficient policies for quadrupedal gaits under different commands. We propose a training–synthesizing framework that integrates learned gait-conditioned locomotion policies into an efficient multiskill locomotion policy. The resulting control policy achieves low-cost smooth switching and controllable gaits. Our results of the learned multiskill policy demonstrate seamless gait transitions while maintaining energy optimality across all commands.

关键词

Keywords

references

Acero F , Yuan K , Li ZB , 2022 . Learning perceptual locomotion on uneven terrains using sparse visual observations . IEEE Robot Autom Lett , 7 ( 4 ): 8611 - 8618 . https://doi.org/10.1109/LRA.2022.3188108 https://doi.org/10.1109/LRA.2022.3188108

Agarwal A , Kumar A , Malik J , et al. , 2022 . Legged locomotion in challenging terrains using egocentric vision . Proc 6 th Annual Conf on Robot Learning , p. 403 - 415 .

Chen D , Zhou B , Koltun V , et al. , 2019 . Learning by cheating . Proc 3 rd Annual Conf on Robot Learning , p. 66 - 75 .

di Carlo J , Wensing PM , Katz B , et al. , 2018 . Dynamic locomotion in the MIT Cheetah 3 through convex model-predictive control . Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems , p. 1 - 9 . https://doi.org/10.1109/IROS.2018.8594448 https://doi.org/10.1109/IROS.2018.8594448

Fu ZP , Kumar A , Malik J , et al. , 2021 . Minimizing energy consumption leads to the emergence of gaits in legged robots . Proc 5 th Annual Conf on Robot Learning , p. 928 - 937 .

Haynes GC , Rizzi AA , 2006 . Gaits and gait transitions for legged robots . Proc IEEE Int Conf on Robotics and Automation , p. 1117 - 1122 . https://doi.org/10.1109/ROBOT.2006.1641859 https://doi.org/10.1109/ROBOT.2006.1641859

Hildebrand M , 1965 . Symmetrical gaits of horses . Science , 150 ( 3697 ): 701 - 708 . https://doi.org/10.1126/science.150.3697.701 https://doi.org/10.1126/science.150.3697.701

Hinton G , Vinyals O , Dean J , 2015 . Distilling the knowledge in a neural network . https://arxiv.org/abs/1503.02531 https://arxiv.org/abs/1503.02531

Hoyt DF , Taylor CR , 1981 . Gait and the energetics of locomotion in horses . Nature , 292 ( 5820 ): 239 - 240 . https://doi.org/10.1038/292239a0 https://doi.org/10.1038/292239a0

Hsiao-Wecksler ET , Polk JD , Rosengren KS , et al. , 2010 . A review of new analytic techniques for quantifying symmetry in locomotion . Symmetry , 2 ( 2 ): 1135 - 1155 . https://doi.org/10.3390/sym2021135 https://doi.org/10.3390/sym2021135

Hwangbo J , Lee J , Hutter M , 2018 . Per-contact iteration method for solving contact dynamics . IEEE Robot Autom Lett , 3 ( 2 ): 895 - 902 . https://doi.org/10.1109/LRA.2018.2792536 https://doi.org/10.1109/LRA.2018.2792536

Ijspeert AJ , 2008 . Central pattern generators for locomotion control in animals and robots: a review . Neur Netw , 21 ( 4 ): 642 - 653 . https://doi.org/10.1016/j.neunet.2008.03.014 https://doi.org/10.1016/j.neunet.2008.03.014

Iscen A , Caluwaerts K , Tan J , et al. , 2018 . Policies modulating trajectory generators . Proc 2 nd Annual Conf on Robot Learning , p. 916 - 926 .

Jacobs RA , Jordan MI , Nowlan SJ , et al. , 1991 . Adaptive mixtures of local experts . Neur Comput , 3 ( 1 ): 79 - 87 . https://doi.org/10.1162/neco.1991.3.1.79 https://doi.org/10.1162/neco.1991.3.1.79

Jin YB , Liu XW , Shao YC , et al. , 2022 . High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning . Nat Mach Intell , 4 ( 12 ): 1198 - 1208 . https://doi.org/10.1038/s42256-022-00576-3 https://doi.org/10.1038/s42256-022-00576-3

Kumar A , Fu ZP , Pathak D , et al. , 2021 . RMA: rapid motor adaptation for legged robots . Proc 17 th Robotics: Science and Systems , p. 1 - 9 . https://doi.org/10.15607/rss.2021.xvii.011 https://doi.org/10.15607/rss.2021.xvii.011

Kumar A , Li Z , Zeng J , et al. , 2022 . Adapting rapid motor adaptation for bipedal robots . Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems , p. 1161 - 1168 . https://doi.org/10.1109/IROS47612.2022.9981091 https://doi.org/10.1109/IROS47612.2022.9981091

Lee J , Hwangbo J , Hutter M , 2019 . Robust recovery controller for a quadrupedal robot using deep reinforcement learning . https://arxiv.org/abs/1901.07517 https://arxiv.org/abs/1901.07517

Lee J , Hwangbo J , Wellhausen L , et al. , 2020 . Learning quadrupedal locomotion over challenging terrain . Sci Robot , 5 ( 47 ): eabc5986 . https://doi.org/10.1126/scirobotics.abc5986 https://doi.org/10.1126/scirobotics.abc5986

Loquercio A , Kumar A , Malik J , 2023 . Learning visual locomotion with cross-modal supervision . Proc Int Conf on Robotics and Automation , p. 7295 - 7302 . https://doi.org/10.1109/ICRA48891.2023.10160760 https://doi.org/10.1109/ICRA48891.2023.10160760

Luo YS , Soeseno JH , Chen TPC , et al. , 2020 . CARL: controllable agent with reinforcement learning for quadruped locomotion . ACM Trans Graph , 39 ( 4 ): 38 . https://doi.org/10.1145/3386569.3392433 https://doi.org/10.1145/3386569.3392433

Makoviychuk V , Wawrzyniak L , Guo YR , et al. , 2021 . Isaac Gym: high performance GPU based physics simulation for robot learning . Proc 35 th Conf on Neural Information Processing Systems , p. 1 - 12 .

Margolis GB , Agrawal P , 2022 . Walk these ways: tuning robot control for generalization with multiplicity of behavior . Proc 6 th Annual Conf on Robot Learning , p. 1 - 9 .

Margolis GB , Yang G , Paigwar K , et al. , 2022 . Rapid locomotion via reinforcement learning . Proc 18 th Robotics: Science and Systems , p. 1 - 9 .

Miki T , Lee J , Hwangbo J , et al. , 2022 . Learning robust perceptive locomotion for quadrupedal robots in the wild . Sci Robot , 7 ( 62 ): eabk2822 . https://doi.org/10.1126/scirobotics.abk2822 https://doi.org/10.1126/scirobotics.abk2822

Nahrendra IMA , Yu B , Myung H , 2023 . DreamWaq: learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning . Proc IEEE Int Conf on Robotics and Automation , p. 5078 - 5084 . https://doi.org/10.1109/ICRA48891.2023.10161144 https://doi.org/10.1109/ICRA48891.2023.10161144

Paszke A , Gross S , Massa F , et al. , 2019 . PyTorch: an imperative style, high-performance deep learning library . Proc 33 rd Int Conf on Neural Information Processing Systems , p. 8024 - 8035 .

Peng XB , Chang M , Zhang G , et al. , 2019 . MCP: learning composable hierarchical control with multiplicative compositional policies . Proc 33 rd Int Conf on Neural Information Processing Systems , Article 331 .

Peng XB , Coumans E , Zhang TN , et al. , 2020 . Learning agile robotic locomotion skills by imitating animals . Proc 16 th Robotics: Science and Systems , p. 1 - 9 .

Peng XB , Ma Z , Abbeel P , et al. , 2021 . AMP: adversarial motion priors for stylized physics-based character control . ACM Trans Graph , 40 ( 4 ): 144 . https://doi.org/10.1145/3450626.3459670 https://doi.org/10.1145/3450626.3459670

Peng XB , Guo YR , Halper L , et al. , 2022 . ASE: large-scale reusable adversarial skill embeddings for physically simulated characters . ACM Trans Graph , 41 ( 4 ): 94 . https://doi.org/10.1145/3528223.3530110 https://doi.org/10.1145/3528223.3530110

Schulman J , Wolski F , Dhariwal P , et al. , 2017 . Proximal policy optimization algorithms . https://arxiv.org/abs/1707.06347 https://arxiv.org/abs/1707.06347

Seok S , Wang A , Chuah MY , et al. , 2013 . Design principles for highly efficient quadrupeds and implementation on the MIT Cheetah robot . IEEE Int Conf on Robotics and Automation , p. 3307 - 3312 . https://doi.org/10.1109/ICRA.2013.6631038 https://doi.org/10.1109/ICRA.2013.6631038

Shao YC , Jin YB , Liu XW , et al. , 2022 . Learning free gait transition for quadruped robots via phase-guided controller . IEEE Robot Autom Lett , 7 ( 2 ): 1230 - 1237 . https://doi.org/10.1109/LRA.2021.3136645 https://doi.org/10.1109/LRA.2021.3136645

Siekmann J , Valluri S , Dao J , et al. , 2020 . Learning memory-based control for human-scale bipedal locomotion . Proc 16 th Robotics: Science and Systems , p. 1 - 8 . https://doi.org/10.15607/rss.2020.xvi.031 https://doi.org/10.15607/rss.2020.xvi.031

Siekmann J , Green K , Warila J , et al. , 2021a . Blind bipedal stair traversal via sim-to-real reinforcement learning . Proc 17 th Robotics: Science and Systems , p. 1 - 9 . https://doi.org/10.15607/rss.2021.xvii.061 https://doi.org/10.15607/rss.2021.xvii.061

Siekmann J , Godse Y , Fern A , et al. , 2021b . Sim-to-real learning of all common bipedal gaits via periodic reward composition . Proc IEEE Int Conf on Robotics and Automation , p. 7309 - 7315 . https://doi.org/10.1109/ICRA48506.2021.9561814 https://doi.org/10.1109/ICRA48506.2021.9561814

Tan DCH , Zhang J , Chuah M , et al. , 2023 . Perceptive locomotion with controllable pace and natural gait transitions over uneven terrains . https://arxiv.org/abs/2301.10894 https://arxiv.org/abs/2301.10894

Tan J , Zhang TN , Coumans E , et al. , 2018 . Sim-to-real: learning agile locomotion for quadruped robots . Proc 14 th Robotics: Science and Systems , p. 1 - 9 . https://doi.org/10.15607/RSS.2018.XIV.010 https://doi.org/10.15607/RSS.2018.XIV.010

Tsounis V , Alge M , Lee J , et al. , 2020 . DeepGait: planning and control of quadrupedal gaits using deep reinforcement learning . IEEE Robot Autom Lett , 5 ( 2 ): 3699 - 3706 . https://doi.org/10.1109/LRA.2020.2979660 https://doi.org/10.1109/LRA.2020.2979660

Xi WT , Yesilevskiy Y , Remy CD , 2016 . Selecting gaits for economical locomotion of legged robots . Int J Robot Res , 35 ( 9 ): 1140 - 1154 . https://doi.org/10.1177/0278364915612572 https://doi.org/10.1177/0278364915612572

Xie ZM , Da XY , van de Panne M , et al. , 2021 . Dynamics randomization revisited: a case study for quadrupedal locomotion . Proc IEEE Int Conf on Robotics and Automation , p. 4955 - 4961 . https://doi.org/10.1109/ICRA48506.2021.9560837 https://doi.org/10.1109/ICRA48506.2021.9560837

Yang CY , Yuan K , Zhu QG , et al. , 2020 . Multi-expert learning of adaptive legged locomotion . Sci Robot , 5 ( 49 ): eabb2174 . https://doi.org/10.1126/scirobotics.abb2174 https://doi.org/10.1126/scirobotics.abb2174

Zhang H , Starke S , Komura T , et al. , 2018 . Mode-adaptive neural networks for quadruped motion control . ACM Trans Graph , 37 ( 4 ): 145 . https://doi.org/10.1145/3197517.3201366 https://doi.org/10.1145/3197517.3201366

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

Jiu fusion artificial intelligence (JFA): a two-stage reinforcement learning model with hierarchical neural networks and human knowledge for Tibetan Jiu chess

Reinforcement learning based privacy-preserving consensus tracking control of nonstrict-feedback discrete-time multi-agent systems

Domain adaptation in reinforcement learning: a comprehensive and systematic study

Multi-agent reinforcement learning behavioral control for nonlinear second-order systems

An anti-collision algorithm for robotic search-and-rescue tasks in unknown dynamic environments