Probabilistic movement primitive based motion learning for a lower limb exoskeleton with black-box optimization

Jiaqi WANG; Yongzhuo GAO; Dongmei WU; Wei DONG

doi:10.1631/FITEE.2200065

Your Location：

Home >

Browse articles >

Probabilistic movement primitive based motion learning for a lower limb exoskeleton with black-box optimization

Regular Papers | Updated：2023-01-28

- Probabilistic movement primitive based motion learning for a lower limb exoskeleton with black-box optimization
- 针对下肢外骨骼机器人的基于概率运动基元的黑盒优化运动学习
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 1, Pages: 104-116(2023)
- Affiliations：
  
  State Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin 150001, China
- Author bio：
  
  E-mail: wangjq@hit.edu.cn;
  gaoyongzhuo@hit.edu.cn;
  wdm@hit.edu.cn;
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2200065
  CLC： TP242.6
- Received：22 February 2022，
  
  Revised：2022-08-08，
  
  Published：2023-01
- Accepted：
Scan QR Code
Jiaqi WANG, Yongzhuo GAO, Dongmei WU, et al. Probabilistic movement primitive based motion learning for a lower limb exoskeleton with black-box optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(1): 104-116.
DOI：

Jiaqi WANG, Yongzhuo GAO, Dongmei WU, et al. Probabilistic movement primitive based motion learning for a lower limb exoskeleton with black-box optimization[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(1): 104-116. DOI： 10.1631/FITEE.2200065.

摘要

外骨骼作为一种可穿戴的机器人，通过拟人化的构型直接传递机械动力来辅助或增强穿戴者运动。当外骨骼用于促进穿戴者的运动时，运动生成过程通常在高层控制中发挥重要作用。该领域的主要挑战之一是实时生成符合人类意图且可以适应不同情况的参考轨迹。在本文中，我们首先提出了一种基于概率运动基元（ProMP）的下肢外骨骼运动建模方法，它是一种用于生成运动轨迹的新型且强大的代表性工具。为了在不同穿戴者使用外骨骼时使轨迹适应不同情况，我们接着提出了一种基于黑盒优化PI

结合ProMP的新型运动学习方案。运动模型首先由ProMP离线学习，它可以生成参考轨迹供外骨骼控制器在线使用，再采用PI

在线学习和更新模型，提供了系统的自适应能力，消除了不确定性的影响。使用下肢外骨骼HEXO对六名受试者进行的模拟和实验证明了所提出方法的有效性。

Abstract

As a wearable robot

an exoskeleton provides a direct transfer of mechanical power to assist or augment the wearer’s movement with an anthropomorphic configuration. When an exoskeleton is used to facilitate the wearer’s movement

a motion generation process often plays an important role in high-level control. One of the main challenges in this area is to generate in real time a reference trajectory that is parallel with human intention and can adapt to different situations. In this paper

we first describe a novel motion modeling method based on probabilistic movement primitive (ProMP) for a lower limb exoskeleton

which is a new and powerful representative tool for generating motion trajectories. To adapt the trajectory to different situations when the exoskeleton is used by different wearers

we propose a novel motion learning scheme based on black-box optimization (BBO) PI

combined with ProMP. The motion model is first learned by ProMP offline

which can generate reference trajectories for use by exoskeleton controllers online. PI

is adopted to learn and update the model for online trajectory generation

which provides the capability of adaptation of the system and eliminates the effects of uncertainties. Simulations and experiments involving six subjects using the lower limb exoskeleton HEXO demonstrate the effectiveness of the proposed methods.

关键词

Keywords

references

Al-Shuka HFN , Corves B , Zhu WH , et al. , 2016 . Multi-level control of zero-moment point-based humanoid biped robots: a review . Robotica , 34 ( 11 ): 2440 - 2466 . https://doi.org/10.1017/S0263574715000107 https://doi.org/10.1017/S0263574715000107

Colombo G , Joerg M , Schreier R , et al. , 2000 . Treadmill training of paraplegic patients using a robotic orthosis . J Rehabil Res Dev , 37 ( 6 ): 693 - 700 .

d'Avella A , Bizzi E , 2005 . Shared and specific muscle synergies in natural motor behaviors . Proc Nat Acad Sci USA , 102 ( 8 ): 3076 - 3081 . https://doi.org/10.1073/pnas.0500199102 https://doi.org/10.1073/pnas.0500199102

Deng MD , Li ZJ , Kang Y , et al. , 2020 . A learning-based hierarchical control scheme for an exoskeleton robot in human-robot cooperative manipulation . IEEE Trans Cybern , 50 ( 1 ): 112 - 125 . https://doi.org/10.1109/TCYB.2018.2864784 https://doi.org/10.1109/TCYB.2018.2864784

Esquenazi A , Talaty M , Packel A , et al. , 2012 . The ReWalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury . Am J Phys Med Rehabil , 91 ( 11 ): 911 - 921 . https://doi.org/10.1097/PHM.0b013e318269d9a3 https://doi.org/10.1097/PHM.0b013e318269d9a3

Fu CL , Chen K , 2008 . Gait synthesis and sensory control of stair climbing for a humanoid robot . IEEE Trans Ind Electron , 55 ( 5 ): 2111 - 2120 . https://doi.org/10.1109/TIE.2008.921205 https://doi.org/10.1109/TIE.2008.921205

Guizzo E , Goldstein H , 2005 . The rise of the body bots [robotic exoskeletons] . IEEE Spectr , 42 ( 10 ): 50 - 56 . https://doi.org/10.1109/MSPEC.2005.1515961 https://doi.org/10.1109/MSPEC.2005.1515961

Hassan M , Kadone H , Suzuki K , et al. , 2014 . Wearable gait measurement system with an instrumented cane for exoskeleton control . Sensors , 14 ( 1 ): 1705 - 1722 . https://doi.org/10.3390/s140101705 https://doi.org/10.3390/s140101705

He W , Li ZJ , Chen CLP , 2017 . A survey of human-centered intelligent robots: issues and challenges . IEEE/CAA J Autom Sin , 4 ( 4 ): 602 - 609 . https://doi.org/10.1109/JAS.2017.7510604 https://doi.org/10.1109/JAS.2017.7510604

Huang R , Cheng H , Guo H , et al. , 2018 . Hierarchical learning control with physical human-exoskeleton interaction . Inform Sci , 432 : 584 - 595 . https://doi.org/10.1016/j.ins.2017.09.068 https://doi.org/10.1016/j.ins.2017.09.068

Huang R , Cheng H , Qiu J , et al. , 2019 . Learning physical human-robot interaction with coupled cooperative primitives for a lower exoskeleton . IEEE Trans Autom Sci Eng , 16 ( 4 ): 1566 - 1574 . https://doi.org/10.1109/TASE.2018.2886376 https://doi.org/10.1109/TASE.2018.2886376

Ijspeert AJ , Nakanishi J , Schaal S , 2002 . Movement imitation with nonlinear dynamical systems in humanoid robots . Proc IEEE Int Conf on Robotics and Automation , p. 1398 - 1403 . https://doi.org/10.1109/ROBOT.2002.1014739 https://doi.org/10.1109/ROBOT.2002.1014739

Ijspeert AJ , Nakanishi J , Hoffmann H , et al. , 2013 . Dynamical movement primitives: learning attractor models for motor behaviors . Neur Comput , 25 ( 2 ): 328 - 373 . https://doi.org/10.1162/NECO_a_00393 https://doi.org/10.1162/NECO_a_00393

Jenison RL , Fissell K , 1995 . A comparison of the von Mises and Gaussian basis functions for approximating spherical acoustic scatter . IEEE Trans Neur Netw , 6 ( 5 ): 1284 - 1287 . https://doi.org/10.1109/72.410375 https://doi.org/10.1109/72.410375

Kagawa T , Ishikawa H , Kato T , et al. , 2015 . Optimization-based motion planning in joint space for walking assistance with wearable robot . IEEE Trans Rob , 31 ( 2 ): 415 - 424 . https://doi.org/10.1109/TRO.2015.2409434 https://doi.org/10.1109/TRO.2015.2409434

Kazemi J , Ozgoli S , 2019 . Real-time walking pattern generation for a lower limb exoskeleton, implemented on the exoped robot . Rob Auton Syst , 116 : 1 - 23 . https://doi.org/10.1016/j.robot.2019.02.012 https://doi.org/10.1016/j.robot.2019.02.012

Kazerooni H , Steger R , 2006 . The Berkeley lower extremity exoskeleton . J Dynam Syst Meas Contr , 128 ( 1 ): 14 - 25 . https://doi.org/10.1115/1.2168164 https://doi.org/10.1115/1.2168164

Komura T , Nagano A , Leung H , et al. , 2005 . Simulating pathological gait using the enhanced linear inverted pendulum model . IEEE Trans Biomed Eng , 52 ( 9 ): 1502 - 1513 . https://doi.org/10.1109/TBME.2005.851530 https://doi.org/10.1109/TBME.2005.851530

Krüger V , Kragic D , Ude A , et al. , 2007 . The meaning of action: a review on action recognition and mapping . Adv Rob , 21 ( 13 ): 1473 - 1501 . https://doi.org/10.1163/156855307782148578 https://doi.org/10.1163/156855307782148578

Kulić D , Ott C , Lee D , et al. , 2012 . Incremental learning of full body motion primitives and their sequencing through human motion observation . Int J Rob Res , 31 ( 3 ): 330 - 345 . https://doi.org/10.1177/0278364911426178 https://doi.org/10.1177/0278364911426178

Lee SW , Yi T , Jung JW , et al. , 2015 . Design of a gait phase recognition system that can cope with EMG electrode location variation . IEEE Trans Autom Sci Eng , 14 ( 3 ):‍ 1429 - 1439 . https://doi.org/10.1109/TASE.2015.2477283 https://doi.org/10.1109/TASE.2015.2477283

Paraschos A , Daniel C , Peters J , et al. , 2013 . Probabilistic movement primitives . Proc 26 th Int Conf on Neural Information Processing Systems , p. 2616 - 2624 .

Paraschos A , Daniel C , Peters J , et al. , 2018 . Using probabilistic movement primitives in robotics . Auton Rob , 42 ( 3 ): 529 - 551 . https://doi.org/10.1007/s10514-017-9648-7 https://doi.org/10.1007/s10514-017-9648-7

Sankai Y , 2010 . HAL: hybrid assistive limb based on cybernics . In: Kaneko M , Nakamura Y (Eds.), Robotics Research . Springer , Berlin, Heidelberg , p. 25 - 34 . https://doi.org/10.1007/978-3-642-14743-2_3 https://doi.org/10.1007/978-3-642-14743-2_3

Sanz-Merodio D , Cestari M , Arevalo JC , et al. , 2014 . Generation and control of adaptive gaits in lower-limb exoskeletons for motion assistance . Adv Rob , 28 ( 5 ): 329 - 338 . https://doi.org/10.1080/01691864.2013.867284 https://doi.org/10.1080/01691864.2013.867284

Schaal S , Ijspeert A , Billard A , 2003 . Computational approaches to motor learning by imitation . Phil Trans R Soc B Biol Sci , 358 ( 1431 ): 537 - 547 . https://doi.org/10.1098/rstb.2002.1258 https://doi.org/10.1098/rstb.2002.1258

Schaal S , Peters J , Nakanishi J , et al. , 2005 . Learning movement primitives . 11 th Int Symp on Robotics Research , p. 561 - 572 . https://doi.org/10.1007/11008941_60 https://doi.org/10.1007/11008941_60

Schmidhuber J , 2015 . Deep learning in neural networks: an overview . Neur Netw , 61 : 85 - 117 . https://doi.org/10.1016/j.neunet.2014.09.003 https://doi.org/10.1016/j.neunet.2014.09.003

Spiegelhalter DJ , Best NG , Carlin BP , et al. , 2002 . Bayesian measures of model complexity and fit . J R Stat Soc Ser B (Stat Methodol) , 64 ( 4 ): 583 - 639 . https://doi.org/10.1111/1467-9868.00353 https://doi.org/10.1111/1467-9868.00353

Strausser KA , Kazerooni H , 2011 . The development and testing of a human machine interface for a mobile medical exoskeleton . Proc IEEE/RSJ Int Conf on Intelligent Robots and Systems , p. 4911 - 4916 . https://doi.org/10.1109/IROS.2011.6095025 https://doi.org/10.1109/IROS.2011.6095025

Stulp F , Sigaud O , 2012 . Policy Improvement Methods: Between Black-Box Optimization and Episodic Reinforcement Learning . https://hal.‍archives-ouvertes.‍fr/hal-00738463 https://hal.‍archives-ouvertes.‍fr/hal-00738463 [ Accessed on Jan. 12, 2021 ]

Theodorou E , Buchli J , Schaal S , 2010 . A generalized path integral control approach to reinforcement learning . J Mach Learn Res , 11 : 3137 - 3181 .

Todorov E , Jordan MI , 2002 . Optimal feedback control as a theory of motor coordination . Nat Neurosci , 5 ( 11 ): 1226 - 1235 . https://doi.org/10.1038/nn963 https://doi.org/10.1038/nn963

Tucker MR , Olivier J , Pagel A , et al. , 2015 . Control strategies for active lower extremity prosthetics and orthotics: a review . J Neuroeng Rehabil , 12 ( 1 ): 1 . https://doi.org/10.1186/1743-0003-12-1 https://doi.org/10.1186/1743-0003-12-1

Veneman JF , Kruidhof R , Hekman EEG , et al. , 2007 . Design and evaluation of the LOPES exoskeleton robot for interactive gait rehabilitation . IEEE Trans Neur Syst Rehabil Eng , 15 ( 3 ): 379 - 386 . https://doi.org/10.1109/TNSRE.2007.903919 https://doi.org/10.1109/TNSRE.2007.903919

Vukobratović M , Borovac B , 2004 . Zero-moment point—thirty five years of its life . Int J Human Rob , 1 ( 1 ): 157 - 173 . https://doi.org/10.1142/S0219843604000083 https://doi.org/10.1142/S0219843604000083

Walsh CJ , Endo K , Herr H , 2007 . A quasi-passive leg exoskeleton for load-carrying augmentation . Int J Human Rob , 4 ( 3 ): 487 - 506 . https://doi.org/10.1142/S0219843607001126 https://doi.org/10.1142/S0219843607001126

Xu B , Sun FC , 2018 . Composite intelligent learning control of strict-feedback systems with disturbance . IEEE Trans Cybern , 48 ( 2 ): 730 - 741 . https://doi.org/10.1109/TCYB.2017.2655053 https://doi.org/10.1109/TCYB.2017.2655053

Yan TF , Cempini M , Oddo CM , et al. , 2015 . Review of assistive strategies in powered lower-limb orthoses and exoskeletons . Rob Auton Syst , 64 : 120 - 136 . https://doi.org/10.1016/j.robot.2014.09.032 https://doi.org/10.1016/j.robot.2014.09.032

Yang CG , Chen CZ , Wang N , et al. , 2019 . Biologically inspired motion modeling and neural control for robot learning from demonstrations . IEEE Trans Cogn Dev Syst , 11 ( 2 ): 281 - 291 . https://doi.org/10.1109/TCDS.2018.2866477 https://doi.org/10.1109/TCDS.2018.2866477

Yuan YX , Li ZJ , Zhao T , et al. , 2020 . DMP-based motion generation for a walking exoskeleton robot using reinforcement learning . IEEE Trans Ind Electron , 67 ( 5 ):‍ 3830 - 3839 . https://doi.org/10.1109/TIE.2019.2916396 https://doi.org/10.1109/TIE.2019.2916396

Zoss AB , Kazerooni H , Chu A , 2006 . Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX) . IEEE/ASME Trans Mechatr , 11 ( 2 ): 128 - 138 . https://doi.org/10.1109/TMECH.2006.871087 https://doi.org/10.1109/TMECH.2006.871087

Views

256

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Autonomous flying blimp interaction with human in an indoor space

Related Author

Ning-shi YAO

Qiu-yang TAO

Wei-yu LIU

Zhen LIU

Ye TIAN

Pei-yu WANG

Timothy LI

Fumin ZHANG

Related Institution

School of Electrical and Computer Engineering, Georgia Institute of Technology

College of Computing, Georgia Institute of Technology

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰