面向未知连续非线性系统的间接自适应模糊规划最优控制方法

张海运; 孟德远; 王进; 陆国栋

doi:10.1631/FITEE.1900610

Your Location：

Home >

Browse articles >

面向未知连续非线性系统的间接自适应模糊规划最优控制方法

常规文章 | Updated：2022-05-19

- 面向未知连续非线性系统的间接自适应模糊规划最优控制方法
- Indirect adaptive fuzzy-regulated optimal control for unknown continuous-time nonlinear systems
- 信息与电子工程前沿（英文） 2021年22卷第2期页码：155-169
- Affiliations：
  
  State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China
  Department of Mechatronic Engineering, China University of Mining and Technology, Xuzhou 221116, China
- Author bio：
  
  [ "Haiyun ZHANG, E-mail: gray_sun@zju.edu" ]
  [ "Deyuan MENG, E-mail: tinydreams@126.com" ]
  Jin WANG, E-mail: dwjcom@zju.edu.cn
  [ "Guodong LU, E-mail: lugd@zju.edu.cn" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 51805531 and 51675470), the Natural Science Foundation of Jiangsu Province, China (No. BK20150200), the Key R & D Program of Zhejiang Province, China (No. 2020C01026), and the China Postdoctoral Science Foundation (No. 2020M671706)
- DOI：10.1631/FITEE.1900610
  中图分类号： TP13
- 收稿：2019-11-11，
  
  修回：2020-;9-28，
  
  网络出版：2021-01-08，
  
  纸质出版：2021-02
- Accepted：
Scan QR Code
张海运, 孟德远, 王进, 等. 面向未知连续非线性系统的间接自适应模糊规划最优控制方法[J]. 信息与电子工程前沿（英文）, 2021,22(2):155-169.

Haiyun ZHANG, Deyuan MENG, Jin WANG, et al. Indirect adaptive fuzzy-regulated optimal control for unknown continuous-time nonlinear systems[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(2): 155-169.
张海运, 孟德远, 王进, 等. 面向未知连续非线性系统的间接自适应模糊规划最优控制方法[J]. 信息与电子工程前沿（英文）, 2021,22(2):155-169. DOI： 10.1631/FITEE.1900610.

Haiyun ZHANG, Deyuan MENG, Jin WANG, et al. Indirect adaptive fuzzy-regulated optimal control for unknown continuous-time nonlinear systems[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(2): 155-169. DOI： 10.1631/FITEE.1900610.

摘要

针对动力学未知、不匹配和扰动条件下的连续非线性系统，提出一种新的间接自适应模糊规划最优控制方案。首先，建立非线性系统汉密尔顿-雅各比-贝尔曼（HJB）方程及其匹配的性能函数。与现有自适应动态规划（ADP）方法不同，在执行器-评判器架构下，所提方案采用特殊的非二次变量性能函数作为强化媒介。构造一个自适应模糊规划的评判器结构来配置性能函数的权重矩阵，以逼近和平衡非线性HJB方程。同时，设计一种并行的自组织学习技术用于自适应更新该评判器的权重。在此基础上，提出一种自适应最优反馈控制器与一个新形式的增广黎卡提方程作为执行器，实时优化模糊规划后的性能函数。通过设计上述执行器-评判器架构获得一种在线间接自适应最优控制机制，可同时实现最优成本函数和最优控制策略的连续实时自适应调整。该方法的控制收敛性和闭环稳定性得到证明和保证。最后，仿真和比较表明所提方案的有效性和可靠性。

Abstract

We present a novel indirect adaptive fuzzy-regulated optimal control scheme for continuous-time nonlinear systems with unknown dynamics

mismatches

and disturbances. Initially

the Hamilton-Jacobi-Bellman (HJB) equation associated with its performance function is derived for the original nonlinear systems. Unlike existing adaptive dynamic programming (ADP) approaches

this scheme uses a special non-quadratic variable performance function as the reinforcement medium in the actor-critic architecture. An adaptive fuzzy-regulated critic structure is correspondingly constructed to configure the weighting matrix of the performance function for the purpose of approximating and balancing the HJB equation. A concurrent self-organizing learning technique is designed to adaptively update the critic weights. Based on this particular critic

an adaptive optimal feedback controller is developed as the actor with a new form of augmented Riccati equation to optimize the fuzzy-regulated variable performance function in real time. The result is an online indirect adaptive optimal control mechanism implemented as an actor-critic structure

which involves continuous-time adaptation of both the optimal cost and the optimal control policy. The convergence and closed-loop stability of the proposed system are proved and guaranteed. Simulation examples and comparisons show the effectiveness and advantages of the proposed method.

关键词

Keywords

references

M Abu-Khalaf , , , FL Lewis . . Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach . . Automatica , , 2005 . . 41 ( ( 5 ): ): 779 - - 791 . . DOI: 10.1016/j.automatica.2004.11.034 http://doi.org/10.1016/j.automatica.2004.11.034 . .

S Bhasin , , , R Kamalapurkar , , , M Johnson , , , 等 . . A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems . . Automatica , , 2013 . . 49 ( ( 1 ): ): 82 - - 92 . . DOI: 10.1016/j.automatica.2012.09.019 http://doi.org/10.1016/j.automatica.2012.09.019 . .

T Bian , , , ZP Jiang . . Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design . . Automatica , , 2016 . . 71 348 - - 360 . . DOI: 10.1016/j.automatica.2016.05.003 http://doi.org/10.1016/j.automatica.2016.05.003 . .

XH Chang , , , C Yang , , , J Xiong . . Quantized fuzzy output feedback H ∞ control for nonlinear systems with adjustment of dynamic parameters . . IEEE Trans Syst Man Cybern Syst , , 2019 . . 49 ( ( 10 ): ): 2005 - - 2015 . . DOI: 10.1109/TSMC.2018.2867213 http://doi.org/10.1109/TSMC.2018.2867213 . .

Y Chang , , , YQ Wang , , , FE Alsaadi , , , 等 . . Adaptive fuzzy output-feedback tracking control for switched stochastic pure-feedback nonlinear systems . . Int J Adapt Contr Signal Process , , 2019 . . 33 ( ( 10 ): ): 1567 - - 1582 . . DOI: 10.1002/acs.3052 http://doi.org/10.1002/acs.3052 . .

BA Finlayson . . The Method of Weighted Residuals and Variational Principles , , : : New York Academic Press , , 1990 . . .

X Huo , , , L Ma , , , XD Zhao , , , 等 . . Event-triggered adaptive fuzzy output feedback control of MIMO switched nonlinear systems with average dwell time . . Appl Math Comput , , 2020 . . 365 124665 DOI: 10.1016/j.amc.2019.124665 http://doi.org/10.1016/j.amc.2019.124665 . .

PA Ioannou , , , B Fidan . . Advances in Design and Control. Adaptive Control Tutorial , , : : SIAM, Philadelphia, USA 2006 . . .

Y Jiang , , , ZP Jiang . . Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics . . Automatica , , 2012 . . 48 ( ( 10 ): ): 2699 - - 2704 . . DOI: 10.1016/j.automatica.2012.06.096 http://doi.org/10.1016/j.automatica.2012.06.096 . .

Y Jiang , , , ZP Jiang . . Robust adaptive dynamic programming and feedback stabilization of nonlinear systems . . IEEE Trans Neur Netw Learn Syst , , 2014 . . 25 ( ( 5 ): ): 882 - - 893 . . DOI: 10.1109/TNNLS.2013.2294968 http://doi.org/10.1109/TNNLS.2013.2294968 . .

B Kiumarsi , , , FL Lewis , , , H Modares , , , 等 . . Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics . . Automatica , , 2014 . . 50 ( ( 4 ): ): 1167 - - 1175 . . DOI: 10.1016/j.automatica.2014.02.015 http://doi.org/10.1016/j.automatica.2014.02.015 . .

JM Lee , , , JH Lee . . Approximate dynamic programming strategies and their applicability for process control: a review and future directions . . Int J Contr Autom Syst , , 2004 . . 2 ( ( 3 ): ): 263 - - 278 . . . .

JY Lee , , , JB Park , , , YH Choi . . Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems . . Automatica , , 2012 . . 48 ( ( 11 ): ): 2850 - - 2859 . . DOI: 10.1016/j.automatica.2012.06.008 http://doi.org/10.1016/j.automatica.2012.06.008 . .

JY Lee , , , JB Park , , , YH Choi . . Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations . . IEEE Trans Neur Netw Learn Syst , , 2015 . . 26 ( ( 5 ): ): 916 - - 932 . . DOI: 10.1109/TNNLS.2014.2328590 http://doi.org/10.1109/TNNLS.2014.2328590 . .

FL Lewis , , , DL Vrabie , , , VL Syrmos . . Optimal Control (3 rd Ed.) , , : : Wiley, Hoboken, USA 2012a . . .

FL Lewis , , , D Vrabie , , , KG Vamvoudakis . . Reinforcement learning and feedback control: using natural decision methods to design optimal adaptive controllers . . IEEE Contr Syst Mag , , 2012b . . 32 ( ( 6 ): ): 76 - - 105 . . DOI: 10.1109/MCS.2012.2214134 http://doi.org/10.1109/MCS.2012.2214134 . .

YM Li , , , SC Tong , , , TS Li . . Hybrid fuzzy adaptive output feedback control design for uncertain MIMO nonlinear systems with time-varying delays and input saturation . . IEEE Trans Fuzzy Syst , , 2016 . . 24 ( ( 4 ): ): 841 - - 853 . . DOI: 10.1109/TFUZZ.2015.2486811 http://doi.org/10.1109/TFUZZ.2015.2486811 . .

WS Lin . . Optimality and convergence of adaptive optimal control by reinforcement synthesis . . Automatica , , 2011 . . 47 ( ( 5 ): ): 1047 - - 1052 . . DOI: 10.1016/j.automatica.2011.01.060 http://doi.org/10.1016/j.automatica.2011.01.060 . .

DR Liu , , , QL Wei . . Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems . . IEEE Trans Cybern , , 2013 . . 43 ( ( 2 ): ): 779 - - 789 . . DOI: 10.1109/TSMCB.2012.2216523 http://doi.org/10.1109/TSMCB.2012.2216523 . .

DR Liu , , , X Yang , , , HL Li . . Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics . . Neur Comput Appl , , 2013 . . 23 ( ( 7 ): ): 1843 - - 1850 . . DOI: 10.1007/s00521-012-1249-y http://doi.org/10.1007/s00521-012-1249-y . .

DR Liu , , , D Wang , , , FY Wang , , , 等 . . Neural-network-based online HJB solution for optimal robust guaranteed cost control of continuous-time uncertain nonlinear systems . . IEEE Trans Cybern , , 2014 . . 44 ( ( 12 ): ): 2834 - - 2847 . . DOI: 10.1109/TCYB.2014.2357896 http://doi.org/10.1109/TCYB.2014.2357896 . .

L Ma , , , X Huo , , , XD Zhao , , , 等 . . Adaptive fuzzy tracking control for a class of uncertain switched nonlinear systems with multiple constraints: a small-gain approach . . Int J Fuzzy Syst , , 2019 . . 21 ( ( 8 ): ): 2609 - - 2624 . . DOI: 10.1007/s40815-019-00708-9 http://doi.org/10.1007/s40815-019-00708-9 . .

H Modares , , , FL Lewis . . Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning . . Automatica , , 2014 . . 50 ( ( 7 ): ): 1780 - - 1792 . . DOI: 10.1016/j.automatica.2014.05.011 http://doi.org/10.1016/j.automatica.2014.05.011 . .

H Modares , , , Sistani MB Naghibi , , , FL Lewis . . A policy iteration approach to online optimal control of continuous-time constrained-input systems . . ISA Trans , , 2013 . . 52 ( ( 5 ): ): 611 - - 621 . . DOI: 10.1016/j.isatra.2013.04.004 http://doi.org/10.1016/j.isatra.2013.04.004 . .

JJ Murray , , , CJ Cox , , , GG Lendaris , , , 等 . . Adaptive dynamic programming . . IEEE Trans Syst Man Cybern Part C , , 2002 . . 32 ( ( 2 ): ): 140 - - 153 . . DOI: 10.1109/TSMCC.2002.801727 http://doi.org/10.1109/TSMCC.2002.801727 . .

R Padhi , , , N Unnikrishnan , , , XH Wang , , , 等 . . A Single Network Adaptive Critic (SNAC) architecture for optimal control synthesis for a class of nonlinear systems . . Neur Netw , , 2006 . . 19 ( ( 10 ): ): 1648 - - 1660 . . DOI: 10.1016/j.neunet.2006.08.010 http://doi.org/10.1016/j.neunet.2006.08.010 . .

WB Powell . . : : Approximate Dynamic Programming Solving the Curses of Dimensionality , , Wiley, New York, USA. , , 2007 . . .

SS Sastry . . : : Nonlinear Systems Analysis, Stability, and Control , , Springer-Verlag, New York, USA , , 1999 . . .

JE Slotine , , , W Li . . Applied Nonlinear Control , , : : Prentice Hall, Englewood Cliffs, NJ, USA 1991 . . .

RZ Song , , , WD Xiao , , , HG Zhang , , , 等 . . Adaptive dynamic programming for a class of complex-valued nonlinear systems . . IEEE Trans Neur Netw Learn Syst , , 2014 . . 25 ( ( 9 ): ): 1733 - - 1739 . . DOI: 10.1109/TNNLS.2014.2306201 http://doi.org/10.1109/TNNLS.2014.2306201 . .

G Tao . . Adaptive Control Design and Analysis. In: Adaptive and Learning Systems for Signal Processing, Communications and Control Series , , : : Wiley-Interscience, Hoboken, NJ, USA 2003 . . .

KG Vamvoudakis . . Q-learning for continuous-time linear systems: a model-free infinite horizon optimal control approach . . Syst Contr Lett , , 2017 . . 100 14 - - 20 . . DOI: 10.1016/j.sysconle.2016.12.003 http://doi.org/10.1016/j.sysconle.2016.12.003 . .

KG Vamvoudakis , , , FL Lewis . . Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem . . Automatica , , 2010 . . 46 ( ( 5 ): ): 878 - - 888 . . DOI: 10.1016/j.automatica.2010.02.018 http://doi.org/10.1016/j.automatica.2010.02.018 . .

AJ van der Schaft . . L 2 -gain analysis of nonlinear systems and nonlinear state-feedback H 1 control . . IEEE Trans Autom Contr , , 1992 . . 37 ( ( 6 ): ): 770 - - 784 . . DOI: 10.1109/9.256331 http://doi.org/10.1109/9.256331 . .

D Vrabie , , , O Pastravanu , , , M Abu-Khalaf , , , 等 . . Adaptive optimal control for continuous-time linear systems based on policy iteration . . Automatica , , 2009 . . 45 ( ( 2 ): ): 477 - - 484 . . DOI: 10.1016/j.automatica.2008.08.017 http://doi.org/10.1016/j.automatica.2008.08.017 . .

FY Wang , , , HG Zhang , , , DR Liu . . Adaptive dynamic programming: an introduction . . IEEE Comput Intell Mag , , 2009 . . 4 ( ( 2 ): ): 39 - - 47 . . DOI: 10.1109/MCI.2009.932261 http://doi.org/10.1109/MCI.2009.932261 . .

QL Wei , , , HG Zhang , , , J Dai . . Model-free multiobjective approximate dynamic programming for discrete-time nonlinear systems with general performance index functions . . Neurocomputing , , 2009 . . 72 ( ( 8-9 ): ): 1839 - - 1848 . . DOI: 10.1016/j.neucom.2008.05.012 http://doi.org/10.1016/j.neucom.2008.05.012 . .

P Werbos . . ADP: goals, opportunities and principles. In: Si J, Barto A, Powell W, et al. (Eds.), Handbook of Learning and Approximate Dynamic Programming , , : : New York Institute of Electrical and Electronics Engineers , , 2004 . . 3 - - 44 . . . .

X Yang , , , HB He . . Self-learning robust optimal control for continuous-time nonlinear systems with mismatched disturbances . . Neur Netw , , 2018 . . 99 19 - - 30 . . DOI: 10.1016/j.neunet.2017.11.022 http://doi.org/10.1016/j.neunet.2017.11.022 . .

X Yang , , , DR Liu , , , B Luo , , , 等 . . Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning . . Inform Sci , , 2016 . . 369 731 - - 747 . . DOI: 10.1016/j.ins.2016.07.051 http://doi.org/10.1016/j.ins.2016.07.051 . .

XY Yang , , , DR Liu , , , YZ Huang . . Neural-network-based online optimal control for uncertain non-linear continuous-time systems with control constraints . . IET Contr Theory Appl , , 2013 . . 7 ( ( 17 ): ): 2037 - - 2047 . . DOI: 10.1049/iet-cta.2013.0472 http://doi.org/10.1049/iet-cta.2013.0472 . .

YF Yin , , , XD Zhao , , , XL Zheng . . New stability and stabilization conditions of switched systems with mode-dependent average dwell time . . Circ Syst Signal Process , , 2017 . . 36 ( ( 1 ): ): 82 - - 98 . . DOI: 10.1007/s00034-016-0306-7 http://doi.org/10.1007/s00034-016-0306-7 . .

ZX Yu , , , YK Yang , , , SG Li , , , 等 . . Observer-based adaptive finite-time quantized tracking control of nonstrict-feedback nonlinear systems with asymmetric actuator saturation . . IEEE Trans Syst Man Cyber Syst , , 2018 . . 50 ( ( 11 ): ): 545 - - 4556 . . DOI: 10.1109/TSMC.2018.2854927 http://doi.org/10.1109/TSMC.2018.2854927 . .

SH Zak . . Systems and Control , , : : New York, USA Oxford University Press , , 2003 . . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据