Adaptive dynamic programming for linear impulse systems

Xiao-hua WANG; Juan-juan YU; Yao HUANG; Hua WANG; Zhong-hua MIAO

doi:10.1631/jzus.C1300145

Your Location：

Home >

Browse articles >

Adaptive dynamic programming for linear impulse systems

Computer & Automation | Updated：2022-05-19

- Adaptive dynamic programming for linear impulse systems
- 线性脉冲系统的自适应动态规划方法
- Frontiers of Information Technology & Electronic Engineering Vol. 15, Issue 1, Pages: 43-50(2014)
- Affiliations：
  
  1.School of Mechatronics Engineering and Automation, Shanghai University, Shanghai 200072, China
  2.Shanghai Key Laboratory of Power Station Automation Technology, Shanghai University, Shanghai 200072, China
- Author bio：
  
  [ "Xiao-hua WANG, E-mail: x.wang@shu.edu.cn" ]
  Zhong-hua MIAO, E-mail: zhhmiao@shu.edu.cn
- Funds：
- DOI：10.1631/jzus.C1300145
  CLC： TP273.1
- Received：27 May 2013，
  
  Revised：2013-12-19，
  
  Published：2014-01
- Accepted：
Scan QR Code
Xiao-hua WANG, Juan-juan YU, Yao HUANG, et al. Adaptive dynamic programming for linear impulse systems[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(1): 43-50.
DOI：

Xiao-hua WANG, Juan-juan YU, Yao HUANG, et al. Adaptive dynamic programming for linear impulse systems[J]. Frontiers of Information Technology & Electronic Engineering, 2014, 15(1): 43-50. DOI： 10.1631/jzus.C1300145.

摘要

研究目的

针对线性脉冲系统最优控制，研究了基于自适应动态规划的递归方法。通过神经网络逼近最优目标函数，得出最优控制率。求解过程适用于一般脉冲系统，无需初始稳定控制器，为此类系统的最优控制提供理论依据。

创新要点

目前自适应动态规划方法研究局限于连续和离散系统，对脉冲系统研究较少。本文研究了线性脉冲系统的最优控制问题，运用自适应动态规划思路，完成了脉冲系统相关理论证明，证实了方法的收敛性。通过神经网络逼近最优目标函数，当迭代稳定后，神经网络获得稳定参数，同时获得最优脉冲控制率。

方法提亮

线性脉冲系统的最优目标函数是一个状态二次型，但其中的P矩阵表现为跳跃的脉冲形式。基于此发现，以迭代学习为基础的自适应动态规划方法适用于最优脉冲求解。本文提出的方法避免了直接迭代的矩阵求逆，大大降低了运算量。

重要结论

线性脉冲系统的最优目标函数表现为状态二次型，可通过自适应动态规划方法迭代求解，求解过程稳定。通过神经网络逼近最优目标函数，可避免矩阵求逆，大大降低计算量。

Abstract

We investigate the optimization of linear impulse systems with the reinforcement learning based adaptive dynamic programming (ADP) method. For linear impulse systems

the optimal objective function is shown to be a quadric form of the pre-impulse states. The ADP method provides solutions that iteratively converge to the optimal objective function. If an initial guess of the pre-impulse objective function is selected as a quadratic form of the pre-impulse states

the objective function iteratively converges to the optimal one through ADP. Though direct use of the quadratic objective function of the states within the ADP method is theoretically possible

the numerical singularity problem may occur due to the matrix inversion therein when the system dimensionality increases. A neural network based ADP method can circumvent this problem. A neural network with polynomial activation functions is selected to approximate the pre-impulse objective function and trained iteratively using the ADP method to achieve optimal control. After a successful training

optimal impulse control can be derived. Simulations are presented for illustrative purposes.

关键词

Keywords

references

N.U. Ahmed . . Existence of optimal controls for a general class of impulsive systems on Banach spaces . . SIAM J. Control Optim. , , 2003 . . 42 ( ( 2 ): ): 669 - - 685 . . DOI: 10.1137/S0363012901391299 http://doi.org/10.1137/S0363012901391299 . .

D.D. Bainov , , , P.S. Simeonov . . Impulsive Differential Equations: Asymptotic Properties of the Solutions . . World Scientific, Singapore , , 1995 . . .

S.N. Balakrishnan , , , J. Ding , , , F.L. Lewis . . Issues on stability of ADP feedback controllers for dynamical systems . . IEEE Tran. Syst. Man Cybern. B , , 2008 . . 38 ( ( 4 ): ): 913 - - 917 . . DOI: 10.1109/TSMCB.2008.926599 http://doi.org/10.1109/TSMCB.2008.926599 . .

D.P. Bertsekas . . Approximate policy iteration: a survey and some new methods . . J. Control Theory Appl. , , 2011 . . 9 ( ( 3 ): ): 310 - - 335 . . . .

T. Dierks , , , S. Jagannathan . . Online optimal control of nonlinear discrete-time systems using approximate dynamic programming . . J. Control Theory Appl. , , 2011 . . 9 ( ( 3 ): ): 361 - - 369 . . DOI: 10.1007/s11768-011-0178-0 http://doi.org/10.1007/s11768-011-0178-0 . .

S.L. Fraga , , , F.L. Pereira . . Hamilton-Jacobi-Bellman equation and feedback synthesis for impulsive control . . IEEE Trans. Automat. Control , , 2012 . . 57 ( ( 1 ): ): 244 - - 249 . . DOI: 10.1109/TAC.2011.2167822 http://doi.org/10.1109/TAC.2011.2167822 . .

Y. Jiang , , , Z.P. Jiang . . Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics . . Automatica , , 2012 . . 48 ( ( 10 ): ): 2699 - - 2704 . . DOI: 10.1016/j.automatica.2012.06.096 http://doi.org/10.1016/j.automatica.2012.06.096 . .

Z.P. Jiang , , , Y. Jiang . . Robust adaptive dynamic programming for linear and nonlinear systems: an overview . . Eur. J. Control , , 2013 . . 19 ( ( 5 ): ): 417 - - 425 . . DOI: 10.1016/j.ejcon.2013.05.017 http://doi.org/10.1016/j.ejcon.2013.05.017 . .

A.B. Kurzhanski , , , A.N. Daryin . . Dynamic programming for impulse controls . . Ann. Rev. Control , , 2008 . . 32 ( ( 2 ): ): 213 - - 227 . . DOI: 10.1016/j.arcontrol.2008.08.001 http://doi.org/10.1016/j.arcontrol.2008.08.001 . .

V. Lakshmikantham , , , D.D. Bainov , , , P.S. Simeonov . . Theory of Impulsive Differential Equations . . World Scientific, Singapore , , 1989 . . .

F.L. Lewis , , , D. Vrabie . . Reinforcement learning and adaptive dynamic programming for feedback control . . IEEE Circ. Syst. Mag. , , 2009 . . 9 ( ( 3 ): ): 32 - - 50 . . DOI: 10.1109/MCAS.2009.933854 http://doi.org/10.1109/MCAS.2009.933854 . .

B. Liu , , , K.L. Teo , , , X.Z. Liu . . Optimal control and robust stability of uncertain impulsive dynamical systems . . Asian J. Control , , 2008 . . 10 ( ( 3 ): ): 314 - - 326 . . DOI: 10.1002/asjc.37 http://doi.org/10.1002/asjc.37 . .

D.R. Liu , , , Q.L. Wei . . Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems . . IEEE Trans. Cybern. , , 2013 . . 43 ( ( 2 ): ): 779 - - 789 . . DOI: 10.1109/TSMCB.2012.2216523 http://doi.org/10.1109/TSMCB.2012.2216523 . .

X. Liu . . Impulsive control and optimization . . Appl. Math. Comput. , , 1995 . . 73 ( ( 1 ): ): 77 - - 98 . . DOI: 10.1016/0096-3003(94)00204-H http://doi.org/10.1016/0096-3003(94)00204-H . .

G.N. Silva , , , R.B. Vinter . . Necessary conditions for optimal impulsive control problems . . SIAM J. Control Optim. , , 1997 . . 35 ( ( 6 ): ): 1829 - - 1846 . . DOI: 10.1137/S0363012995281857 http://doi.org/10.1137/S0363012995281857 . .

K.G. Vamvoudakis , , , F.L. Lewis . . Online actor--critic algorithm to solve the continuous-time infinite horizon optimal control problem . . Automatica , , 2010 . . 46 ( ( 5 ): ): 878 - - 888 . . DOI: 10.1016/j.automatica.2010.02.018 http://doi.org/10.1016/j.automatica.2010.02.018 . .

F.Y. Wang , , , H.G. Zhang , , , D.R. Liu . . Adaptive dynamic programming: an introduction . . IEEE Comput. Intell. Mag. , , 2009 . . 4 ( ( 2 ): ): 39 - - 47 . . DOI: 10.1109/MCI.2009.932261 http://doi.org/10.1109/MCI.2009.932261 . .

J.R. Wang , , , Y.L. Yang . . Optimal control of linear impulsive antiperiodic boundary value problem on infinite dimensional spaces . . Discr. Dynam. Nat. Soc. , , 2010 . . Article ID 673013 DOI: 10.1155/2010/673013 http://doi.org/10.1155/2010/673013 . .

X.H. Wang . . Optimal Control of Impulsive Systems Using Adaptive Critic Neural Network . . PhD Thesis, Missouri University of Science and Technology, Rolla, Missouri, USA. , , 2008 . . .

X.H. Wang , , , S.N. Balakrishnan . . Optimal neurocontroller synthesis for impulse-driven systems . . Neur. Networks , , 2010 . . 23 ( ( 1 ): ): 125 - - 134 . . DOI: 10.1016/j.neunet.2009.08.009 http://doi.org/10.1016/j.neunet.2009.08.009 . .

X.H. Wang , , , W.Z. Luo , , , S.N. Balakrishnan . . Linear impulsive system optimization using adaptive dynamic programming . . 12th Int. Conf. on Control Automation Robotics and Vision , , 2012 . . p.725 - - 730 . . DOI: 10.1109/ICARCV.2012.6485247 http://doi.org/10.1109/ICARCV.2012.6485247 . .

P.J. Werbos . . Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences . . PhD Thesis, Harvard University, USA , , 1974 . . .

P.J. Werbos . . Foreword-ADP: the key direction for future research in intelligent control and understanding brain intelligence . . IEEE Trans. Syst. Man Cybern. B , , 2008 . . 38 ( ( 4 ): ): 898 - - 900 . . DOI: 10.1109/TSMCB.2008.924139 http://doi.org/10.1109/TSMCB.2008.924139 . .

P.J. Werbos , , , T. McAvoy , , , T. Su . . Handbook of Intelligent Control . . Van Nostrand Reinhold, New York , , 1992 . . .

Z. Wu , , , F. Zhang . . Stochastic maximum principle for optimal control problems of forward-backward systems involving impulse controls . . IEEE Trans. Automat. Control , , 2011 . . 56 ( ( 6 ): ): 1401 - - 1406 . . . .

T. Yang . . Impulsive control . . IEEE Trans. Automat. Control , , 1999 . . 44 ( ( 5 ): ): 1081 - - 1083 . . . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Neural mesh refinement

Accurate estimation of 6-DoF tooth pose in 3D intraoral scans for dental applications using deep learning

Attention-based efficient robot grasp detection network

Adaptive neural network based boundary control of a flexible marine riser system with output constraints

Adaptive tracking control of high-order MIMO nonlinear systems with prescribed performance

Related Author

Yiyi LIAO

Lu YU

Xiang GAO

Zhiwei ZHU

Wanghui DING

Kaiwei SUN

Mengfei YU

Hangzheng LIN

Related Institution

Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking (IPCAN)

College of Information Science and Electronic Engineering, Zhejiang University

Hangzhou Dental Hospital

Angel Align Inc

Zhejiang University–University of Illinois at Urbana-Champaign Institute, Zhejiang University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰