Modeling deterministic echo state network with loop reservoir

Xiao-chuan SUN; Hong-yan CUI; Ren-ping LIU; Jian-ya CHEN; Yun-jie LIU

doi:10.1631/jzus.C1200069

Your Location：

Home >

Browse articles >

Modeling deterministic echo state network with loop reservoir

计算机与自动化 | Updated：2022-12-09

- Modeling deterministic echo state network with loop reservoir
- Modeling deterministic echo state network with loop reservoir
- 信息与电子工程前沿（英文） 2012年13卷第9期页码：689-701
- Affiliations：
  
  1.Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China
  2.ICT Centre, Commonwealth Scientific and Industrial Research Organization, Sydney 1710, Australia
- Author bio：
  
  [ "Xiao-chuan SUN, E-mail: jrsxc@163.com" ]
- Funds：
  
  Project supported by the National Basic Research Program (973) of China (No. 2012CB315805), the Fundamental Research Funds for the Central Universities, China (No. 2009RC0124), the National Key Science and Technology Projects, China (No. 2010ZX03004-002-02), and the Australian Centre for Broadband Innovation (ACBI)
- DOI：10.1631/jzus.C1200069
  中图分类号： TP183
- 收稿：2012-03-15，
  
  修回：2012-;7-;6，
  
  纸质出版：2012-09
- Accepted：
Scan QR Code
Modeling deterministic echo state network with loop reservoir[J]. 信息与电子工程前沿（英文）, 2012,13(9):689-701.

Xiao-chuan SUN, Hong-yan CUI, Ren-ping LIU, et al. Modeling deterministic echo state network with loop reservoir[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(9): 689-701.
Modeling deterministic echo state network with loop reservoir[J]. 信息与电子工程前沿（英文）, 2012,13(9):689-701. DOI： 10.1631/jzus.C1200069.

Xiao-chuan SUN, Hong-yan CUI, Ren-ping LIU, et al. Modeling deterministic echo state network with loop reservoir[J]. Frontiers of Information Technology & Electronic Engineering, 2012, 13(9): 689-701. DOI： 10.1631/jzus.C1200069.

摘要

Abstract

Echo state network (ESN)

which efficiently models nonlinear dynamic systems

has been proposed as a special form of recurrent neural network. However

most of the proposed ESNs consist of complex reservoir structures

leading to excessive computational cost. Recently

minimum complexity ESNs were proposed and proved to exhibit high performance and low computational cost. In this paper

we propose a simple deterministic ESN with a loop reservoir

i.e.

an ESN with an adjacent- feedback loop reservoir. The novel reservoir is constructed by introducing regular adjacent feedback based on the simplest loop reservoir. Only a single free parameter is tuned

which considerably simplifies the ESN construction. The combination of a simplified reservoir and fewer free parameters provides superior prediction performance. In the benchmark datasets and real-world tasks

our scheme obtains higher prediction accuracy with relatively low complexity

compared to the classic ESN and the minimum complexity ESN. Furthermore

we prove that all the linear ESNs with the simplest loop reservoir possess the same memory capacity

arbitrarily converging to the optimal value.

关键词

Keywords

references

H. Abbasi Nozari , , , H. Dehghan Banadaki , , , M. Mokhtare , , , S. Hekmati Vahed . . Intelligent non-linear modeling of an industrial winding process using recurrent local linear neuro-fuzzy networks . . J. Zhejiang Univ.-Sci. C (Comput. & Electron.) , , 2012 . . 13 ( ( 6 ): ): 403 - - 412 . . DOI: 10.1631/jzus.C11a0278 http://doi.org/10.1631/jzus.C11a0278 . .

A.F. Atiya , , , A.G. Parlos . . New results on recurrent network training: unifying the algorithms and accelerating convergence . . IEEE Trans. Neur. Networks , , 2000 . . 11 ( ( 3 ): ): 697 - - 709 . . DOI: 10.1109/72.846741 http://doi.org/10.1109/72.846741 . .

S.P. Chatzis , , , Y. Demiris . . Echo state Gaussian process . . IEEE Trans. Neur. Networks , , 2011 . . 22 ( ( 9 ): ): 1435 - - 1445 . . DOI: 10.1109/TNN.2011.2162109 http://doi.org/10.1109/TNN.2011.2162109 . .

Z.D. Deng , , , Y. Zhang . . Collective behavior of a small-world recurrent neural system with scale-free distribution . . IEEE Trans. Neur. Networks , , 2007 . . 18 ( ( 5 ): ): 1364 - - 1375 . . DOI: 10.1109/TNN.2007.894082 http://doi.org/10.1109/TNN.2007.894082 . .

M. Henon . . A 2-D mapping with a strange attractor . . Commun. Math. Phys. , , 1976 . . 50 ( ( 1 ): ): 69 - - 77 . . DOI: 10.1007/BF01608556 http://doi.org/10.1007/BF01608556 . .

S. Hochreiter , , , Y. Bengio , , , P. Frasconi , , , J. Schmidhuber . . Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies . . In: Kolen J., Kremer, S. (Eds.), A Field Guide to Dynamical Recurrent Networks. Wiley-IEEE Press, New York , , 2001 . . p.237 - - 243 . . DOI: 10.1109/9780470544037.ch14 http://doi.org/10.1109/9780470544037.ch14 . .

G. Holzmann , , , H. Hauser . . Echo state networks with filter neurons and a delay & sum readout . . Neur. Networks , , 2010 . . 23 ( ( 2 ): ): 244 - - 256 . . DOI: 10.1016/j.neunet.2009.07.004 http://doi.org/10.1016/j.neunet.2009.07.004 . .

K. Ikeda , , , H. Daido , , , O. Akimoto . . Optical turbulence: chaotic behavior of transmitted light from a ring cavity . . Phys. Rev. Lett. , , 1980 . . 45 ( ( 9 ): ): 709 - - 712 . . DOI: 10.1103/PhysRevLett.45.709 http://doi.org/10.1103/PhysRevLett.45.709 . .

H. Jaeger . . The 'Echo State' Approach to Analysing and Training Recurrent Neural Networks . . Echo State' Approach to Analysing and Training Recurrent Neural Networks. Technical Report No. 148, German National Research Center for Information Technology, Bremen, Germany , , 2001 . . .

H. Jaeger . . A Tutorial on Training Recurrent Neural Networks, Covering BPPT, RTRL, EKF and the 'Echo State Network' Approach . . Technical Report No. GMD 159, German National Research Center for Information Technology, Sankt Augustin, Germany , , 2002a . . .

H. Jaeger . . Short Term Memory in Echo State Networks . . Technical Report No. GMD 152, German National Research Center for Information Technology, Sankt Augustin, Germany , , 2002b . . .

H. Jaeger . . Adaptive Nonlinear System Identification with Echo State Network . . Advances in Neural Information Processing Systems , , 2002c . . 15 593 - - 600 . . . .

H. Jaeger , , , H. Hass . . Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication . . Science , , 2004 . . 304 ( ( 5667 ): ): 78 - - 80 . . DOI: 10.1126/science.1091277 http://doi.org/10.1126/science.1091277 . .

M. Lukosevicius , , , H. Jaeger . . Reservoir computing approaches to recurrent neural network training . . Comput. Sci. Rev. , , 2009 . . 3 ( ( 3 ): ): 127 - - 149 . . DOI: 10.1016/j.cosrev.2009.03.005 http://doi.org/10.1016/j.cosrev.2009.03.005 . .

D.P. Mandic , , , J.A. Chambers . . Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability . . Wiley, New York , , 2001 . . p.31 - - 47 . . DOI: 10.1002/047084535X.ch3 http://doi.org/10.1002/047084535X.ch3 . .

National Geophysical Data Center . . Sunspot Numbers . . Available from , , http://www.ngdc.noaa.gov/stp/iono/sunspot.html http://www.ngdc.noaa.gov/stp/iono/sunspot.html , , [Accessed on Sept. 23, 2011] 2007 . . .

M.C. Ozturk , , , J.C. Principe . . An associative memory readout for ESNs with applications to dynamical pattern recognition . . Neur. Networks , , 2007 . . 20 ( ( 3 ): ): 377 - - 390 . . DOI: 10.1016/j.neunet.2007.04.012 http://doi.org/10.1016/j.neunet.2007.04.012 . .

A. Rodan , , , P. Tino . . Minimum complexity echo state network . . IEEE Trans. Neur. Networks , , 2011 . . 22 ( ( 1 ): ): 131 - - 144 . . DOI: 10.1109/TNN.2010.2089641 http://doi.org/10.1109/TNN.2010.2089641 . .

M. Salmen , , , P.G. Ploger . . Echo State Networks Used for Motor Control . . Proc. IEEE Int. Conf. on Robotics and Automation , , 2005 . . p.1953 - - 1958 . . DOI: 10.1109/ROBOT.2005.1570399 http://doi.org/10.1109/ROBOT.2005.1570399 . .

F. Schwenker , , , A. Labib . . Echo State Networks and Neural Network Ensembles to Predict Sunspots Activity . . 17th European Symp. on Artificial Neural Networks , , 2009 . . p.379 - - 384 . . . .

Z.W. Shi , , , M. Han . . Support vector echo-state machine for chaotic time-series prediction . . IEEE Trans. Neur. Networks , , 2007 . . 18 ( ( 2 ): ): 359 - - 372 . . DOI: 10.1109/TNN.2006.885113 http://doi.org/10.1109/TNN.2006.885113 . .

H.T. Siegelmann , , , E.D. Sontag . . Turing computability with neural nets . . Appl. Math. Lett. , , 1991 . . 4 ( ( 6 ): ): 77 - - 80 . . DOI: 10.1016/0893-9659(91)90080-F http://doi.org/10.1016/0893-9659(91)90080-F . .

J.J. Steil . . Memory in backpropagation-decorrelation O ( N ) efficient online recurrent learning . . LNCS , , 2005 . . 3697 649 - - 654 . . DOI: 10.1007/11550907_103 http://doi.org/10.1007/11550907_103 . .

P. Tino , , , C. Schittenkopf , , , G. Dorffner . . Financial volatility trading using recurrent neural networks . . IEEE Trans. Neur. Networks , , 2001 . . 12 ( ( 4 ): ): 865 - - 874 . . DOI: 10.1109/72.935096 http://doi.org/10.1109/72.935096 . .

F. Wyffels , , , B. Schrauwen , , , D. Stroobandt . . Stable output feedback in reservoir computing using ridge regression . . LNCS , , 2008 . . 5163 808 - - 817 . . DOI: 10.1007/978-3-540-87536-9_83 http://doi.org/10.1007/978-3-540-87536-9_83 . .

Y.L. Xia , , , B. Jelfs , , , M.M. van Hulle , , , J.C. Principe , , , D.P. Mandic . . An augmented echo state network for nonlinear adaptive filtering of complex noncircular signals . . IEEE Trans. Neur. Networks , , 2011 . . 22 ( ( 1 ): ): 74 - - 83 . . DOI: 10.1109/TNN.2010.2085444 http://doi.org/10.1109/TNN.2010.2085444 . .

Y.B. Xue , , , L. Yang , , , S. Haykin . . Decoupled echo state networks with lateral inhibition . . Neur. Networks , , 2007 . . 20 ( ( 3 ): ): 365 - - 376 . . DOI: 10.1016/j.neunet.2007.04.014 http://doi.org/10.1016/j.neunet.2007.04.014 . .

B. Zhang , , , D.J. Miller , , , Y. Wang . . Nonlinear system modeling with random matrices: echo state networks revisited . . IEEE Trans. Neur. Networks Learn. Syst. , , 2012 . . 23 ( ( 1 ): ): 175 - - 182 . . DOI: 10.1109/TNNLS.2011.2178562 http://doi.org/10.1109/TNNLS.2011.2178562 . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

暂无数据