A novel spiking neural network of receptive field encoding with groups of neurons decision

Yong-qiang MA; Zi-ru WANG; Si-yu YU; Ba-dong CHEN; Nan-ning ZHENG; Peng-ju REN

doi:10.1631/FITEE.1700714

Your Location：

Home >

Browse articles >

A novel spiking neural network of receptive field encoding with groups of neurons decision

2018 Special Issue on Artificial Intelligence 2.0: Theories and Applications | Updated：2022-05-19

- A novel spiking neural network of receptive field encoding with groups of neurons decision
- 基于感受野编码的多神经元决策脉冲神经网络
- Frontiers of Information Technology & Electronic Engineering Vol. 19, Issue 1, Pages: 139-150(2018)
- Affiliations：
  
  Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University, Xi'an 710049, China
- Author bio：
  
  [ "Yong-qiang MA, E-mail: musaqiang@stu.xjtu.edu.cn" ]
  [ "Nan-ning ZHENG, E-mail: nnzheng@mail.xjtu.edu.cn" ]
  Peng-ju REN, E-mail: pengjuren@mail.xjtu.edu.cn
- Funds：
- DOI：10.1631/FITEE.1700714
  CLC： TP183
- Received：31 October 2017，
  
  Revised：2018-;1-25，
  
  Published：2018-01
- Accepted：
Scan QR Code
Yong-qiang MA, Zi-ru WANG, Si-yu YU, et al. A novel spiking neural network of receptive field encoding with groups of neurons decision[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 139-150.
DOI：

Yong-qiang MA, Zi-ru WANG, Si-yu YU, et al. A novel spiking neural network of receptive field encoding with groups of neurons decision[J]. Frontiers of Information Technology & Electronic Engineering, 2018, 19(1): 139-150. DOI： 10.1631/FITEE.1700714.

摘要

人类对信息的处理主要依赖数十亿个神经元构成的复杂神经网络，信息传输通过神经元释放电脉冲信号实现。本文提出一个名为MD-SNN的脉冲神经网络模型，其具有以下3个主要特点：（1）使用感受野模型对图片编码，产生相应脉冲序列；（2）随机选取脉冲序列中部分脉冲作为每个神经元的输入信号，并以这种方式模拟生物神经元的绝对不应期;（3）使用多组神经元对输出结果作出共同决策。我们在手写数字数据集（MNIST）上对MD-SNN进行测试，结果表明：（1）不同大小感受野对图像分类结果有显著影响；（2）由于MD-SNN模型引入了生物神经元绝对不应期特征，同时增加的学习层神经元极大缩短了训练时间，因此有效降低了过拟合概率，与引入绝对不应期与增加学习层神经元的SNN模型相比，图像分类准确率提高了8.77％；（3）与其他SNN方法相比，MD-SNN对图像分类更加有效--与卷积神经网络（CNN）相比，MD-SNN在图像发生翻转或旋转时仍能保持有效分类（测试集上的分类精度可以保持在90.44％），同时更适合小样本学习（1000个训练样本的分类准确率可以达到80.15％，即CNN的7.8倍）。

Abstract

Human information processing depends mainly on billions of neurons which constitute a complex neural network

and the information is transmitted in the form of neural spikes. In this paper

we propose a spiking neural network (SNN)

named MD-SNN

with three key features: (1) using receptive field to encode spike trains from images; (2) randomly selecting partial spikes as inputs for each neuron to approach the absolute refractory period of the neuron; (3) using groups of neurons to make decisions. We test MD-SNN on the MNIST data set of handwritten digits

and results demonstrate that: (1) Different sizes of receptive fields influence classification results significantly. (2) Considering the neuronal refractory period in the SNN model

increasing the number of neurons in the learning layer could greatly reduce the training time

effectively reduce the probability of over-fitting

and improve the accuracy by 8.77%. (3) Compared with other SNN methods

MD-SNN achieves a better classification; compared with the convolution neural network

MD-SNN maintains flip and rotation invariance (the accuracy can remain at 90.44% on the test set)

and it is more suitable for small sample learning (the accuracy can reach 80.15% for 1000 training samples

which is 7.8 times that of CNN).

关键词

Keywords

references

SS Al-Amri , , , NV Kalyankar , , , SD Khamitkar . . Image segmentation by using edge detection . . Int J Comput Sci Eng , , 2010 . . 2 ( ( 3 ): ): 804 - - 807 . . . .

MJII Berry , , , M Meister . . Refractoriness and neural precision . . Proc Conf on Advances in Neural Information Processing Systems 10 , , 1998 . . p.110 - - 116 . . . .

GQ Bi , , , MM Poo . . Synaptic modification by correlated activity: Hebb's postulate revisited . . Ann Rev Neurosci , , 2001 . . 24 ( ( 1 ): ): 139 - - 166 . . DOI: 10.1146/annurev.neuro.24.1.139 http://doi.org/10.1146/annurev.neuro.24.1.139 . .

R Brette , , , W Gerstner . . Adaptive exponential integrateand-fire model as an effective description of neuronal activity . . J Neurophysiol , , 2005 . . 94 ( ( 5 ): ): 3637 - - 3642 . . DOI: 10.1152/jn.00686.2005 http://doi.org/10.1152/jn.00686.2005 . .

P Burt , , , E Adelson . . The laplacian pyramid as a compact image code . . IEEE Trans Commun , , 1983 . . 31 ( ( 4 ): ): 532 - - 540 . . DOI: 10.1109/TCOM.1983.1095851 http://doi.org/10.1109/TCOM.1983.1095851 . .

J Canny . . A computational approach to edge detection . . IEEE Trans Patt Anal Mach Intell , , 1986 . . 8 ( ( 6 ): ): 679 - - 698 . . DOI: 10.1109/TPAMI.1986.4767851 http://doi.org/10.1109/TPAMI.1986.4767851 . .

A Coates , , , A Ng , , , H Lee . . An analysis of single-layer networks in unsupervised feature learning . . Proc 14th Int Conf on Artificial Intelligence and Statistics , , 2011 . . p.215 - - 223 . . . .

S Dasgupta , , , CF Stevens , , , S Navlakha . . A neural algorithm for a fundamental computing problem . . Science , , 2017 . . 358 ( ( 6364 ): ): 793 - - 796 . . DOI: 10.1126/science.aam9868 http://doi.org/10.1126/science.aam9868 . .

S Dora , , , S Suresh , , , N Sundararajan . . A sequential learning algorithm for a spiking neural classifier . . Appl Soft Comput , , 2015a . . 36 255 - - 268 . . DOI: 10.1016/j.asoc.2015.06.062 http://doi.org/10.1016/j.asoc.2015.06.062 . .

S Dora , , , S Sundaram , , , N Sundararajan . . A two stage learning algorithm for a growing-pruning spiking neural network for pattern classification problems . . Int Joint Conf on Neural Networks , , 2015b . . p.1 - - 7 . . DOI: 10.1109/ijcnn.2015.7280592 http://doi.org/10.1109/ijcnn.2015.7280592 . .

S Dora , , , K Subramanian , , , S Suresh , , , 等 . . Development of a self-regulating evolving spiking neural network for classification problem . . Neurocomputing , , 2016 . . 171 1216 - - 1229 . . DOI: 10.1016/j.neucom.2015.07.086 http://doi.org/10.1016/j.neucom.2015.07.086 . .

S Dora , , , S Suresh , , , N Sundararajan . . Online meta-neuron based learning algorithm for a spiking neural classifier . . Inform Sci , , 2017 . . 414 19 - - 32 . . DOI: 10.1016/j.ins.2017.05.050 http://doi.org/10.1016/j.ins.2017.05.050 . .

K Fukushima . . Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position . . Biol Cybern , , 1980 . . 36 ( ( 4 ): ): 193 - - 202 . . DOI: 10.1007/bf00344251 http://doi.org/10.1007/bf00344251 . .

W Gerstner , , , W Kistler . . Spiking Neuron Models: Single Neurons, Populations, Plasticity , , : : Cambridge, UK Cambridge University Press , , 2002 . . DOI: 10.1017/CBO9780511815706 http://doi.org/10.1017/CBO9780511815706 . .

S Ghosh Dastidar , , , H Adeli . . Improved spiking neural networks for eeg classification and epilepsy and seizure detection . . Integr Comput Aided Eng , , 2007 . . 14 ( ( 3 ): ): 187 - - 212 . . . .

CD Gilbert , , , TN Wiesel . . Receptive field dynamics in adult primary visual cortex . . Nature , , 1992 . . 356 ( ( 6365 ): ): 150 - - 152 . . DOI: 10.1038/356150a0 http://doi.org/10.1038/356150a0 . .

R Gütig , , , H Sompolinsky . . The tempotron: a neuron that learns spike timing-based decisions . . Nat Neurosci , , 2006 . . 9 ( ( 3 ): ): 420 - - 428 . . DOI: 10.1038/nn1643 http://doi.org/10.1038/nn1643 . .

AY Hannun , , , C Case , , , J Casper , , , 等 . . Deep speech: Scaling up end-to-end speech recognition . . 2014 . . https://arxiv.org/abs/1412.5567 https://arxiv.org/abs/1412.5567 , , . .

KM He , , , XY Zhang , , , SQ Ren , , , 等 . . Deep residual learning for image recognition . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.770 - - 778 . . DOI: 10.1109/CVPR.2016.90 http://doi.org/10.1109/CVPR.2016.90 . .

AL Hodgkin , , , AF Huxley . . A quantitative description of membrane current and its application to conduction and excitation in nerve . . J Physiol , , 1952 . . 117 ( ( 4 ): ): 500 - - 544 . . DOI: 10.1113/jphysiol.1952.sp004764 http://doi.org/10.1113/jphysiol.1952.sp004764 . .

DH Hubel , , , TN Wiesel . . Receptive fields, binocular interaction and functional architecture in the cat's visual cortex . . J Physiol , , 1962 . . 160 ( ( 1 ): ): 106 - - 154 . . DOI: 10.1113/jphysiol.1962.sp006837 http://doi.org/10.1113/jphysiol.1962.sp006837 . .

S Hussain , , , SC Liu , , , A Basu . . Improved margin multiclass classification using dendritic neurons with morphological learning . . IEEE Int Symp on Circuits and Systems , , 2014 . . p.2640 - - 2643 . . DOI: 10.1109/iscas.2014.6865715 http://doi.org/10.1109/iscas.2014.6865715 . .

EM Izhikevich . . Resonate-and-fire neurons . . Neur Networks , , 2001 . . 14 ( ( 6-7 ): ): 883 - - 894 . . DOI: 10.1016/s0893-6080(01)00078-8 http://doi.org/10.1016/s0893-6080(01)00078-8 . .

EM Izhikevich . . Simple model of spiking neurons . . IEEE Trans Neur Networks , , 2003 . . 14 ( ( 6 ): ): 1569 - - 1572 . . DOI: 10.1109/tnn.2003.820440 http://doi.org/10.1109/tnn.2003.820440 . .

EM Izhikevich . . Which model to use for cortical spiking neurons . . IEEE Trans Neur Networks , , 2004 . . 15 ( ( 5 ): ): 1063 - - 1070 . . DOI: 10.1109/tnn.2004.832719 http://doi.org/10.1109/tnn.2004.832719 . .

Y LeCun , , , Y Bengio , , , G Hinton . . Deep learning . . Nature , , 2015 . . 521 ( ( 7553 ): ): 436 - - 444 . . DOI: 10.1038/nature14539 http://doi.org/10.1038/nature14539 . .

R Legenstein , , , C Naeger , , , W Maass . . What can a neuron learn with spike-timing-dependent plasticity . . Neur Comput , , 2006 . . 17 ( ( 11 ): ): 2337 - - 2382 . . DOI: 10.1162/0899766054796888 http://doi.org/10.1162/0899766054796888 . .

YQ Ma , , , H Wu , , , MJ Zhu , , , 等 . . Reconstruction of visual image from functional magnetic resonance imaging using spiking neuron model . . IEEE Trans Cogn Dev Syst , , 2017 . . in press DOI: 10.1109/tcds.2017.2764948 http://doi.org/10.1109/tcds.2017.2764948 . .

W Maass . . Networks of spiking neurons: the third generation of neural network models . . Neur Networks , , 1997 . . 10 ( ( 9 ): ): 1659 - - 1671 . . DOI: 10.1016/s0893-6080(97)00011-7 http://doi.org/10.1016/s0893-6080(97)00011-7 . .

T Masquelier , , , R Guyonneau , , , SJ Thorpe . . Competitive stdp-based spike pattern learning . . Neur Comput , , 2009 . . 21 ( ( 5 ): ): 1259 - - 1276 . . DOI: 10.1162/neco.2008.06-08-804 http://doi.org/10.1162/neco.2008.06-08-804 . .

P Merolla , , , J Arthur , , , F Akopyan , , , 等 . . A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm . . IEEE Custom Integrated Circuits Conf , , 2011 . . p.1 - - 4 . . DOI: 10.1109/cicc.2011.6055294 http://doi.org/10.1109/cicc.2011.6055294 . .

F Ponulak , , , A Kasiński . . Supervised learning in spiking neural networks with resume: sequence learning, classification, and spike shifting . . Neur Comput , , 2010 . . 22 ( ( 2 ): ): 467 - - 510 . . DOI: 10.1162/neco.2009.11-08-901 http://doi.org/10.1162/neco.2009.11-08-901 . .

RW Rodieck . . Quantitative analysis of cat retinal ganglion cell response to visual stimuli . . Vis Res , , 1965 . . 5 ( ( 11 ): ): 583 - - 601 . . DOI: 10.1016/0042-6989(65)90033-7 http://doi.org/10.1016/0042-6989(65)90033-7 . .

J Schmidhuber . . Deep learning in neural networks: An overview . . Neur networks , , 2015 . . 61 85 - - 117 . . DOI: 10.1016/j.neunet.2014.09.003 http://doi.org/10.1016/j.neunet.2014.09.003 . .

I Sobel . . History and definition of the sobel operator . . 2014 . . https://www.scribd.com/document/271811982/Historyand-Definition-of-Sobel-Operator https://www.scribd.com/document/271811982/Historyand-Definition-of-Sobel-Operator , , . .

H Tang , , , Q Yu , , , KC Tan . . Learning real-world stimuli by single-spike coding and tempotron rule . . Int Joint Conf on Neural Networks , , 2012 . . p.1 - - 6 . . DOI: 10.1109/ijcnn.2012.6252369 http://doi.org/10.1109/ijcnn.2012.6252369 . .

A Tavanaei , , , AS Maida . . A minimal spiking neural network to rapidly train and classify handwritten digits in binary and 10-digit tasks . . Int J Adv Res Artif Intell , , 2015 . . 4 ( ( 7 ): ): 1 - - 8 . . DOI: 10.14569/ijarai.2015.040701 http://doi.org/10.14569/ijarai.2015.040701 . .

S Thorpe , , , A Delorme , , , R van Rullen . . Spike-based strategies for rapid processing . . Neur Netw , , 2001 . . 14 ( ( 67 ): ): 715 - - 725 . . DOI: 10.1016/s0893-6080(01)00083-1 http://doi.org/10.1016/s0893-6080(01)00083-1 . .

JD Victor , , , KP Purpura . . Nature and precision of temporal coding in visual cortex: a metric-space analysis . . J Neurophysiol , , 1996 . . 76 ( ( 2 ): ): 1310 - - 1326 . . DOI: 10.1152/jn.1996.76.2.1310 http://doi.org/10.1152/jn.1996.76.2.1310 . .

JJ Wade , , , LJ Mcdaid , , , JA Santos , , , 等 . . SWAT: a spiking neural network training algorithm for classification problems . . IEEE Trans Neur Networks , , 2010 . . 21 ( ( 11 ): ): 1817 - - 1830 . . DOI: 10.1109/TNN.2010.2074212 http://doi.org/10.1109/TNN.2010.2074212 . .

XR Xie , , , H Qu , , , Z Yi , , , 等 . . Efficient training of supervised spiking neural network via accurate synapticefficiency adjustment method . . IEEE Trans Neur Networks , , 2017 . . 28 ( ( 6 ): ): 1411 - - 1424 . . DOI: 10.1109/tnnls.2016.2541339 http://doi.org/10.1109/tnnls.2016.2541339 . .

JS Yeomans . . The absolute refractory periods of selfstimulation neurons . . Phys Behav , , 1979 . . 22 ( ( 5 ): ): 911 - - 919 . . DOI: 10.1016/0031-9384(79)90336-6 http://doi.org/10.1016/0031-9384(79)90336-6 . .

Q Yu , , , HJ Tang , , , KC Tan , , , 等 . . Rapid feedforward computation by temporal encoding and learning with spiking neurons . . IEEE Trans Neur Networks , , 2013 . . 24 ( ( 10 ): ): 1539 - - 1552 . . DOI: 10.1109/TNNLS.2013.2245677 http://doi.org/10.1109/TNNLS.2013.2245677 . .

Q Yu , , , HJ Tang , , , KC Tan , , , 等 . . A brain-inspired spiking neural network model with temporal encoding and learning . . Neurocomputing , , 2014 . . 138 3 - - 13 . . DOI: 10.1016/j.neucom.2013.06.052 http://doi.org/10.1016/j.neucom.2013.06.052 . .

F Zenke , , , S Ganguli . . Superspike: supervised learning in multi-layer spiking neural networks . . 2017 . . https://arxiv.org/abs/1705.11146 https://arxiv.org/abs/1705.11146 , , . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰