S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion

Li CHEN; Fan ZHANG; Guangwei XIE; Yanzhao GAO; Xiaofeng QI; Mingqian SUN

doi:10.1631/FITEE.2400594

Your Location：

Home >

Browse articles >

S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion

Regular Papers | Updated：2025-06-09

- S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion
- S3Det：一种基于人工—脉冲神经网络转换的遥感影像目标快速检测模型
- Frontiers of Information Technology & Electronic Engineering Vol. 26, Issue 5, Pages: 713-727(2025)
- Affiliations：
  
  1.National Digital Switching System Engineering & Technological R&D Center, Zhengzhou 450003, China
  2.Computation and Artificial Intelligence Innovative College, Fudan University, Shanghai 201203, China
  3.School of Cyber Science and Engineering, Southeast University, Nanjing 211189, China
- Author bio：
  
  ‡ Corresponding authors
- Funds：
- DOI：10.1631/FITEE.2400594
  CLC： TP391
- Received：14 July 2024，
  
  Revised：11 October 2024，
  
  Published：2025-05
- Accepted：
Scan QR Code
Li CHEN, Fan ZHANG, Guangwei XIE, et al. S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion[J]. Frontiers of information technology & electronic engineering, 2025, 26(5): 713-727.
DOI：

Li CHEN, Fan ZHANG, Guangwei XIE, et al. S3Det: a fast object detector for remote sensing images based on artificial to spiking neural network conversion[J]. Frontiers of information technology & electronic engineering, 2025, 26(5): 713-727. DOI： 10.1631/FITEE.2400594.

摘要

人工神经网络（ANN）在遥感影像目标检测方面取得显著进展。然而，低检测效率和高能耗一直是遥感领域的重要瓶颈。脉冲神经网络（SNN）以稀疏脉冲的形式处理信息，为计算机视觉任务带来高效能优势。不过，大部分研究工作集中在简单分类任务上，仅有少数研究者将其应用于自然图像的目标检测。本文考虑到生物大脑的简约特性，提出一种人工—脉冲神经网络快速转换方法，用于遥感影像检测。基于群组稀疏特征建立快速稀疏模型进行脉冲序列感知，并对原始图像进行变换域内的稀疏重采样，从而快速感知图像特征和编码的脉冲序列。此外，为满足相关遥感场景中的精度要求，从理论上分析了转换误差，提出通道自衰减加权归一化方法，以消除神经元过度激活。所提遥感影像目标检测模型被称作S3Det。基于一个大型公开遥感数据集的实验表明，S3Det实现了与ANN相似的精度。同时，我们的转换网络稀疏度为原始算法的24.32%；能耗仅为1.46 W，是原始算法的1/122。

Abstract

Artificial neural networks (ANNs) have made great strides in the field of remote sensing image object detection. However

low detection efficiency and high power consumption have always been significant bottlenecks in remote sensing. Spiking neural networks (SNNs) process information in the form of sparse spikes

creating the advantage of high energy efficiency for computer vision tasks. However

most studies have focused on simple classification tasks

and only a few researchers have applied SNNs to object detection in natural images. In this study

we consider the parsimonious nature of biological brains and propose a fast ANN-to-SNN conversion method for remote sensing image detection. We establish a fast sparse model for pulse sequence perception based on group sparse features and conduct transform-domain sparse resampling of the original images to enable fast perception of image features and encoded pulse sequences. In addition

to meet accuracy requirements in relevant remote sensing scenarios

we theoretically analyze the transformation error and propose channel self-decaying weighted normalization (CSWN) to eliminate neuron overactivation. We propose S3Det

a remote sensing image object detection model. Our experiments

based on a large publicly available remote sensing dataset

show that S3Det achieves an accuracy performance similar to that of the ANN. Meanwhile

our transformed network is only 24.32% as sparse as the benchmark and consumes only 1.46 W

which is 1/122 of the original algorithm’s power consumption.

关键词

Keywords

references

Azimi SM , Vig E , Bahmanyar R , et al. , 2018 . Towards multi-class object detection in unconstrained remote sensing imagery . Proc 14 th Asian Conf on Computer Vision , p.‍ 150 - 165 . https://doi.org/10.1007/978-3-030-20893-6_10 https://doi.org/10.1007/978-3-030-20893-6_10

Chen GH , Pei GS , Tang Y , et al. , 2022 . A novel multi-sample data augmentation method for oriented object detection in remote sensing images . Proc IEEE 24 th Int Workshop on Multimedia Signal Processing , p. 1 - 7 . https://doi.org/10.1109/MMSP55362.2022.9949615 https://doi.org/10.1109/MMSP55362.2022.9949615

Chen K , Wang JQ , Pang JM , et al. , 2019 . MMDetection: open MMlab detection toolbox and benchmark . https://arxiv.org/abs/1906.07155 https://arxiv.org/abs/1906.07155

Chen L , Zhang F , Guo W , et al. , 2023 . SFTN: fast object detection for aerial images . IET Image Process , 17 ( 13 ): 3897 - 3907 . https://doi.org/10.1049/ipr2.12906 https://doi.org/10.1049/ipr2.12906

Cheng G , Zhou PC , Han JW , et al. , 2016a . Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images . IEEE Trans Geosci Remote Sens , 54 ( 12 ): 7405 - 7415 . https://doi.org/10.1109/TGRS.2016.2601622 https://doi.org/10.1109/TGRS.2016.2601622

Cheng G , Zhou PC , Han JW , 2016b . RIFD-CNN: rotation-invariant and Fisher discriminative convolutional neural networks for object detection . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 2884 - 2893 . https://doi.org/10.1109/CVPR.2016.315 https://doi.org/10.1109/CVPR.2016.315

Ding J , Xue N , Long Y , et al. , 2019 . Learning RoI Transformer for oriented object detection in aerial images . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 2844 - 2853 . https://doi.org/10.1109/CVPR.2019.00296 https://doi.org/10.1109/CVPR.2019.00296

Eldar YC , Kutyniok G , 2012 . Compressed Sensing: Theory and Applications . Cambridge University Press , Cambridge, UK . https://doi.org/10.1017/CBO9780511794308 https://doi.org/10.1017/CBO9780511794308

Everingham M , van Gool L , Williams CK , et al. , 2010 . The PASCAL Visual Object Classes (VOC) challenge . Int J Comput Vis , 88 ( 2 ): 303 - 338 . https://doi.org/10.1007/s11263-009-0275-4 https://doi.org/10.1007/s11263-009-0275-4

Gong MG , Li JZ , Zhang YR , et al. , 2022 . Two-path aggregation attention network with quad-patch data augmentation for few-shot scene classification . IEEE Trans Geosci Remote Sens , 60 : 4511616 . https://doi.org/10.1109/TGRS.2022.3197445 https://doi.org/10.1109/TGRS.2022.3197445

Han JM , Ding J , Li J , et al. , 2022 . Align deep features for oriented object detection . IEEE Trans Geosci Remote Sens , 60 : 5602511 . https://doi.org/10.1109/TGRS.2021.3062048 https://doi.org/10.1109/TGRS.2021.3062048

He X , Ma SP , He LY , et al. , 2022 . High-resolution polar network for object detection in remote sensing images . IEEE Geosci Remote Sens Lett , 19 : 6000605 . https://doi.org/10.1109/LGRS.2020.3039240 https://doi.org/10.1109/LGRS.2020.3039240

Horowitz M , 2014 . 1.1 Computing's energy problem (and what we can do about it) . Proc IEEE Int Solid-State Circuits Conf Digest of Technical Papers , p. 10 - 14 . https://doi.org/10.1109/ISSCC.2014.6757323 https://doi.org/10.1109/ISSCC.2014.6757323

Hu YF , Zheng Q , Jiang XD , et al. , 2023 . Fast-SNN: fast spiking neural network by converting quantized ANN . IEEE Trans Patt Anal Mach Intell , 45 ( 12 ): 14546 - 14562 . https://doi.org/10.1109/TPAMI.2023.3275769 https://doi.org/10.1109/TPAMI.2023.3275769

Huang ZC , Li W , Xia XG , et al. , 2022 . A general Gaussian heatmap label assignment for arbitrary-oriented object detection . IEEE Trans Image Process , 31 : 1895 - 1910 . https://doi.org/10.1109/TIP.2022.3148874 https://doi.org/10.1109/TIP.2022.3148874

Jiang YQ , Tan ZY , Wang JY , et al. , 2022 . GiraffeDet: a heavy-neck paradigm for object detection . https://arxiv.org/abs/2202.04256 https://arxiv.org/abs/2202.04256

Jiang YY , Zhu XY , Wang XB , et al. , 2018 . R 2 CNN: rotational region CNN for arbitrarily-oriented scene text detection . Proc 24 th Int Conf on Pattern Recognition , p. 3610 - 3615 . https://doi.org/10.1109/ICPR.2018.8545598 https://doi.org/10.1109/ICPR.2018.8545598

Kim S , Park S , Na B , et al. , 2020 . Spiking-YOLO: spiking neural network for energy-efficient object detection . Proc 34 th AAAI Conf on Artificial Intelligence , p. 11270 - 11277 . https://doi.org/10.1609/aaai.v34i07.6787 https://doi.org/10.1609/aaai.v34i07.6787

Komárek A , Lesaffre E , 2008 . Generalized linear mixed model with a penalized Gaussian mixture as a random effects distribution . Comput Stat Data Anal , 52 ( 7 ): 3441 - 3458 . https://doi.org/10.1016/j.csda.2007.10.024 https://doi.org/10.1016/j.csda.2007.10.024

LeCun Y , Bottou L , Bengio Y , et al. , 1998 . Gradient-based learning applied to document recognition . Proc IEEE , 86 ( 11 ): 2278 - 2324 . https://doi.org/10.1109/5.726791 https://doi.org/10.1109/5.726791

Li Y , He X , Dong YT , et al. , 2022 . Spike calibration: fast and accurate conversion of spiking neural network for object detection and segmentation . https://arxiv.org/abs/2207.02702 https://arxiv.org/abs/2207.02702

Lin TY , Maire M , Belongie S , et al. , 2014 . Microsoft COCO: common objects in context . Proc 13 th European Conf on Computer Vision , p. 740 - 755 . https://doi.org/10.1007/978-3-319-10602-1_48 https://doi.org/10.1007/978-3-319-10602-1_48

Lin TY , Goyal P , Girshick R , et al. , 2017 . Focal loss for dense object detection . Proc IEEE Int Conf on Computer Vision , p. 2999 - 3007 . https://doi.org/10.1109/ICCV.2017.324 https://doi.org/10.1109/ICCV.2017.324

Liu WX , Luo B , Liu J , et al. , 2022 . Synthetic data augmentation using multiscale attention CycleGAN for aircraft detection in remote sensing images . IEEE Geosci Remote Sens Lett , 19 : 4009205 . https://doi.org/10.1109/LGRS.2021.3052017 https://doi.org/10.1109/LGRS.2021.3052017

Liu ZK , Wang HZ , Weng LB , et al. , 2016 . Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds . IEEE Geosci Remote Sens Lett , 13 ( 8 ): 1074 - 1078 . https://doi.org/10.1109/LGRS.2016.2565705 https://doi.org/10.1109/LGRS.2016.2565705

Ma JQ , Shao WY , Ye H , et al. , 2018 . Arbitrary-oriented scene text detection via rotation proposals . IEEE Trans Multim , 20 ( 11 ): 3111 - 3122 . https://doi.org/10.1109/TMM.2018.2818020 https://doi.org/10.1109/TMM.2018.2818020

Maass W , 1997 . Networks of spiking neurons: the third generation of neural network models . Neur Netw , 10 ( 9 ):‍ 1659 - 1671 . https://doi.org/10.1016/S0893-6080(97)00011-7 https://doi.org/10.1016/S0893-6080(97)00011-7

Merolla PA , Arthur JV , Alvarez-Icaza R , et al. , 2014 . A million spiking-neuron integrated circuit with a scalable communication network and interface . Science , 345 ( 6197 ):‍ 668 - 673 . https://doi.org/10.1126/science.1254642 https://doi.org/10.1126/science.1254642

Ming Q , Zhou ZQ , Miao LJ , et al. , 2021 . Dynamic anchor learning for arbitrary-oriented object detection . Proc 35 th AAAI Conf on Artificial Intelligence, Electronic Network , p. 2355 - 2363 . https://doi.org/10.1609/aaai.v35i3.16336 https://doi.org/10.1609/aaai.v35i3.16336

Rathi N , Roy K , 2023 . DIET-SNN: a low-latency spiking neural network with direct input encoding and leakage and threshold optimization . IEEE Trans Neur Netw Learn Syst , 34 ( 6 ): 3174 - 3182 . https://doi.org/10.1109/TNNLS.2021.3111897 https://doi.org/10.1109/TNNLS.2021.3111897

Rueckauer B , Liu SC , 2021 . Temporal pattern coding in deep spiking neural networks . Proc Int Joint Conf on Neural Networks , p. 1 - 8 . https://doi.org/10.1109/IJCNN52387.2021.9533837 https://doi.org/10.1109/IJCNN52387.2021.9533837

Rueckauer B , Lungu IA , Hu YH , et al. , 2017 . Conversion of continuous-valued deep networks to efficient event-driven networks for image classification . Front Neurosci , 11 : 682 . https://doi.org/10.3389/fnins.2017.00682 https://doi.org/10.3389/fnins.2017.00682

Sinha D , El-Sharkawy M , 2019 . Thin MobileNet: an enhanced MobileNet architecture . Proc IEEE 10 th Annual Ubiquitous Computing, Electronics & Mobile Communication Conf , p. 280 - 285 . https://doi.org/10.1109/UEMCON47517.2019.8993089 https://doi.org/10.1109/UEMCON47517.2019.8993089

Vaswani A , Shazeer N , Parmar N , et al. , 2017 . Attention is all you need . Proc 31 st Int Conf on Neural Information Processing Systems , p. 6000 - 6010 .

Wang A , Chen H , Liu LH , et al. , 2024 . YOLOv10: real-time end-to-end object detection . https://arxiv.org/abs/2405.14458 https://arxiv.org/abs/2405.14458

Xia GS , Bai X , Ding J , et al. , 2018 . DOTA: a large-scale dataset for object detection in aerial images . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 3974 - 3983 . https://doi.org/10.1109/CVPR.2018.00418 https://doi.org/10.1109/CVPR.2018.00418

Xie XX , Lang CB , Miao SC , et al. , 2023 . Mutual-assistance learning for object detection . IEEE Trans Patt Anal Mach Intell , 45 ( 12 ): 15171 - 15184 . https://doi.org/10.1109/TPAMI.2023.3319634 https://doi.org/10.1109/TPAMI.2023.3319634

Xie XX , Cheng G , Li QY , et al. , 2024a . Fewer is more: efficient object detection in large aerial images . Sci China Inform Sci , 67 ( 1 ): 112106 . https://doi.org/10.1007/s11432-022-3718-5 https://doi.org/10.1007/s11432-022-3718-5

Xie XX , Cheng G , Rao CF , et al. , 2024b . Oriented object detection via contextual dependence mining and penalty-incentive allocation . IEEE Trans Geosci Remote Sens , 62 : 5618010 . https://doi.org/10.1109/TGRS.2024.3385985 https://doi.org/10.1109/TGRS.2024.3385985

Xie XX , Cheng G , Wang JB , et al. , 2024c . Oriented R-CNN and beyond . Int J Comput Vis , 132 ( 7 ): 2420 - 2442 . https://doi.org/10.1007/s11263-024-01989-w https://doi.org/10.1007/s11263-024-01989-w

Xiong YY , Liu HX , Gupta S , et al. , 2021 . MobileDets: searching for object detection architectures for mobile accelerators . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 3824 - 3833 . https://doi.org/10.1109/CVPR46437.2021.00382 https://doi.org/10.1109/CVPR46437.2021.00382

Yang X , Yan JC , 2022 . On the arbitrary-oriented object detection: classification based approaches revisited . Int J Comput Vis , 130 ( 5 ): 1340 - 1365 . https://doi.org/10.1007/s11263-022-01593-w https://doi.org/10.1007/s11263-022-01593-w

Yang X , Yang JR , Yan JC , et al. , 2019 . SCRDet: towards more robust detection for small, cluttered and rotated objects . Proc IEEE/CVF Int Conf on Computer Vision , p. 8231 - 8240 . https://doi.org/10.1109/ICCV.2019.00832 https://doi.org/10.1109/ICCV.2019.00832

Yang X , Yan JC , Feng ZM , et al. , 2021a . R3Det: refined single-stage detector with feature refinement for rotating object . Proc 35 th AAAI Conf on Artificial Intelligence , p. 3163 - 3171 . https://doi.org/10.1609/aaai.v35i4.16426 https://doi.org/10.1609/aaai.v35i4.16426

Yang X , Yan JC , Ming Q , et al. , 2021b . Rethinking rotated object detection with Gaussian Wasserstein distance loss . Proc 38 th Int Conf on Machine Learning , p. 11830 - 11841 .

Yao M , Zhao GS , Zhang HY , et al. , 2023 . Attention spiking neural networks . IEEE Trans Patt Anal Mach Intell , 45 ( 8 ): 9393 - 9410 . https://doi.org/10.1109/TPAMI.2023.3241201 https://doi.org/10.1109/TPAMI.2023.3241201

Zhang C , Lam KM , Wang Q , 2023 . CoF-Net: a progressive coarse-to-fine framework for object detection in remote-sensing imagery . IEEE Trans Geosci Remote Sens , 61 : 5600617 . https://doi.org/10.1109/TGRS.2022.3233881 https://doi.org/10.1109/TGRS.2022.3233881

Zhang GJ , Lu SJ , Zhang W , 2019 . CAD-Net: a context-aware detection network for objects in remote sensing imagery . IEEE Trans Geosci Remote Sens , 57 ( 12 ): 10015 - 10024 . https://doi.org/10.1109/TGRS.2019.2930982 https://doi.org/10.1109/TGRS.2019.2930982

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

UI layers merger: merging UI layers via visual learning and boundary prior

A modified YOLOv4 detection method for a vision-based underwater garbage cleaning robot^&

Automatic analysis of deep-water remotely operated vehicle footage for estimation of Norway lobster abundance

Detection of engineering vehicles in high-resolution monitoring images

Related Author

Yanfang CHANG

Tingting ZHOU

Lingyun SUN

Zejian LI

Liuqing CHEN

Jiazhi LI

Chuning SHI

Yankun ZHEN

Related Institution

School of Software Technology, Zhejiang University

College of Computer Science and Technology, Zhejiang University

Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies

Alibaba Group

Singapore Innovation and AI Joint Research Lab

Map

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰