Underwater object detection by fusing features from different representations of sonar data

Fei WANG; Wanyu LI; Miao LIU; Jingchun ZHOU; Weishi ZHANG

doi:10.1631/FITEE.2200429

Your Location：

Home >

Browse articles >

Underwater object detection by fusing features from different representations of sonar data

Regular Papers | Updated：2023-07-03

- Underwater object detection by fusing features from different representations of sonar data
- 基于多表征声呐数据特征融合的水下目标检测方法
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 6, Pages: 828-843(2023)
- Affiliations：
  
  1.College of Information Science and Technology, Dalian Maritime University, Dalian 116026, China
  2.College of Transportation Engineering, Dalian Maritime University, Dalian 116026, China
- Author bio：
  
  †E-mail: feiwang@dlmu.edu.cn
  ‡ Corresponding author
  teesiv@dlmu.edu.cn
- Funds：
- DOI：10.1631/FITEE.2200429
  CLC： TN911.73; TP391.41
- Received：03 October 2022，
  
  Revised：2023-02-02，
  
  Published：0 June 2023
- Accepted：
Scan QR Code
Fei WANG, Wanyu LI, Miao LIU, et al. Underwater object detection by fusing features from different representations of sonar data[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(6): 828-843.
DOI：

Fei WANG, Wanyu LI, Miao LIU, et al. Underwater object detection by fusing features from different representations of sonar data[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(6): 828-843. DOI： 10.1631/FITEE.2200429.

摘要

现有水下目标检测方法多基于物体的几何形状从声呐数据中识别物体，这些方法几乎忽略数据采集和数据表征过程所产生的形状畸变问题。为此，本文对声呐数据的不同表示形式进行了对比分析，在此基础上，提出了一个特征融合框架，以充分利用从极坐标图像中提取的强度特征和从点云表示形式中学习的几何特征。该框架中设计了三种特征融合策略，以分析特征融合对检测器不同模块的影响。同时，这些融合策略可以直接集成到其他检测器中，如YOLO系列。通过公开水下实景声呐数据集上的一系列对比实验，验证了所提框架和特征融合策略的有效性。实验结果表明，所提特征融合方法对检测器中候选区域模块和分类模块的结果都有所增益。

Abstract

Modern underwater object detection methods recognize objects from sonar data based on their geometric shapes. However

the distortion of objects during data acquisition and representation is seldom considered. In this paper

we present a detailed summary of representations for sonar data and a concrete analysis of the geometric characteristics of different data representations. Based on this

a feature fusion framework is proposed to fully use the intensity features extracted from the polar image representation and the geometric features learned from the point cloud representation of sonar data. Three feature fusion strategies are presented to investigate the impact of feature fusion on different components of the detection pipeline. In addition

the fusion strategies can be easily integrated into other detectors

such as the You Only Look Once (YOLO) series. The effectiveness of our proposed framework and feature fusion strategies is demonstrated on a public sonar dataset captured in real-world underwater environments. Experimental results show that our method benefits both the region proposal and the object classification modules in the detectors.

关键词

Keywords

references

Ben Tamou A , Benzinou A , Nasreddine K , 2021 . Multi-stream fish detection in unconstrained underwater videos by the fusion of two convolutional neural network detectors . Appl Intell , 51 ( 8 ): 5809 - 5821 . https://doi.org/10.1007/s10489-020-02155-8 https://doi.org/10.1007/s10489-020-02155-8

Bochkovskiy A , Wang CY , Liao HYM , 2020 . YOLOv4: optimal speed and accuracy of object detection . https://arxiv.org/abs/2004.10934 https://arxiv.org/abs/2004.10934

Charles RQ , Su H , Mo KC , et al. , 2017 . PointNet: deep learning on point sets for 3D classification and segmentation . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 77 - 85 . https://doi.org/10.1109/CVPR.2017.16 https://doi.org/10.1109/CVPR.2017.16

Chen K , Wang JQ , Pang JM , et al. , 2019 . MMDetection: open MMLab detection toolbox and benchmark . https://arxiv.org/abs/1906.07155 https://arxiv.org/abs/1906.07155

Chen XL , Mu XQ , Guan J , et al. , 2022 . Marine target detection based on Marine-Faster R-CNN for navigation radar plane position indicator images . Front Inform Technol Electron Eng , 23 ( 4 ): 630 - 643 . https://doi.org/10.1631/FITEE.2000611 https://doi.org/10.1631/FITEE.2000611

Chen ZH , Yang CHY , Li QF , et al. , 2021 . Disentangle your dense object detector . Proc 29 th ACM Int Conf on Multimedia , p. 4939 - 4948 . https://doi.org/10.1145/3474085.3475351 https://doi.org/10.1145/3474085.3475351

Feng CJ , Zhong YJ , Gao Y , et al. , 2021 . TOOD: task-aligned one-stage object detection . Proc IEEE/CVF Int Conf on Computer Vision , p. 3490 - 3499 . https://doi.org/10.1109/ICCV48922.2021.00349 https://doi.org/10.1109/ICCV48922.2021.00349

Ge Z , Liu ST , Wang F , et al. , 2021 . YOLOX: exceeding YOLO series in 2021 . https://doi.org/10.48550/arXiv.2107.08430 https://doi.org/10.48550/arXiv.2107.08430

Ghiasi G , Lin TY , Le QV , 2019 . NAS-FPN: learning scalable feature pyramid architecture for object detection . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 7029 - 7038 . https://doi.org/10.1109/CVPR.2019.00720 https://doi.org/10.1109/CVPR.2019.00720

Girshick R , 2015 . Fast R-CNN . Proc IEEE Int Conf on Computer Vision , p. 1440 - 1448 . https://doi.org/10.1109/ICCV.2015.169 https://doi.org/10.1109/ICCV.2015.169

Girshick R , Donahue J , Darrell T , et al. , 2014 . Rich feature hierarchies for accurate object detection and semantic segmentation . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 580 - 587 . https://doi.org/10.1109/CVPR.2014.81 https://doi.org/10.1109/CVPR.2014.81

Girshick R , Donahue J , Darrell T , et al. , 2016 . Region-based convolutional networks for accurate object detection and segmentation . IEEE Trans Patt Anal Mach Intell , 38 ( 1 ): 142 - 158 . https://doi.org/10.1109/TPAMI.2015.2437384 https://doi.org/10.1109/TPAMI.2015.2437384

Huang H , Zhou H , Yang X , et al. , 2019 . Faster R-CNN for marine organisms detection and recognition using data augmentation . Neurocomputing , 337 : 372 - 384 . https://doi.org/10.1016/j.neucom.2019.01.084 https://doi.org/10.1016/j.neucom.2019.01.084

Kim J , Yu SC , 2016 . Convolutional neural network-based real-time ROV detection using forward-looking sonar image . Proc IEEE/OES Autonomous Underwater Vehicles , p. 396 - 400 . https://doi.org/10.1109/AUV.2016.7778702 https://doi.org/10.1109/AUV.2016.7778702

Kong WZ , Hong JC , Jia MY , et al. , 2020 . YOLOv3-DPFIN: a dual-path feature fusion neural network for robust real-time sonar target detection . IEEE Sens J , 20 ( 7 ):‍ 3745 - 3756 . https://doi.org/10.1109/JSEN.2019.2960796 https://doi.org/10.1109/JSEN.2019.2960796

Lin TY , Dollár P , Girshick R , et al. , 2017 . Feature pyramid networks for object detection . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 936 - 944 . https://doi.org/10.1109/CVPR.2017.106 https://doi.org/10.1109/CVPR.2017.106

Liu D , Cheng F , 2021 . SRM-FPN: a small target detection method based on FPN optimized feature . Proc 18 th Int Computer Conf on Wavelet Active Media Technology and Information Processing , p. 506 - 509 . https://doi.org/10.1109/ICCWAMTIP53232.2021.9674107 https://doi.org/10.1109/ICCWAMTIP53232.2021.9674107

Otsu N , 1979 . A threshold selection method from gray-level histograms . IEEE Trans Syst Man Cybern , 9 ( 1 ): 62 - 66 . https://doi.org/10.1109/TSMC.1979.4310076 https://doi.org/10.1109/TSMC.1979.4310076 .

Pu SL , Zhao W , Chen WJ , et al. , 2021 . Unsupervised object detection with scene-adaptive concept learning . Front Inform Technol Electron Eng , 22 ( 5 ): 638 - 651 . https://doi.org/10.1631/FITEE.2000567 https://doi.org/10.1631/FITEE.2000567

Redmon J , Farhadi A , 2018 . YOLOv3: an incremental improvement . https://doi.org/10.48550/arXiv.1804.02767 https://doi.org/10.48550/arXiv.1804.02767

Redmon J , Divvala S , Girshick R , et al. , 2016 . You Only Look Once: unified, real-time object detection . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 779 - 788 . https://doi.org/10.1109/CVPR.2016.91 https://doi.org/10.1109/CVPR.2016.91

Ren SQ , He KM , Girshick R , et al. , 2015 . Faster R-CNN: towards real-time object detection with region proposal networks . Proc 28 th Int Conf on Neural Information Processing Systems , p. 91 - 99 .

Song Y , He B , Liu P , 2021 . Real-time object detection for AUVs using self-cascaded convolutional neural networks . IEEE J Oceanic Eng , 46 ( 1 ): 56 - 67 . https://doi.org/10.1109/JOE.2019.2950974 https://doi.org/10.1109/JOE.2019.2950974

Sun PZ , Zhang RF , Jiang Y , et al. , 2021 . Sparse R-CNN: end-to-end object detection with learnable proposals . Proc IEEE Conf on Computer Vision and Pattern Recognition , p. 14449 - 14458 . https://doi.org/10.1109/CVPR46437.2021.01422 https://doi.org/10.1109/CVPR46437.2021.01422

Tian MJ , Li XL , Kong SH , et al. , 2022 . A modified YOLOv4 detection method for a vision-based underwater garbage cleaning robot . Front Inform Technol Electron Eng , 23 ( 8 ):‍ 1217 - 1228 . https://doi.org/10.1631/FITEE.2100473 https://doi.org/10.1631/FITEE.2100473

Valdenegro-Toro M , 2016 . Object recognition in forward-looking sonar images with convolutional neural networks . Proc OCEANS MTS/IEEE Monterey , p. 1 - 6 . https://doi.org/10.1109/OCEANS.2016.7761140 https://doi.org/10.1109/OCEANS.2016.7761140

Wang Z , Guo JX , Huang WZ , et al. , 2022 . Side-scan sonar image segmentation based on multi-channel fusion convolution neural networks . IEEE Sens J , 22 ( 6 ): 5911 - 5928 . https://doi.org/10.1109/JSEN.2022.3149841 https://doi.org/10.1109/JSEN.2022.3149841

Yang GY , Wang ZY , Zhuang SN , 2021 . PFF-FPN: a parallel feature fusion module based on FPN in pedestrian detection . Proc Int Conf on Computer Engineering and Artificial Intelligence , p. 377 - 381 . https://doi.org/10.1109/ICCEAI52939.2021.00075 https://doi.org/10.1109/ICCEAI52939.2021.00075

Zhang HK , Chang H , Ma BP , et al. , 2020 . Dynamic R-CNN: towards high quality object detection via dynamic training . Proc 16 th European Conf on Computer Vision , p. 260 - 275 . https://doi.org/10.1007/978-3-030-58555-6_16 https://doi.org/10.1007/978-3-030-58555-6_16

Zhou JC , Zhang DH , Ren WQ , et al. , 2022a . Auto color correction of underwater images utilizing depth information . IEEE Geosci Remote Sens Lett , 19 : 1504805 . https://doi.org/10.1109/LGRS.2022.3170702 https://doi.org/10.1109/LGRS.2022.3170702

Zhou JC , Yang TY , Chu WS , et al. , 2022b . Underwater image restoration via backscatter pixel prior and color compensation . Eng Appl Artif Intell , 111 : 104785 . https://doi.org/10.1016/j.engappai.2022.104785 https://doi.org/10.1016/j.engappai.2022.104785

Views

169

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰