Filter-cluster attention based recursive network for low-light enhancement

Zhixiong HUANG; Jinjiang LI; Zhen HUA; Linwei FAN

doi:10.1631/FITEE.2200344

Your Location：

Home >

Browse articles >

Filter-cluster attention based recursive network for low-light enhancement

Regular Papers | Updated：2023-07-24

- Filter-cluster attention based recursive network for low-light enhancement
- 基于过滤—群聚注意力的低光增强递归网络
- Frontiers of Information Technology & Electronic Engineering Vol. 24, Issue 7, Pages: 1028-1044(2023)
- Affiliations：
  
  1.School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai 264005, China
  2.School of Computer Science and Technology, Shandong Technology and Business University, Yantai 264005, China
  3.Co-innovation Center of Shandong Colleges and Universities: Future Intelligent Computing, Shandong Technology and Business University, Yantai 264005, China
  4.School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan 250014, China
- Author bio：
  
  †E-mail: hzxcyanwind@163.com
  ‡Corresponding author
- Funds：
- DOI：10.1631/FITEE.2200344
  CLC： TP391
- Received：11 August 2022，
  
  Accepted：28 November 2022，
  
  Published：0 July 2023
- Accepted：
Scan QR Code
Zhixiong HUANG, Jinjiang LI, Zhen HUA, et al. Filter-cluster attention based recursive network for low-light enhancement[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(7): 1028-1044.
DOI：

Zhixiong HUANG, Jinjiang LI, Zhen HUA, et al. Filter-cluster attention based recursive network for low-light enhancement[J]. Frontiers of Information Technology & Electronic Engineering, 2023, 24(7): 1028-1044. DOI： 10.1631/FITEE.2200344.

摘要

在低光环境下拍摄的图像质量不佳，影响其进一步应用。为提升低光图像可视性，提出一种基于过滤—群聚注意力（FCA）的递归网络，其中主体由3个单元组成：差异关注、门控递归以及迭代残差。该网络对低光图像进行多阶段递归学习，进而提取更深层次特征信息。为算得更加精确的相关性，设计了一种关注特征通道突出性的FCA。FCA与自注意力被用以突出特征的低光区域与重要通道。此外，设计了密集连接金字塔（DenCP）来提取低光反转图的色彩特征，使图像的色彩信息损失得以补偿。在6种公开数据集上的实验结果表明，本文方法在视觉和指标上有着突出表现。

Abstract

The poor quality of images recorded in low-light environments affects their further applications. To improve the visibility of low-light images

we propose a recurrent network based on filter-cluster attention (FCA)

the main body of which consists of three units: difference concern

gate recurrent

and iterative residual. The network performs multi-stage recursive learning on low-light images

and then extracts deeper feature information. To compute more accurate dependence

we design a novel FCA that focuses on the saliency of feature channels. FCA and self-attention are used to highlight the low-light regions and important channels of the feature. We also design a dense connection pyramid (DenCP) to extract the color features of the low-light inversion image

to compensate for the loss of the image’s color information. Experimental results on six public datasets show that our method has outstanding performance in subjective and quantitative comparisons.

关键词

Keywords

references

Abdullah-Al-Wadud M , Kabir H , Dewan MAA , et al. , 2007 . A dynamic histogram equalization for image contrast enhancement . IEEE Trans Consum Electron , 53 ( 2 ): 593 - 600 . doi: 10.1109/TCE.2007.381734 http://doi.org/10.1109/TCE.2007.381734

Aradi S , 2022 . Survey of deep reinforcement learning for motion planning of autonomous vehicles . IEEE Trans Intell Transp Syst , 23 ( 2 ): 740 - 759 . doi: 10.1109/TITS.2020.3024655 http://doi.org/10.1109/TITS.2020.3024655

Bychkovsky V , Paris S , Chan E , et al. , 2011 . Learning photographic global tonal adjustment with a database of input/output image pairs . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 97 - 104 . doi: 10.1109/CVPR.2011.5995413 http://doi.org/10.1109/CVPR.2011.5995413

Celik T , Tjahjadi T , 2011 . Contextual and variational contrast enhancement . IEEE Trans Image Process , 20 ( 12 ): 3431 - 3441 . doi: 10.1109/TIP.2011.2157513 http://doi.org/10.1109/TIP.2011.2157513

Chen BH , Wu YL , Shi LF , 2019 . A fast image contrast enhancement algorithm using entropy-preserving mapping prior . IEEE Trans Circ Syst Video Technol , 29 ( 1 ): 38 - 49 . doi: 10.1109/TCSVT.2017.2773461 http://doi.org/10.1109/TCSVT.2017.2773461

Cheng HD , Shi XJ , 2004 . A simple and effective histogram equalization approach to image enhancement . Dig Signal Process , 14 ( 2 ): 158 - 170 . doi: 10.1016/j.dsp.2003.07.002 http://doi.org/10.1016/j.dsp.2003.07.002

Cho K , van Merriënboer B , Gulcehre C , et al. , 2014 . Learning phrase representations using RNN encoder-decoder for statistical machine translation . Proc Conf on Empirical Methods in Natural Language Processing , p. 1724 - 1734 . doi: 10.3115/v1/D14-1179 http://doi.org/10.3115/v1/D14-1179

Guo CL , Li CY , Guo JC , et al. , 2020 . Zero-reference deep curve estimation for low-light image enhancement . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 1780 - 1789 . doi: 10.1109/CVPR42600.2020.00185 http://doi.org/10.1109/CVPR42600.2020.00185

Guo XJ , Li Y , Ling HB , 2017 . LIME: low-light image enhancement via illumination map estimation . IEEE Trans Image Process , 26 ( 2 ): 982 - 993 . doi: 10.1109/TIP.2016.2639450 http://doi.org/10.1109/TIP.2016.2639450

Hao SJ , Han X , Guo YR , et al. , 2020 . Low-light image enhancement with semi-decoupled decomposition . IEEE Trans Multim , 22 ( 12 ): 3025 - 3038 . doi: 10.1109/TMM.2020.2969790 http://doi.org/10.1109/TMM.2020.2969790

Hochreiter S , Schmidhuber J , 1997 . Long short-term memory . Neur Comput , 9 ( 8 ): 1735 - 1780 . doi: 10.1162/neco.1997.9.8.1735 http://doi.org/10.1162/neco.1997.9.8.1735

Huang G , Liu Z , van der Maaten L , et al. , 2017 . Densely connected convolutional networks . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 2261 - 2269 . doi: 10.1109/CVPR.2017.243 http://doi.org/10.1109/CVPR.2017.243

Huang ZX , Li JJ , Hua Z , et al. , 2022 . Underwater image enhancement via adaptive group attention-based multiscale cascade transformer . IEEE Trans Instrum Meas , 71 : 5015618 . doi: 10.1109/TIM.2022.3189630 http://doi.org/10.1109/TIM.2022.3189630

Jiang YF , Gong XY , Liu D , et al. , 2021 . EnlightenGAN: deep light enhancement without paired supervision . IEEE Trans Image Process , 30 : 2340 - 2349 . doi: 10.1109/TIP.2021.3051462 http://doi.org/10.1109/TIP.2021.3051462

Jung E , Yang N , Cremers D , 2020 . Multi-frame GAN: image enhancement for stereo visual odometry in low light . Proc 3 rd Annual Conf on Robot Learning , p. 651 - 660 .

Kingma DP , Ba J , 2014 . Adam: a method for stochastic optimization . Proc 3 rd Int Conf on Learning Representations .

Lee C , Lee C , Kim CS , 2012 . Contrast enhancement based on layered difference representation . 19 th IEEE Int Conf on Image Processing , p. 965 - 968 . doi: 10.1109/ICIP.2012.6467022 http://doi.org/10.1109/ICIP.2012.6467022

Lee C , Lee C , Kim CS , 2013 . Contrast enhancement based on layered difference representation of 2D histograms . IEEE Trans Image Process , 22 ( 12 ): 5372 - 5384 . doi: 10.1109/TIP.2013.2284059 http://doi.org/10.1109/TIP.2013.2284059

Li CL , Tang SQ , Yan JW , et al. , 2020 . Low-light image enhancement based on quasi-symmetric correction functions by fusion . Symmetry , 12 ( 9 ): 1561 . doi: 10.3390/sym12091561 http://doi.org/10.3390/sym12091561

Li JJ , Feng XM , Hua Z , 2021 . Low-light image enhancement via progressive-recursive network . IEEE Trans Circ Syst Video Technol , 31 ( 11 ): 4227 - 4240 . doi: 10.1109/TCSVT.2021.3049940 http://doi.org/10.1109/TCSVT.2021.3049940

Li L , Wang RG , Wang WM , et al. , 2015 . A low-light image enhancement method for both denoising and contrast enlarging . IEEE Int Conf on Image Processing , p. 3730 - 3734 . doi: 10.1109/ICIP.2015.7351501 http://doi.org/10.1109/ICIP.2015.7351501

Li MD , Liu JY , Yang WH , et al. , 2018 . Structure-revealing low-light image enhancement via robust retinex model . IEEE Trans Image Process , 27 ( 6 ): 2828 - 2841 . doi: 10.1109/TIP.2018.2810539 http://doi.org/10.1109/TIP.2018.2810539

Li PL , Liang JL , Zhang MH , 2021 . A degradation model for simultaneous brightness and sharpness enhancement of low-light image . Signal Process , 189 : 108298 . doi: 10.1016/j.sigpro.2021.108298 http://doi.org/10.1016/j.sigpro.2021.108298

Lim KL , Jiang XD , Yi CY , 2020 . Deep clustering with variational autoencoder . IEEE Signal Process Lett , 27 : 231 - 235 . doi: 10.1109/LSP.2020.2965328 http://doi.org/10.1109/LSP.2020.2965328

Liu L , Ouyang WL , Wang XG , et al. , 2020 . Deep learning for generic object detection: a survey . Int J Comput Vis , 128 ( 2 ): 261 - 318 . doi: 10.1007/s11263-019-01247-4 http://doi.org/10.1007/s11263-019-01247-4

Liu RS , Ma L , Zhang JA , et al. , 2021 . Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 10556 - 10565 . doi: 10.1109/CVPR46437.2021.01042 http://doi.org/10.1109/CVPR46437.2021.01042

Liu YJ , Wang ZN , Zeng Y , et al. , 2021 . PD-GAN: perceptual-details GAN for extremely noisy low light image enhancement . IEEE Int Conf on Acoustics, Speech and Signal Processing , p. 1840 - 1844 . doi: 10.1109/ICASSP39728.2021.9413433 http://doi.org/10.1109/ICASSP39728.2021.9413433

Loh YP , Chan CS , 2019 . Getting to know low-light images with the exclusively dark dataset . Comput Vis Image Underst , 178 : 30 - 42 . doi: 10.1016/j.cviu.2018.10.010 http://doi.org/10.1016/j.cviu.2018.10.010

Lore KG , Akintayo A , Sarkar S , 2017 . LLNet: a deep autoencoder approach to natural low-light image enhancement . Patt Recogn , 61 : 650 - 662 . doi: 10.1016/j.patcog.2016.06.008 http://doi.org/10.1016/j.patcog.2016.06.008

Lv FF , Li Y , Lu F , 2021 . Attention guided low-light image enhancement with a large scale low-light simulation dataset . Int J Comput Vis , 129 ( 7 ): 2175 - 2193 . doi: 10.1007/s11263-021-01466-8 http://doi.org/10.1007/s11263-021-01466-8

Ma L , Liu RS , Zhang JA , et al. , 2022 . Learning deep context-sensitive decomposition for low-light image enhancement . IEEE Trans Neur Netw Learn Syst , 33 ( 10 ): 5666 - 5680 . doi: 10.1109/TNNLS.2021.3071245 http://doi.org/10.1109/TNNLS.2021.3071245

Mittal A , Soundararajan R , Bovik AC , 2013 . Making a “completely blind” image quality analyzer . IEEE Signal Process Lett , 20 ( 3 ): 209 - 212 . doi: 10.1109/LSP.2012.2227726 http://doi.org/10.1109/LSP.2012.2227726

Peng T , Su LL , Zhang RH , et al. , 2020 . A new safe lane-change trajectory model and collision avoidance control method for automatic driving vehicles . Expert Syst Appl , 141 : 112953 . doi: 10.1016/j.eswa.2019.112953 http://doi.org/10.1016/j.eswa.2019.112953

Ren WQ , Liu SF , Ma L , et al. , 2019 . Low-light image enhancement via a deep hybrid network . IEEE Trans Image Process , 28 ( 9 ): 4364 - 4375 . doi: 10.1109/TIP.2019.2910412 http://doi.org/10.1109/TIP.2019.2910412

Ren XT , Li MD , Cheng WH , et al. , 2018 . Joint enhancement and denoising method via sequential decomposition . IEEE Int Symp on Circuits and Systems , p. 1 - 5 . doi: 10.1109/ISCAS.2018.8351427 http://doi.org/10.1109/ISCAS.2018.8351427

Shiau YH , Chen PY , Yang HY , et al. , 2015 . A low-cost hardware architecture for illumination adjustment in real-time applications . IEEE Trans Intell Transp Syst , 16 ( 2 ): 934 - 946 . doi: 10.1109/TITS.2014.2347701 http://doi.org/10.1109/TITS.2014.2347701

Singh H , Kumar A , Balyan LK , et al. , 2017 . A novel optimally gamma corrected intensity span maximization approach for dark image enhancement . 22 nd Int Conf on Digital Signal Processing , p. 1 - 5 . doi: 10.1109/ICDSP.2017.8096035 http://doi.org/10.1109/ICDSP.2017.8096035

Singh N , Bhandari AK , 2021 . Principal component analysis-based low-light image enhancement using reflection model . IEEE Trans Instrum Meas , 70 : 70:5012710 . doi: 10.1109/TIM.2021.3096266 http://doi.org/10.1109/TIM.2021.3096266

Wang LW , Liu ZS , Siu WC , et al. , 2020 . Lightening network for low-light image enhancement . IEEE Trans Image Process , 29 : 7984 - 7996 . doi: 10.1109/TIP.2020.3008396 http://doi.org/10.1109/TIP.2020.3008396

Wang QL , Wu BG , Zhu PF , et al. , 2020 . ECA-Net: efficient channel attention for deep convolutional neural networks . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 11531 - 11539 . doi: 10.1109/CVPR42600.2020.01155 http://doi.org/10.1109/CVPR42600.2020.01155

Wang SH , Luo G , 2018 . Naturalness preserved image enhancement using a priori multi-layer lightness statistics . IEEE Trans Image Process , 27 ( 2 ): 938 - 948 . doi: 10.1109/TIP.2017.2771449 http://doi.org/10.1109/TIP.2017.2771449

Wang SH , Zheng J , Hu HM , et al. , 2013 . Naturalness preserved enhancement algorithm for non-uniform illumination images . IEEE Trans Image Process , 22 ( 9 ): 3538 - 3548 . doi: 10.1109/TIP.2013.2261309 http://doi.org/10.1109/TIP.2013.2261309

Wang W , Sun N , Ng MK , 2019 . A variational gamma correction model for image contrast enhancement . Inv Probl Imag , 13 ( 3 ): 461 - 478 . doi: 10.3934/ipi.2019023 http://doi.org/10.3934/ipi.2019023

Wang YF , Liu HM , Fu ZW , 2019 . Low-light image enhancement via the absorption light scattering model . IEEE Trans Image Process , 28 ( 11 ): 5679 - 5690 . doi: 10.1109/TIP.2019.2922106 http://doi.org/10.1109/TIP.2019.2922106

Wei C , Wang WJ , Yang WH , et al. , 2018 . Deep retinex decomposition for low-light enhancement . British Machine Vision Conf , Article 155 .

Wu XM , Liu XH , Hiramatsu K , et al. , 2017 . Contrast-accumulated histogram equalization for image enhancement . IEEE Int Conf on Image Processing , p. 3190 - 3194 . doi: 10.1109/ICIP.2017.8296871 http://doi.org/10.1109/ICIP.2017.8296871

Xie EZ , Ding J , Wang WH , et al. , 2021 . DetCo: unsupervised contrastive learning for object detection . IEEE/CVF Int Conf on Computer Vision , p. 8372 - 8381 . doi: 10.1109/ICCV48922.2021.00828 http://doi.org/10.1109/ICCV48922.2021.00828

Xu CR , Peng ZZ , Hu XZ , et al. , 2020 . FPGA-based low-visibility enhancement accelerator for video sequence by adaptive histogram equalization with dynamic clip-threshold . IEEE Trans Circ Syst I Regul Papers , 67 ( 11 ): 3954 - 3964 . doi: 10.1109/TCSI.2020.3010634 http://doi.org/10.1109/TCSI.2020.3010634

Xu K , Yang X , Yin BC , et al. , 2020 . Learning to restore low-light images via decomposition-and-enhancement . IEEE/CVF Conf on Computer Vision and Pattern Recognition , p. 2278 - 2287 . doi: 10.1109/CVPR42600.2020.00235 http://doi.org/10.1109/CVPR42600.2020.00235

Xu YD , Yang C , Sun BB , et al. , 2021 . A novel multi-scale fusion framework for detail-preserving low-light image enhancement . Inform Sci , 548 : 378 - 397 . doi: 10.1016/j.ins.2020.09.066 http://doi.org/10.1016/j.ins.2020.09.066

Yan XA , Liu Y , Jia MP , 2020a . Multiscale cascading deep belief network for fault identification of rotating machinery under various working conditions . Knowl-Based Syst , 193 : 105484 . doi: 10.1016/j.knosys.2020.105484 http://doi.org/10.1016/j.knosys.2020.105484

Yan XA , Liu Y , Xu YD , et al. , 2020b . Multistep forecasting for diurnal wind speed based on hybrid deep learning model with improved singular spectrum decomposition . Energy Conv Manag , 225 : 113456 . doi: 10.1016/j.enconman.2020.113456 http://doi.org/10.1016/j.enconman.2020.113456

Yang B , Cao XL , Yuen C , et al. , 2021 . Offloading optimization in edge computing for deep-learning-enabled target tracking by Internet of UAVs . IEEE Int Things J , 8 ( 12 ): 9878 - 9893 . doi: 10.1109/JIOT.2020.3016694 http://doi.org/10.1109/JIOT.2020.3016694

Yang WH , Wang WJ , Huang HF , et al. , 2021a . Sparse gradient regularized deep retinex network for robust low-light image enhancement . IEEE Trans Image Process , 30 : 2072 - 2086 . doi: 10.1109/TIP.2021.3050850 http://doi.org/10.1109/TIP.2021.3050850

Yang WH , Wang SQ , Fang YM , et al. , 2021b . Band representation-based semi-supervised low-light image enhancement: bridging the gap between signal fidelity and perceptual quality . IEEE Trans Image Process , 30 : 3461 - 3473 . doi: 10.1109/TIP.2021.3062184 http://doi.org/10.1109/TIP.2021.3062184

Ying ZQ , Li G , Ren YR , et al. , 2017 . A new low-light image enhancement algorithm using camera response model . IEEE Int Conf on Computer Vision Workshops , p. 3015 - 3022 . doi: 10.1109/ICCVW.2017.356 http://doi.org/10.1109/ICCVW.2017.356

Yu SY , Zhu H , 2019 . Low-illumination image enhancement algorithm based on a physical lighting model . IEEE Trans Circ Syst Video Technol , 29 ( 1 ): 28 - 37 . doi: 10.1109/TCSVT.2017.2763180 http://doi.org/10.1109/TCSVT.2017.2763180

Zamir SW , Arora A , Khan S , et al. , 2020 . Learning enriched features for real image restoration and enhancement . Proc 16 th European Conf on Computer Vision , p. 492 - 511 . doi: 10.1007/978-3-030-58595-2_30 http://doi.org/10.1007/978-3-030-58595-2_30

Zhang L , Zhang L , Mou XQ , et al. , 2011 . FSIM: a feature similarity index for image quality assessment . IEEE Trans Image Process , 20 ( 8 ): 2378 - 2386 . doi: 10.1109/TIP.2011.2109730 http://doi.org/10.1109/TIP.2011.2109730

Zhang TL , Li JJ , Fan H , 2022 . Progressive edge-sensing dynamic scene deblurring . Comput Visual Media , 8 ( 3 ): 495 - 508 . doi: 10.1007/s41095-021-0246-4 http://doi.org/10.1007/s41095-021-0246-4

Zhang YH , Zhang JW , Guo XJ , 2019 . Kindling the darkness: a practical low-light image enhancer . Proc 27 th ACM Int Conf on Multimedia , p. 1632 - 1640 . doi: 10.1145/3343031.3350926 http://doi.org/10.1145/3343031.3350926

Views

150

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰