聚合上下文信息的人群计数

余思悦; 浦剑

doi:10.1631/FITEE.1900481

Your Location：

Home >

Browse articles >

聚合上下文信息的人群计数

常规文章 | Updated：2022-05-19

- 聚合上下文信息的人群计数
- Aggregated context network for crowd counting
- 信息与电子工程前沿（英文） 2020年21卷第11期页码：1626-1638
- Affiliations：
  
  School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
  Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai 200433, China
- Author bio：
  
  [ "Si-yue YU, E-mail:51174500148@stu.ecnu.edu.cn" ]
  [ "", "Jian PU received his Ph.D. degree from Fudan University, China, in 2014. He was a postdoctoral researcher at the Institute of Neuroscience, Chinese Academy of Sciences from 2014 to 2016. He was an associate professor at the School of Computer Science and Technology, East China Normal University in China from 2016 to 2019. Currently, he is a professor at the Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, China. He is a corresponding expert of Frontiers of Information Technology & Electronic Engineering. His current research interests include machine learning, computer vision, autopilot, and medical image analysis" ]
- Funds：
  
  Project supported by the National Natural Science Foundation of China (Nos. 61702186, 61672236, and 61602459)
- DOI：10.1631/FITEE.1900481
  中图分类号： TP391
- 收稿：2019-09-08，
  
  修回：2020-;5-18，
  
  网络出版：2020-08-05，
  
  纸质出版：2020-11
- Accepted：
Scan QR Code
余思悦, 浦剑. 聚合上下文信息的人群计数[J]. 信息与电子工程前沿（英文）, 2020,21(11):1626-1638.

Si-yue YU, Jian PU. Aggregated context network for crowd counting[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(11): 1626-1638.
余思悦, 浦剑. 聚合上下文信息的人群计数[J]. 信息与电子工程前沿（英文）, 2020,21(11):1626-1638. DOI： 10.1631/FITEE.1900481.

Si-yue YU, Jian PU. Aggregated context network for crowd counting[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(11): 1626-1638. DOI： 10.1631/FITEE.1900481.

摘要

人群计数被大量应用于视频监控、交通监控、汇编控制以及其它公共安全应用场景。上下文信息相关的透视扭曲和背景干扰是影响人群计数准确性的两个关键因素。区别于只解决其中一种特定因素的传统方法，本文提出一种人群计数网络，其充分聚合上下文信息，达到同时解决两种因素的目的。提出一个多任务的全卷积网络结构，学习人群密度估计和语义分割辅助任务，前者通过提取多尺度和空间上下文信息学习人群密度图，辅助语义分割任务通过学习背景和前景信息，后期将语义分割提取的信息融入人群密度估计任务。结果表明，提出的人群计数网络具有较好的人群计数准确率；与其它方法相比，提出的方法在3个具有挑战性的人群数据集上具有更高鲁棒性。

Abstract

Crowd counting has been applied to a variety of applications such as video surveillance

traffic monitoring

assembly control

and other public safety applications. Context information

such as perspective distortion and background interference

is a crucial factor in achieving high performance for crowd counting. While traditional methods focus merely on solving one specific factor

we aggregate sufficient context information into the crowd counting network to tackle these problems simultaneously in this study. We build a fully convolutional network with two tasks

i.e.

main density map estimation and auxiliary semantic segmentation. The main task is to extract the multi-scale and spatial context information to learn the density map. The auxiliary semantic segmentation task gives a comprehensive view of the background and foreground information

and the extracted information is finally incorporated into the main task by late fusion. We demonstrate that our network has better accuracy of estimation and higher robustness on three challenging datasets compared with state-of-the-art methods.

关键词

Keywords

references

C Arteta , , , V Lempitsky , , , JA Noble , , , 等 . . Interactive object counting . . European Conf on Computer Vision , , 2014 . . p.504 - - 518 . . DOI: 10.1007/978-3-319-10578-9_33 http://doi.org/10.1007/978-3-319-10578-9_33 . .

L Boominathan , , , SSS Kruthiventi , , , RV Babu . . CrowdNet: a deep convolutional network for dense crowd counting . . ACM Int Conf on Multimedia , , 2016 . . p.640 - - 644 . . DOI: 10.1145/2964284.2967300 http://doi.org/10.1145/2964284.2967300 . .

XK Cao , , , ZP Wang , , , YY Zhao , , , 等 . . Scale aggregation network for accurate and efficient crowd counting . . European Conf on Computer Vision , , 2018 . . p.757 - - 773 . . DOI: 10.1007/978-3-030-01228-1_45 http://doi.org/10.1007/978-3-030-01228-1_45 . .

AB Chan , , , N Vasconcelos . . Counting people with low-level features and Bayesian regression . . IEEE Trans Image Process , , 2012 . . 21 ( ( 4 ): ): 2160 - - 2177 . . DOI: 10.1109/TIP.2011.2172800 http://doi.org/10.1109/TIP.2011.2172800 . .

AB Chan , , , ZSJ Liang , , , N Vasconcelos . . Privacy preserving crowd monitoring: counting people without people models or tracking . . IEEE Conf on Computer Vision and Pattern Recognition , , 2008 . . p.1 - - 7 . . DOI: 10.1109/CVPR.2008.4587569 http://doi.org/10.1109/CVPR.2008.4587569 . .

K Chen , , , CC Loy , , , SG Gong , , , 等 . . Feature mining for localised crowd counting . . British Machine Vision Conf, Article 21 , , 2012 . . DOI: 10.5244/c.26.21 http://doi.org/10.5244/c.26.21 . .

LC Chen , , , G Papandreou , , , F Schroff , , , 等 . . Rethinking atrous convolution for semantic image segmentation , , https://arxiv.org/abs/1706.05587 https://arxiv.org/abs/1706.05587 , , 2017 . . .

LC Chen , , , G Papandreou , , , I Kokkinos , , , 等 . . DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs . . IEEE Trans Patt Anal Mach Intell , , 2018 . . 40 ( ( 4 ): ): 834 - - 848 . . DOI: 10.1109/TPAMI.2017.2699184 http://doi.org/10.1109/TPAMI.2017.2699184 . .

J Cheng , , , PS Wang , , , G Li , , , 等 . . Recent advances in efficient computation of deep convolutional neural networks . . Front Inform Technol Electron Eng , , 2018 . . 19 ( ( 1 ): ): 64 - - 77 . . DOI: 10.1631/FITEE.1700789 http://doi.org/10.1631/FITEE.1700789 . .

RM Cong , , , JJ Lei , , , HZ Fu , , , 等 . . Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation . . IEEE Trans Image Process , , 2018 . . 27 ( ( 2 ): ): 568 - - 579 . . DOI: 10.1109/TIP.2017.2763819 http://doi.org/10.1109/TIP.2017.2763819 . .

RM Cong , , , JJ Lei , , , HZ Fu , , , 等 . . Going from RGB to RGBD saliency: a depth-guided transformation model . . IEEE Trans Cybern, in press , , 2019a . . DOI: 10.1109/TCYB.2019.2932005 http://doi.org/10.1109/TCYB.2019.2932005 . .

RM Cong , , , JJ Lei , , , HZ Fu , , , 等 . . Review of visual saliency detection with comprehensive information . . IEEE Trans Circ Syst Video Technol , , 2019b . . 29 ( ( 10 ): ): 2941 - - 2959 . . DOI: 10.1109/TCSVT.2018.2870832 http://doi.org/10.1109/TCSVT.2018.2870832 . .

RM Cong , , , JJ Lei , , , HZ Fu , , , 等 . . Video saliency detection via sparsity-based reconstruction and propagation . . IEEE Trans Image Process , , 2019c . . 28 ( ( 10 ): ): 4819 - - 4831 . . DOI: 10.1109/TIP.2019.2910377 http://doi.org/10.1109/TIP.2019.2910377 . .

N Dalal , , , B Triggs . . Histograms of oriented gradients for human detection . . IEEE Conf on Computer Vision and Pattern Recognition , , 2005 . . p.886 - - 893 . . DOI: 10.1109/CVPR.2005.177 http://doi.org/10.1109/CVPR.2005.177 . .

D Deb , , , J Ventura . . An aggregated multicolumn dilated convolution network for perspective-free counting . . IEEE Conf on Computer Vision and Pattern Recognition Workshops , , 2018 . . p.195 - - 204 . . DOI: 10.1109/CVPRW.2018.00057 http://doi.org/10.1109/CVPRW.2018.00057 . .

P Dollar , , , C Wojek , , , B Schiele , , , 等 . . Pedestrian detection: an evaluation of the state of the art . . IEEE Trans Patt Anal Mach Intell , , 2012 . . 34 ( ( 4 ): ): 743 - - 761 . . DOI: 10.1109/TPAMI.2011.155 http://doi.org/10.1109/TPAMI.2011.155 . .

L Fiaschi , , , R Nair , , , U Koethe , , , 等 . . Learning to count with regression forest and structured labels . . Int Conf on Pattern Recognition , , 2012 . . p.2685 - - 2688 . . . .

JY Gao , , , Q Wang , , , XL Li . . PCC Net: perspective crowd counting via spatial convolutional network . . IEEE Trans Circ Syst Video Technol, in press , , 2019 . . DOI: 10.1109/TCSVT.2019.2919139 http://doi.org/10.1109/TCSVT.2019.2919139 . .

XT He , , , YX Peng , , , JJ Zhao . . Fast fine-grained image classification via weakly supervised discriminative localization . . IEEE Trans Circ Syst Video Technol , , 2018 . . 29 ( ( 5 ): ): 1394 - - 1407 . . DOI: 10.1109/TCSVT.2018.2834480 http://doi.org/10.1109/TCSVT.2018.2834480 . .

JH Huang , , , XG Di , , , JD Wu , , , 等 . . A novel convolutional neural network method for crowd counting . . Front Inform Technol Electron Eng , , 2020 . . 21 ( ( 8 ): ): DOI: 10.1631/FITEE.1900282 http://doi.org/10.1631/FITEE.1900282 . .

SY Huang , , , X Li , , , ZF Zhang , , , 等 . . Body structure aware deep crowd counting . . IEEE Trans Image Process , , 2018 . . 27 1049 - - 1059 . . DOI: 10.1109/TIP.2017.2740160 http://doi.org/10.1109/TIP.2017.2740160 . .

H Idrees , , , I Saleemi , , , C Seibert , , , 等 . . Multi-source multi-scale counting in extremely dense crowd images . . IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.2547 - - 2554 . . DOI: 10.1109/CVPR.2013.329 http://doi.org/10.1109/CVPR.2013.329 . .

V Lempitsky , , , A Zisserman . . Learning to count objects in images . . Conf and Workshop on Neural Information Processing Systems , , 2010 . . p.1324 - - 1332 . . . .

CY Li , , , RM Cong , , , JH Hou , , , 等 . . Nested network with two-stream pyramid for salient object detection in optical remote sensing images . . IEEE Trans Geosci Remote Sens , , 2019 . . 57 ( ( 11 ): ): 9156 - - 9166 . . DOI: 10.1109/TGRS.2019.2925070 http://doi.org/10.1109/TGRS.2019.2925070 . .

M Li , , , ZX Zhang , , , KQ Huang , , , 等 . . Estimating the number of people in crowded scenes by MID based foreground segmentation and head-shoulder detection . . Int Conf on Pattern Recognition , , 2008 . . p.1 - - 4 . . DOI: 10.1109/ICPR.2008.4761705 http://doi.org/10.1109/ICPR.2008.4761705 . .

YH Li , , , XFF Zhang , , , DM Chen . . CSRNet: dilated convolutional neural networks for understanding the highly congested scenes . . IEEE Conf on Computer Vision and Pattern Recognition , , 2018 . . p.1091 - - 1100 . . DOI: 10.1109/CVPR.2018.00120 http://doi.org/10.1109/CVPR.2018.00120 . .

J Long , , , E Shelhamer , , , T Darrell . . Fully convolutional networks for semantic segmentation . . IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.3431 - - 3440 . . DOI: 10.1109/CVPR.2015.7298965 http://doi.org/10.1109/CVPR.2015.7298965 . .

CC Loy , , , K Chen , , , SG Gong , , , 等 . . Crowd Counting and Profiling: Methodology and Evaluation , , : : New York, USA Springer , , 2013 . . DOI: 10.1007/978-1-4614-8483-7_14 http://doi.org/10.1007/978-1-4614-8483-7_14 . .

D Ooro-Rubio , , , RJ Lpez-Sastre . . Towards perspective-free object counting with deep learning . . European Conf on Computer Vision , , 2016 . . p.615 - - 629 . . DOI: 10.1007/978-3-319-46478-7_38 http://doi.org/10.1007/978-3-319-46478-7_38 . .

A Paszke , , , S Gross , , , S Chintala , , , 等 . . Automatic differentiation in PyTorch . . 31 st Conf on Neural Information Processing Systems , , 2017 . . p.1 - - 4 . . . .

YX Peng , , , XT He , , , JJ Zhao . . Object-part attention model for fine-grained image classification . . IEEE Trans Image Process , , 2018 . . 27 ( ( 3 ): ): 1487 - - 1500 . . DOI: 10.1109/TIP.2017.2774041 http://doi.org/10.1109/TIP.2017.2774041 . .

VQ Pham , , , T Kozakaya , , , O Yamaguchi , , , 等 . . COUNT forest: CO-voting uncertain number of targets using random forest for crowd density estimation . . IEEE Int Conf on Computer Vision , , 2015 . . p.3253 - - 3261 . . DOI: 10.1109/ICCV.2015.372 http://doi.org/10.1109/ICCV.2015.372 . .

J Pu , , , YG Jiang , , , J Wang , , , 等 . . Which looks like which: exploring inter-class relationships in fine-grained visual categorization . . European Conf on Computer Vision , , 2014 . . p.425 - - 440 . . DOI: 10.1007/978-3-319-10578-9_28 http://doi.org/10.1007/978-3-319-10578-9_28 . .

V Rabaud , , , S Belongie . . Counting crowded moving objects . . IEEE Conf on Computer Vision and Pattern Recognition , , 2006 . . p.705 - - 711 . . DOI: 10.1109/CVPR.2006.92 http://doi.org/10.1109/CVPR.2006.92 . .

M Rodriguez , , , I Laptev , , , J Sivic , , , 等 . . Density-aware person detection and tracking in crowds . . IEEE Int Conf on Computer Vision , , 2011 . . p.2423 - - 2430 . . DOI: 10.1109/ICCV.2011.6126526 http://doi.org/10.1109/ICCV.2011.6126526 . .

S Ruder . . An overview of multi-task learning in deep neural networks , , https://arxiv.org/abs/1706.05098 https://arxiv.org/abs/1706.05098 , , 2017 . . .

D Ryan , , , S Denman , , , CB Fookes , , , 等 . . Crowd counting using multiple local features . . Proc Digital Image Computing: Techniques and Applications , , 2010 . . p.81 - - 88 . . DOI: 10.1109/DICTA.2009.22 http://doi.org/10.1109/DICTA.2009.22 . .

DB Sam , , , RV Babu . . Top-down feedback for crowd counting convolutional neural network . . AAAI Conf on Artificial Intelligence , , 2018 . . p.7323 - - 7330 . . . .

DB Sam , , , S Surya , , , RV Babu . . Switching convolutional neural network for crowd counting . . IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p.4031 - - 4039 . . DOI: 10.1109/CVPR.2017.429 http://doi.org/10.1109/CVPR.2017.429 . .

DB Sam , , , NN Sajjan , , , RV Babu . . Divide and grow: capturing huge diversity in crowd images with incrementally growing CNN . . IEEE Conf on Computer Vision and Pattern Recognition , , 2018 . . p.3618 - - 3626 . . DOI: 10.1109/CVPR.2018.00381 http://doi.org/10.1109/CVPR.2018.00381 . .

C Shang , , , HZ Ai , , , B Bai . . End-to-end crowd counting via joint learning local and global count . . IEEE Int Conf on Image Processing , , 2016 . . p.1215 - - 1219 . . DOI: 10.1109/icip.2016.7532551 http://doi.org/10.1109/icip.2016.7532551 . .

Z Shen , , , Y Xu , , , BB Ni , , , 等 . . Crowd counting via adversarial cross-scale consistency pursuit . . IEEE Conf on Computer Vision and Pattern Recognition , , 2018 . . p.5245 - - 5254 . . DOI: 10.1109/CVPR.2018.00550 http://doi.org/10.1109/CVPR.2018.00550 . .

MJ Shi , , , ZH Yang , , , C Xu , , , 等 . . Revisiting perspective information for efficient crowd counting . . IEEE Conf on Computer Vision and Pattern Recognition , , 2019 . . p.7271 - - 7280 . . DOI: 10.1109/CVPR.2019.00745 http://doi.org/10.1109/CVPR.2019.00745 . .

VA Sindagi , , , VM Patel . . CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting . . IEEE Int Conf on Advanced Video and Signal Based Surveillance , , 2017a . . p.1 - - 6 . . DOI: 10.1109/AVSS.2017.8078491 http://doi.org/10.1109/AVSS.2017.8078491 . .

VA Sindagi , , , VM Patel . . Generating high-quality crowd density maps using contextual pyramid CNNs . . IEEE Int Conf on Computer Vision , , 2017b . . p.1879 - - 1888 . . DOI: 10.1109/ICCV.2017.206 http://doi.org/10.1109/ICCV.2017.206 . .

VA Sindagi , , , VM Patel . . A survey of recent advances in CNN-based single image crowd counting and density estimation . . Patt Recogn Lett , , 2018 . . 107 3 - - 16 . . DOI: 10.1016/j.patrec.2017.07.007 http://doi.org/10.1016/j.patrec.2017.07.007 . .

P Viola , , , MJ Jones . . Robust real-time face detection . . Int J Comput Vis , , 2004 . . 57 ( ( 2 ): ): 137 - - 154 . . DOI: 10.1109/ICCV.2001.937709 http://doi.org/10.1109/ICCV.2001.937709 . .

E Walach , , , L Wolf . . Learning to count with CNN boosting . . European Conf on Computer Vision , , 2016 . . p.660 - - 676 . . DOI: 10.1007/978-3-319-46475-6_41 http://doi.org/10.1007/978-3-319-46475-6_41 . .

C Wang , , , H Zhang , , , L Yang , , , 等 . . Deep people counting in extremely dense crowds . . ACM Int Conf on Multimedia , , 2015 . . p.1299 - - 1302 . . DOI: 10.1145/2733373.2806337 http://doi.org/10.1145/2733373.2806337 . .

LY Wang , , , BQ Yin , , , AX Guo , , , 等 . . Skip-connection convolutional neural network for still image crowd counting . . Appl Intell , , 2018 . . 48 3360 - - 3371 . . DOI: 10.1007/s10489-018-1150-1 http://doi.org/10.1007/s10489-018-1150-1 . .

LY Wang , , , BQ Yin , , , X Tang , , , 等 . . Removing background interference for crowd counting via de-background detail convolutional network . . Neurocomputing , , 2019 . . 332 360 - - 371 . . DOI: 10.1016/j.neucom.2018.12.047 http://doi.org/10.1016/j.neucom.2018.12.047 . .

WX Xie , , , YX Peng , , , JG Xiao . . Weakly-supervised image parsing via constructing semantic graphs and hypergraphs . . Proc 22 nd ACM Int Conf on Multimedia , , 2014 . . 277 - - 286 . . DOI: 10.1145/2647868.2654910 http://doi.org/10.1145/2647868.2654910 . .

C Zhang , , , HS Li , , , XG Wang , , , 等 . . Cross-scene crowd counting via deep convolutional neural networks . . IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.833 - - 841 . . DOI: 10.1109/CVPR.2015.7298684 http://doi.org/10.1109/CVPR.2015.7298684 . .

YY Zhang , , , DS Zhou , , , SQ Chen , , , 等 . . Single-image crowd counting via multi-column convolutional neural network . . IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.589 - - 597 . . DOI: 10.1109/CVPR.2016.70 http://doi.org/10.1109/CVPR.2016.70 . .

C Zhu , , , YX Peng . . Group cost-sensitive boosting for multi-resolution pedestrian detection . . 30 th AAAI Conf on Artificial Intelligence , , 2016 . . p.3676 - - 3682 . . . .

浏览量

Downloads

CSCD

文章被引用时，请邮件提醒。

Submit

工具集

关联资源

A novel convolutional neural network method for crowd counting

Recognition method for underwater communication signals that mimic dolphin whistles using phase-shifting modulation

Efficient privacy-preserving scheme for secure neural network inference

CRGT-SA: an interlaced and spatiotemporal deep learning model for network intrusion detection

Ahybrid-model optimization algorithm based on the Gaussian process and particle swarm optimization for mixed-variable CNN hyperparameter automatic search