A novel convolutional neural network method for crowd counting

Jie-hao HUANG; Xiao-guang DI; Jun-de WU; Ai-yue CHEN

doi:10.1631/FITEE.1900282

Your Location：

Home >

Browse articles >

A novel convolutional neural network method for crowd counting

Regular Papers | Updated：2022-05-19

- A novel convolutional neural network method for crowd counting
- 一种新的基于卷积神经网络的人群计数方法
- Frontiers of Information Technology & Electronic Engineering Vol. 21, Issue 8, Pages: 1150-1160(2020)
- Affiliations：
  
  Control and Simulation Center, Harbin Institute of Technology, Harbin 150080, China
- Author bio：
  
  [ "Jie-hao HUANG, E-mail: 18s004055@hit.edu.cn" ]
  Xiao-guang DI, E-mail: dixiaoguang@hit.edu.cn
- Funds：
- DOI：10.1631/FITEE.1900282
  CLC： TP391.4
- Received：05 June 2019，
  
  Revised：2020-;6-11，
  
  Published：2020-08
- Accepted：
Scan QR Code
Jie-hao HUANG, Xiao-guang DI, Jun-de WU, et al. A novel convolutional neural network method for crowd counting[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1150-1160.
DOI：

Jie-hao HUANG, Xiao-guang DI, Jun-de WU, et al. A novel convolutional neural network method for crowd counting[J]. Frontiers of Information Technology & Electronic Engineering, 2020, 21(8): 1150-1160. DOI： 10.1631/FITEE.1900282.

摘要

人群密度估计是一项具有挑战性的任务，因为人群中人头大小存在大范围变化。现有方法均采用多列式结构卷积神经网络去适应这种变化，但会导致密度图上不同密度区域产生平均效应并引入额外噪声。为解决该问题，提出一种新的基于分割先验图的神经网络方法，在分割图基础上生成一个高质量且没有噪声的密度图。该网络主要包括两个部分，即头部的人群前景分割神经网络和尾部的人群回归神经网络。在数据集只提供单点人头标记的情况下，采用均匀函数生成人群头部的掩膜真值图。基于该真值图，前景分割网络输出人群分割图，可有效减少密度图中无人区域噪声。将人群分割图输入人群回归网络，后者能生成高质量人群密度图并提供准确的人数估计。在4个公开数据集（即ShanghaiTech、UCF-CC-50、WorldExpo’10和UCSD）上验证了所提方法有效性；其中，在ShanghaiTech partB和UCF-CC-50两个数据集上该方法取得了当前最好结果。

Abstract

Crowd density estimation

in general

is a challenging task due to the large variation of head sizes in the crowds. Existing methods always use a multi-column convolutional neural network (MCNN) to adapt to this variation

which results in an average effect in areas with different densities and brings a lot of noise to the density map. To address this problem

we propose a new method called the segmentation-aware prior network (SAPNet)

which generates a high-quality density map without noise based on a coarse head-segmentation map. SAPNet is composed of two networks

i.e.

a foreground-segmentation convolutional neural network (FS-CNN) as the front end and a crowd-regression convolutional neural network (CR-CNN) as the back end. With only the single dot annotation

we generate the ground truth of segmentation masks in heads. Then

based on the ground truth

FS-CNN outputs a coarse head-segmentation map

which helps eliminate the noise in regions without people in the density map. By inputting the head-segmentation map generated by the front end

CR-CNN performs accurate crowd counting estimation and generates a high-quality density map. We demonstrate SAPNet on four datasets (i.e.

ShanghaiTech

UCF-CC-50

WorldExpo'10

and UCSD)

and show the state-of-the-art performances on ShanghaiTech part

and UCF-CC-50 datasets.

关键词

Keywords

references

J Canny . . A computational approach to edge detection . . IEEE Trans Patt Anal Mach Intell , , 1986 . . 8 ( ( 6 ): ): 679 - - 698 . . DOI: 10.1109/TPAMI.1986.4767851 http://doi.org/10.1109/TPAMI.1986.4767851 . .

AB Chan , , , N Vasconcelos . . Bayesian Poisson regression for crowd counting . . Proc IEEE 12 th Int Conf on Computer Vision , , 2009 . . p.545 - - 551 . . DOI: 10.1109/ICCV.2009.5459191 http://doi.org/10.1109/ICCV.2009.5459191 . .

AB Chan , , , ZSJ Liang , , , N Vasconcelos . . Privacy preserving crowd monitoring: counting people without people models or tracking . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2008 . . p.1 - - 7 . . DOI: 10.1109/CVPR.2008.4587569 http://doi.org/10.1109/CVPR.2008.4587569 . .

JF Dai , , , Y Li , , , KM He , , , 等 . . R-FCN: object detection via region-based fully convolutional networks . . Proc 30 th Int Conf on Neural Information Processing Systems , , 2016 . . p.379 - - 387 . . . .

P Dollar , , , C Wojek , , , B Schiele , , , 等 . . Pedestrian detection: an evaluation of the state of the art . . IEEE Trans Patt Anal Mach Intell , , 2012 . . 34 ( ( 4 ): ): 743 - - 761 . . DOI: 10.1109/TPAMI.2011.155 http://doi.org/10.1109/TPAMI.2011.155 . .

H Idrees , , , I Saleemi , , , C Seibert , , , 等 . . Multi-source multi-scale counting in extremely dense crowd images . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2013 . . p.2547 - - 2554 . . DOI: 10.1109/CVPR.2013.329 http://doi.org/10.1109/CVPR.2013.329 . .

K Kang , , , XG Wang . . Fully convolutional neural networks for crowd segmentation . . 2014 . . https://arxiv.org/abs/1411.4464 https://arxiv.org/abs/1411.4464 , , . .

V Lempitsky , , , A Zisserman . . Learning to count objects in images . . Proc 23 rd Int Conf on Neural Information Processing Systems , , 2010 . . p.1324 - - 1332 . . . .

HH Li , , , XJ He , , , HF Wu , , , 等 . . Structured inhomogeneous density map learning for crowd counting . . 2018 . . https://arxiv.org/abs/1801.06642 https://arxiv.org/abs/1801.06642 , , . .

JJ Li , , , H Yang , , , S Wu . . Crowd semantic segmentation based on spatial-temporal dynamics . . Proc 13 th IEEE Int Conf on Advanced Video and Signal Based Surveillance , , 2016 . . p.102 - - 108 . . DOI: 10.1109/AVSS.2016.7738032 http://doi.org/10.1109/AVSS.2016.7738032 . .

T Li , , , H Chang , , , M Wang , , , 等 . . Crowded scene analysis: a survey . . IEEE Trans Circ Syst Video Technol , , 2015 . . 25 ( ( 3 ): ): 367 - - 386 . . DOI: 10.1109/TCSVT.2014.2358029 http://doi.org/10.1109/TCSVT.2014.2358029 . .

YH Li , , , XF Zhang , , , DM Chen . . CSRNet: dilated convolutional neural networks for understanding the highly congested scenes . . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , , 2018 . . p.1091 - - 1100 . . DOI: 10.1109/CVPR.2018.00120 http://doi.org/10.1109/CVPR.2018.00120 . .

J Liu , , , CQ Gao , , , DY Meng , , , 等 . . DecideNet: counting varying density crowds through attention guided detection and density estimation . . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , , 2018 . . p.5197 - - 5206 . . DOI: 10.1109/CVPR.2018.00545 http://doi.org/10.1109/CVPR.2018.00545 . .

J Long , , , E Shelhamer , , , T Darrell . . Fully convolutional networks for semantic segmentation . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.3431 - - 3440 . . DOI: 10.1109/CVPR.2015.7298965 http://doi.org/10.1109/CVPR.2015.7298965 . .

DB Sam , , , S Surya , , , RV Babu . . Switching convolutional neural network for crowd counting . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2017 . . p.4031 - - 4039 . . DOI: 10.1109/CVPR.2017.429 http://doi.org/10.1109/CVPR.2017.429 . .

DB Sam , , , NN Sajjan , , , RV Babu . . Divide and grow: capturing huge diversity in crowd images with incrementally growing CNN . . Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition , , 2018 . . p.3618 - - 3626 . . DOI: 10.1109/CVPR.2018.00381 http://doi.org/10.1109/CVPR.2018.00381 . .

Z Shen , , , Y Xu , , , B Ni , , , 等 . . Crowd counting via adversarial cross-scale consistency pursuit . . IEEE/CVF Conf on Computer Vision and Pattern Recognition , , 2018 . . p.5245 - - 5254 . . DOI: 10.1109/CVPR.2018.00550 http://doi.org/10.1109/CVPR.2018.00550 . .

VA Sindagi , , , VM Patel . . Generating high-quality crowd density maps using contextual pyramid CNNs . . Proc IEEE Int Conf on Computer Vision , , 2017 . . p.1879 - - 1888 . . DOI: 10.1109/ICCV.2017.206 http://doi.org/10.1109/ICCV.2017.206 . .

VA Sindagi , , , VM Patel . . A survey of recent advances in CNN-based single image crowd counting and density estimation . . Patt Recogn Lett , , 2018 . . 107 3 - - 16 . . DOI: 10.1016/j.patrec.2017.07.007 http://doi.org/10.1016/j.patrec.2017.07.007 . .

BB Zhan , , , DN Monekosso , , , P Remagnino , , , 等 . . Crowd analysis: a survey . . Mach Vis Appl , , 2008 . . 19 ( ( 5-6 ): ): 345 - - 357 . . DOI: 10.1007/s00138-008-0132-4 http://doi.org/10.1007/s00138-008-0132-4 . .

C Zhang , , , HS Li , , , XG Wang , , , 等 . . Cross-scene crowd counting via deep convolutional neural networks . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2015 . . p.833 - - 841 . . DOI: 10.1109/CVPR.2015.7298684 http://doi.org/10.1109/CVPR.2015.7298684 . .

C Zhang , , , K Zhang , , , HS Li , , , 等 . . Data-driven crowd understanding: a baseline for a large-scale crowd dataset . . IEEE Trans Multim , , 2016 . . 18 ( ( 6 ): ): 1048 - - 1061 . . DOI: 10.1109/TMM.2016.2542585 http://doi.org/10.1109/TMM.2016.2542585 . .

YY Zhang , , , DS Zhou , , , SQ Chen , , , 等 . . Single-image crowd counting via multi-column convolutional neural network . . Proc IEEE Conf on Computer Vision and Pattern Recognition , , 2016 . . p.589 - - 597 . . DOI: 10.1109/CVPR.2016.70 http://doi.org/10.1109/CVPR.2016.70 . .

Views

Downloads

CSCD

Alert me when the article has been cited

Submit

Tools

Publicity Resources

Aggregated context network for crowd counting

Forget less, count better: a domain-incremental self-distillation learning benchmark for lifelong crowd counting

Related Author

Si-yue YU

Jian PU

Jiaqi GAO

Jingqi LI

Hongming SHAN

Yanyun QU

James Z. WANG

Fei-Yue WANG

Related Institution

School of Computer Science and Technology, East China Normal University

Institute of Science and Technology for Brain-inspired Intelligence, Fudan University

Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University

Shanghai Center for Brain Science and Brain-Inspired Technology

School of Information Science and Technology, Xiamen University

Chat

Address：Zhejiang University Press, 148 Tianmushan Road, Hangzhou, China Postal code：310028
Tel：+86-571-88273162 Email：fitee@zju.edu.cn
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰