Method of crowd counting based on attention mechanism

doi:10.16265/j.cnki.issn1003-3033.2022.01.017

Abstract

Abstract:

In order to accurately predict crowd count in a fixed scene, in the field of crowd analysis, a convolutional neural network (CNN) integrating attention mechanism was adopted, which combined spatial domain attention and channel domain attention. The former could encode pixel-level context information of the entire image to express pixel-level density map more accurately, while the latter could extract more distinguishing features in different channels to significantly express local area of the crowd. Through tests on multiple public data sets, it is found that the crowd counting method based on attention mechanism can accurately estimate number of people in crowded scenes, and it proves better than CSRNet in terms of mean absolute error and mean square error.

Key words: attention mechanism, crowd counting, spatial domain attention, channel domain attention, density map

WU Si, ZHANG Xuguang, FANG Yinfeng. Method of crowd counting based on attention mechanism[J]. China Safety Science Journal, 2022, 32(1): 127-134.

Figures/Tables 12

Fig.1

Fig.2

Fig.3

Fig.4

Tab.1

Fig.5

Tab.2

Fig.6

Tab.3

Tab.4

Tab.5

Tab.6

References 33

[1]	ZHAN Cong, KANG Kai, LI Hongsheng, et al. Data-driven crowd understanding: a baseline for a large-scale crowd dataset[J]. IEEE Transactions on Multimedia, 2016, 18(6): 1048-1061. doi: 10.1109/TMM.2016.2542585
[2]	JUNIO J J, MUSSE S R, JUNG C R. Crowd analysis using computer vision techniques[J]. IEEE Signal Processing Magazine, 2010, 27(5): 66-77.
[3]	ZHAN Beibei, MONEKOSSO D, REMAGAINO P, et al. Crowd analysis: a survey[J]. Machine Vision and Applications, 2008, 19(56): 345-357. doi: 10.1007/s00138-008-0132-4
[4]	BENENSON R, OMRAN M, HOSANG J, et al. Ten years of pedestrian detection, what have we learned[C]. European Conference on Computer Vision, 2014: 613-627.
[5]	GAO Chenqiang, LI Pei, ZHANG Yajun, et al. People counting based on head detection combining Adaboost and CNN in crowded surveillance environment[J]. Neurocomputing, 2016, 208(5):108-116. doi: 10.1016/j.neucom.2016.01.097
[6]	SUBBURAMAN V B, DESCAMPS A, CARINCOTTE C. Counting people in the crowd using a generic head detector[C]. 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance, 2012: 470-475.
[7]	RYAN D, FOOKES C, DENMAN S, et al. An evaluation of crowd counting methods, features and regression models[J]. Computer Vision & Image Understanding Cviu, 2015, 130: 1-17.
[8]	徐丹, 代勇, 纪军红. 基于卷积神经网络的驾驶人行为识别方法研究[J]. 中国安全科学学报, 2019, 29(10):12-17. doi: 10.16265/j.cnki.issn1003-3033.2019.10.003
	XU Dan, DAI Yong, JI Junhong. Research on driver behavior recognition method based on convolutional neural network[J]. China Safety Science Journal, 2019, 29(10) : 12-17. doi: 10.16265/j.cnki.issn1003-3033.2019.10.003
[9]	黄平, 文超, 李忠灿, 等. 高速铁路列车晚点时间实时预测的神经网络模型[J]. 中国安全科学学报, 2019, 29(增1):20-26.
	HUANG Ping, WEN Chao, LI Zhongcan, et al. Neural network model for real-time prediction of train delay time on high-speed railway[J]. China Safety Science Journal, 2019, 29(S1) : 20-26. doi: 10.16265/j.cnki.issn1003-3033.2019.S1.005
[10]	戴健非, 杨鹏, 诸利一, 等. 集成PCA和LSTM神经网络的浸润线预测方法[J]. 中国安全科学学报, 2020, 30(3):94-101. doi: 10.16265/j.cnki.issn1003-3033.2020.03.015
	DAI Jianfei, YANG Peng, ZHU Liyi, et al. Infiltration line prediction method integrating PCA and LSTM neural network[J]. China Safety Science Journal, 2020, 30(3) : 94-101. doi: 10.16265/j.cnki.issn1003-3033.2020.03.015
[11]	KRIZHEVSKY A, SUTSKEVER I, GEOFFREY E H, et al. ImageNet classification with deep convolutional neural networks[J]. NIPS, 2012, 26(6): 1097-1105.
[12]	SOURTZINOS P, VELASTIN S A, JARA M, et al. People counting in videos by fusing temporal cues from spatial context-aware convolutional neural networks[C]. European Conference on Computer Vision, 2016: 655-667.
[13]	WALACH E, WOLF L. Learning to count with CNN boosting[C]. European Conference on Computer Vision, 2016: 660-676.
[14]	ZHANG Cong, LI Hongsheng, WANG Xiaogang, et al. Cross-scene crowd counting via deep convolutional neural networks[C]. IEEE Conference on Computer Vision & Pattern Recognition, 2015:833-841.
[15]	ZHANG Yingying, ZHOU Desen, CHEN Siqin, et al. Single-Image crowd counting via multi-column convolutional neural network[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016:589-597.
[16]	CHAN A B, LIANG Z S, VASCONCELOS N. Privacy preserving crowd monitoring: counting people without people models or tracking[C]. 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008:1-7.
[17]	IDREES H, SALEEMI I, SEIBERT C, et al. Counting in extremely dense crowd images[J]. IEEE, 2013, 51(5): 2547-2554.
[18]	CHEN Ke, LOY Chenchange, XIANG Tony, et al. Feature mining for localised crowd counting[C]. British Machine Vision Conference, 2012:1-11.
[19]	LI Yuhong, ZHANG Xiaofan, CHEN Deming. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 1091-1100.
[20]	BEARDSLEY S A, WARD R L, VAINA L M. A neural network model of spiral-planar motion tuning in MSTd[J]. Vision Research, 2003, 43(5):577-595. doi: 10.1016/S0042-6989(02)00608-9
[21]	SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[J]. Advances in Neural Information Processing Systems, 2016, 9 (3): 3104-3112.
[22]	LU Xiaoqiang, CHEN Xiaoqiang, LI Xuedong, et al. Hierarchical recurrent neural hashing for image retrieval with hierarchical convolutional features[J]. IEEE Trans Image Process, 2017, 27 (1): 106-120. doi: 10.1109/TIP.2017.2755766
[23]	VASWANI A, SHAZEER N, PARMAR, et al. Attention is all you need, advances in neural information processing systems[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5998-6008.
[24]	CHEN Long, ZHANG Hanwang, XIAO Jun, et al. SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017:5659-5667.
[25]	WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[J]. The IEEE Conference on Computer Vision and Pattern Recognition, 2018, 9(3):7794-7803.
[26]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[J]. European Conference on Computer Vision, 2018, 25(7): 3-19.
[27]	ROY A G, NAVAB N, WACHINGER C. Concurrent spatial and channel squeeze & excitation in fully convolutional networks[J]. Springer, Cham, 2018. 12(6): 421-429.
[28]	WANG Luyang, YIN Baoqun, TANG Xiao, et al. Removing background interference for crowd counting via de-background detail convolutional network[J]. Neurocomputing, 2019, 332(7):360-371. doi: 10.1016/j.neucom.2018.12.047
[29]	SAM D B, SURYA S, BABU R V. Switching convolutional neural network for crowd counting[C]. Computer Vision & Pattern Recognition. IEEE, 2017: 5744-5752.
[30]	CAO Xinkun, WANG Zhipeng, ZHAO Yanyun et al. Scale aggregation network for accurate and efficient crowd counting[C]. European Conference on Computer Vision, 2018:734-750.
[31]	PHAM V Q, KPZAKAYA T, YAMAGUCHI O, et al. COUNT forest: CO-voting uncertain number of targets using random forest for crowd density estimation[C]. 2015 IEEE International Conference on Computer Vision (ICCV), 2015: 3253-3261.
[32]	ZHANG Youmei, ZHOU Chunluan, CHANG Faliang, et al. Multi-resolution attention convolutional neural network for crowd counting[J]. Neurocomputing, 2019, 329:144-152. doi: 10.1016/j.neucom.2018.10.058
[33]	JIANG Liu, GAO Chenqian, MENG Deyu, et al. DecideNet: counting varying density crowds through attention guided detection and density estimation[C]. 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 5197-5206.

数据集	图片数/幅	分辨率	人数
数据集	图片数/幅	分辨率	最少	平均	最大	总体
Mall	2 000	640×480	13	31	53	62 325
Shanghai Part_A	482	非统一	33	501	3 139	241 667
Shanghai Part_B	716	768×1 024	9	123	578	88 488

数据集	训练集/幅	测试集/幅
Mall	800	1 200
Shanghai Part_A	300	182
Shanghai Part_B	400	316

方法	MAE	MSE
MCNN^[15]	110.2	173.2
方法	MAE	MSE
Switch CNN^[29]	90.4	135.0
CSRNet^[19]	68.2	115.0
SANet^[30]	67.0	104.5
SCANet	66.0	104.3

方法	MAE	MSE
MCNN^[15]	26.4	41.3
Switch CNN^[29]	21.6	33.4
CSRNet^[19]	10.0	16.0
SANet^[30]	8.4	16.0
SCANet	8.4	13.9

方法	MAE	MSE
Count forest^[31]	2.50	10.00
MCNN^[15]	2.24	8.50
方法	MAE	MSE
L.Liu eta^[32]	1.72	2.10
Decidenet^[33]	1.52	1.90
SCANet	1.22	1.58