基于注意力机制的人群计数方法

doi:10.16265/j.cnki.issn1003-3033.2022.01.017

摘要/Abstract

摘要：

为准确预测固定场景中的人群计数,在人群分析领域,采用一种融合注意力机制的卷积神经网络(CNN)进行人群计数,该模块结合空间域注意力和通道域注意力,空间域注意力可以编码整个图像的像素级上下文信息,以更准确地表达像素级别的密度图,而通道域注意力可以在不同的通道中提取更多的区分特征使网络显著表达人群的局部区域,并在多个公开数据集上进行测试。结果表明:基于注意力机制的人群计数方法可以准确地估计拥挤场景中的人群数量,在平均完全误差和均方误差上均优于CSRNet。

关键词: 注意力机制, 人群计数, 空间域注意力, 通道域注意力, 密度图

Abstract:

In order to accurately predict crowd count in a fixed scene, in the field of crowd analysis, a convolutional neural network (CNN) integrating attention mechanism was adopted, which combined spatial domain attention and channel domain attention. The former could encode pixel-level context information of the entire image to express pixel-level density map more accurately, while the latter could extract more distinguishing features in different channels to significantly express local area of the crowd. Through tests on multiple public data sets, it is found that the crowd counting method based on attention mechanism can accurately estimate number of people in crowded scenes, and it proves better than CSRNet in terms of mean absolute error and mean square error.

Key words: attention mechanism, crowd counting, spatial domain attention, channel domain attention, density map

吴思, 张旭光, 方银锋. 基于注意力机制的人群计数方法[J]. 中国安全科学学报, 2022, 32(1): 127-134.

WU Si, ZHANG Xuguang, FANG Yinfeng. Method of crowd counting based on attention mechanism[J]. China Safety Science Journal, 2022, 32(1): 127-134.

图/表 12

图1

图2

图3

图4

表1

图5

表2

图6

表3

表4

表5

表6

参考文献 33

[1]	ZHAN Cong, KANG Kai, LI Hongsheng, et al. Data-driven crowd understanding: a baseline for a large-scale crowd dataset[J]. IEEE Transactions on Multimedia, 2016, 18(6): 1048-1061. doi: 10.1109/TMM.2016.2542585
[2]	JUNIO J J, MUSSE S R, JUNG C R. Crowd analysis using computer vision techniques[J]. IEEE Signal Processing Magazine, 2010, 27(5): 66-77.
[3]	ZHAN Beibei, MONEKOSSO D, REMAGAINO P, et al. Crowd analysis: a survey[J]. Machine Vision and Applications, 2008, 19(56): 345-357. doi: 10.1007/s00138-008-0132-4
[4]	BENENSON R, OMRAN M, HOSANG J, et al. Ten years of pedestrian detection, what have we learned[C]. European Conference on Computer Vision, 2014: 613-627.
[5]	GAO Chenqiang, LI Pei, ZHANG Yajun, et al. People counting based on head detection combining Adaboost and CNN in crowded surveillance environment[J]. Neurocomputing, 2016, 208(5):108-116. doi: 10.1016/j.neucom.2016.01.097
[6]	SUBBURAMAN V B, DESCAMPS A, CARINCOTTE C. Counting people in the crowd using a generic head detector[C]. 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance, 2012: 470-475.
[7]	RYAN D, FOOKES C, DENMAN S, et al. An evaluation of crowd counting methods, features and regression models[J]. Computer Vision & Image Understanding Cviu, 2015, 130: 1-17.
[8]	徐丹, 代勇, 纪军红. 基于卷积神经网络的驾驶人行为识别方法研究[J]. 中国安全科学学报, 2019, 29(10):12-17. doi: 10.16265/j.cnki.issn1003-3033.2019.10.003
	XU Dan, DAI Yong, JI Junhong. Research on driver behavior recognition method based on convolutional neural network[J]. China Safety Science Journal, 2019, 29(10) : 12-17. doi: 10.16265/j.cnki.issn1003-3033.2019.10.003
[9]	黄平, 文超, 李忠灿, 等. 高速铁路列车晚点时间实时预测的神经网络模型[J]. 中国安全科学学报, 2019, 29(增1):20-26.
	HUANG Ping, WEN Chao, LI Zhongcan, et al. Neural network model for real-time prediction of train delay time on high-speed railway[J]. China Safety Science Journal, 2019, 29(S1) : 20-26. doi: 10.16265/j.cnki.issn1003-3033.2019.S1.005
[10]	戴健非, 杨鹏, 诸利一, 等. 集成PCA和LSTM神经网络的浸润线预测方法[J]. 中国安全科学学报, 2020, 30(3):94-101. doi: 10.16265/j.cnki.issn1003-3033.2020.03.015
	DAI Jianfei, YANG Peng, ZHU Liyi, et al. Infiltration line prediction method integrating PCA and LSTM neural network[J]. China Safety Science Journal, 2020, 30(3) : 94-101. doi: 10.16265/j.cnki.issn1003-3033.2020.03.015
[11]	KRIZHEVSKY A, SUTSKEVER I, GEOFFREY E H, et al. ImageNet classification with deep convolutional neural networks[J]. NIPS, 2012, 26(6): 1097-1105.
[12]	SOURTZINOS P, VELASTIN S A, JARA M, et al. People counting in videos by fusing temporal cues from spatial context-aware convolutional neural networks[C]. European Conference on Computer Vision, 2016: 655-667.
[13]	WALACH E, WOLF L. Learning to count with CNN boosting[C]. European Conference on Computer Vision, 2016: 660-676.
[14]	ZHANG Cong, LI Hongsheng, WANG Xiaogang, et al. Cross-scene crowd counting via deep convolutional neural networks[C]. IEEE Conference on Computer Vision & Pattern Recognition, 2015:833-841.
[15]	ZHANG Yingying, ZHOU Desen, CHEN Siqin, et al. Single-Image crowd counting via multi-column convolutional neural network[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016:589-597.
[16]	CHAN A B, LIANG Z S, VASCONCELOS N. Privacy preserving crowd monitoring: counting people without people models or tracking[C]. 2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008:1-7.
[17]	IDREES H, SALEEMI I, SEIBERT C, et al. Counting in extremely dense crowd images[J]. IEEE, 2013, 51(5): 2547-2554.
[18]	CHEN Ke, LOY Chenchange, XIANG Tony, et al. Feature mining for localised crowd counting[C]. British Machine Vision Conference, 2012:1-11.
[19]	LI Yuhong, ZHANG Xiaofan, CHEN Deming. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 1091-1100.
[20]	BEARDSLEY S A, WARD R L, VAINA L M. A neural network model of spiral-planar motion tuning in MSTd[J]. Vision Research, 2003, 43(5):577-595. doi: 10.1016/S0042-6989(02)00608-9
[21]	SUTSKEVER I, VINYALS O, LE Q V. Sequence to sequence learning with neural networks[J]. Advances in Neural Information Processing Systems, 2016, 9 (3): 3104-3112.
[22]	LU Xiaoqiang, CHEN Xiaoqiang, LI Xuedong, et al. Hierarchical recurrent neural hashing for image retrieval with hierarchical convolutional features[J]. IEEE Trans Image Process, 2017, 27 (1): 106-120. doi: 10.1109/TIP.2017.2755766
[23]	VASWANI A, SHAZEER N, PARMAR, et al. Attention is all you need, advances in neural information processing systems[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 5998-6008.
[24]	CHEN Long, ZHANG Hanwang, XIAO Jun, et al. SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017:5659-5667.
[25]	WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[J]. The IEEE Conference on Computer Vision and Pattern Recognition, 2018, 9(3):7794-7803.
[26]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[J]. European Conference on Computer Vision, 2018, 25(7): 3-19.
[27]	ROY A G, NAVAB N, WACHINGER C. Concurrent spatial and channel squeeze & excitation in fully convolutional networks[J]. Springer, Cham, 2018. 12(6): 421-429.
[28]	WANG Luyang, YIN Baoqun, TANG Xiao, et al. Removing background interference for crowd counting via de-background detail convolutional network[J]. Neurocomputing, 2019, 332(7):360-371. doi: 10.1016/j.neucom.2018.12.047
[29]	SAM D B, SURYA S, BABU R V. Switching convolutional neural network for crowd counting[C]. Computer Vision & Pattern Recognition. IEEE, 2017: 5744-5752.
[30]	CAO Xinkun, WANG Zhipeng, ZHAO Yanyun et al. Scale aggregation network for accurate and efficient crowd counting[C]. European Conference on Computer Vision, 2018:734-750.
[31]	PHAM V Q, KPZAKAYA T, YAMAGUCHI O, et al. COUNT forest: CO-voting uncertain number of targets using random forest for crowd density estimation[C]. 2015 IEEE International Conference on Computer Vision (ICCV), 2015: 3253-3261.
[32]	ZHANG Youmei, ZHOU Chunluan, CHANG Faliang, et al. Multi-resolution attention convolutional neural network for crowd counting[J]. Neurocomputing, 2019, 329:144-152. doi: 10.1016/j.neucom.2018.10.058
[33]	JIANG Liu, GAO Chenqian, MENG Deyu, et al. DecideNet: counting varying density crowds through attention guided detection and density estimation[C]. 2018 IEEE Conference on Computer Vision and Pattern Recognition, 2018: 5197-5206.

数据集	图片数/幅	分辨率	人数
数据集	图片数/幅	分辨率	最少	平均	最大	总体
Mall	2 000	640×480	13	31	53	62 325
Shanghai Part_A	482	非统一	33	501	3 139	241 667
Shanghai Part_B	716	768×1 024	9	123	578	88 488

数据集	训练集/幅	测试集/幅
Mall	800	1 200
Shanghai Part_A	300	182
Shanghai Part_B	400	316

方法	MAE	MSE
MCNN^[15]	110.2	173.2
方法	MAE	MSE
Switch CNN^[29]	90.4	135.0
CSRNet^[19]	68.2	115.0
SANet^[30]	67.0	104.5
SCANet	66.0	104.3

方法	MAE	MSE
MCNN^[15]	26.4	41.3
Switch CNN^[29]	21.6	33.4
CSRNet^[19]	10.0	16.0
SANet^[30]	8.4	16.0
SCANet	8.4	13.9

方法	MAE	MSE
Count forest^[31]	2.50	10.00
MCNN^[15]	2.24	8.50
方法	MAE	MSE
L.Liu eta^[32]	1.72	2.10
Decidenet^[33]	1.52	1.90
SCANet	1.22	1.58