基于ST-GCN的空中交通管制员不安全行为识别

doi:10.16265/j.cnki.issn1003-3033.2023.05.1396

摘要/Abstract

摘要：

为预防和监督空中交通管制(ATC)工作中的违章行为,利用智能视频分析技术,研究适用于管制员坐姿工作的不安全行为识别模型。首先,分析管制员不安全工作行为的隐蔽性特征,总结5种典型管制员不安全行为,包括伸懒腰、瞌睡、低头入睡、歪头入睡和半躺入睡,并构建管制员不安全工作状态视频数据集(CUWS);其次,提出一种能描述管制员坐姿的骨架关键点拓展算法,基于时空图卷积网络(ST-GCN)搭建适用于管制员坐姿与腿部遮蔽情况下的不安全行为识别模型ATC-ST-GCN,并给出管制员不安全行为识别的工作流程;最后,利用CUWS数据集进行ATC-ST-GCN模型的训练和测试,并利用管制室实际监控视频开展验证试验。结果表明:该模型能够在有限验证数据集上实现5种典型不安全行为识别,准确率达到93.65%。试验结果证明该模型具有一定的科学性与有效性。

关键词: 时空图卷积网络(ST-GCN), 空中交通管制(ATC), 不安全行为, 管制员, 行为识别

Abstract:

In order to prevent and supervise the violation behavior in ATC, an unsafe behavior recognition model suitable for controllers' sitting posture was studied by intelligent video analysis technology. Firstly, the hidden characteristics of unsafe working behaviors of air traffic controllers were analyzed, and five typical unsafe behaviors of air traffic controllers were summarized, including stretching, dozing, falling asleep, crooked asleep and half lying asleep, and the video data set of controllers unsafe working status (CUWS) was constructed. Secondly, a skeleton key point expansion algorithm that can describe the controller's sitting posture was proposed. Based on ST-GCN, an ATC-ST-GCN model suitable for sitting posture and legs hidden from view was built,and the workflow of the controller's unsafe behavior recognition was given. Finally, the ATC-ST-GCN model was trained and tested using the CUWS dataset, and the verification test was carried out using the actual surveillance video of the control room. The results show that the model can identify five typical unsafe behaviors on the limited validation dataset, and the accuracy rate reaches 93.65%. The experimental results show that the model is scientific and effective.

Key words: spatial temporal graph convolutional network (ST-GCN), air traffic control(ATC), unsafe behavior, air traffic controller, behavior recognition

王超, 徐楚昕, 董杰, 王志锋. 基于ST-GCN的空中交通管制员不安全行为识别[J]. 中国安全科学学报, 2023, 33(5): 42-48.

WANG Chao, XU Chuxin, DONG Jie, WANG Zhifeng. Unsafe behavior recognition of air traffic controllers based on ST-GCN[J]. China Safety Science Journal, 2023, 33(5): 42-48.

图/表 11

图1

表1

图2

图3

图4

图5

图6

图7

图8

表2

图9

参考文献 16

[1]	DOC9966-2016, Manual for the oversight of fatigue management approaches[S].
[2]	靳慧斌, 张静, 吕川. HRV在管制员疲劳检测中的适用性[J]. 北京航空航天大学学报, 2018, 44(11): 2292-2298.
	JIN Huibin, ZHANG Jing, LYU Chuan. Application of HRV in air traffic controllers' fatigue detection[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(11): 2292-2298.
[3]	王莉莉, 朱敏. 基于脑电数据的管制架次对管制员疲劳影响研究[J]. 中国安全科学学报, 2021, 31(2): 173-178. doi: 10.16265/j.cnki.issn 1003-3033.2021.02.024
	WANG Lili, ZHU Min. Research on influence of controlled sorties on controllers' fatigue based on EEG data[J]. China Safety Science Journal, 2021, 31(2): 173-178. doi: 10.16265/j.cnki.issn 1003-3033.2021.02.024
[4]	JI Yingyu, WANG Shigang, LU Yang, et al. Eye and mouth state detection algorithm based on contour feature extraction[J]. Journal of Electronic Imaging, 2018, 27(5): DOI: 10.1117/1.JEI.27.5.051205. doi: 10.1117/1.JEI.27.5.051205
[5]	YANG Hao, LIU Li, MIN Weidong, et al. Driver yawning detection based on subtle facial action recognition[J]. IEEE Transactions on Multimedia, 2020, 23: 572-583. doi: 10.1109/TMM.2020.2985536
[6]	HUANG Rui, WANG Yan, LI Zijian, et al. RF-DCM: Multi-granularity deep convolutional model based on feature recalibration and fusion for driver fatigue detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(1): 630-640. doi: 10.1109/TITS.2020.3017513
[7]	卜建, 刘银鑫, 王艳军. 空中交通管制员的眼动行为与疲劳关系[J]. 航空学报, 2017, 38(增1): 57-62.
	BU Jian, LIU Yinxin, WANG Yanjun. Relationship between air traffic controllers' eye movement and fatigue[J]. Acta Aeronautica et Astronautica Sinica, 2017, 38(S1): 57-62.
[8]	刘忠育. 基于深度学习的矿工不安全行为识别方法研究[D]. 徐州: 中国矿业大学, 2021.
	LIU Zhongyu. Research on recognition methods of miners' unsafe behavior based on deep learning[D]. Xuzhou: China University of Mining and Technology, 2021.
[9]	崔铁军, 王凌霄. YOLOv4目标检测算法在煤矿工人口罩佩戴监测工作中的应用研究[J]. 中国安全生产科学技术, 2021, 17(10): 66-71.
	CUI Tiejun, WANG Lingxiao. Research on application of YOLOv4 object detection algorithm in monitoring on masks wearing of coal miners[J]. Journal of Safety Science and Technology, 2021, 17(10): 66-71.
[10]	刘耀, 焦双健. ST-GCN在建筑工人不安全动作识别中的应用[J]. 中国安全科学学报, 2022, 32(4): 30-35. doi: 10.16265/j.cnki.issn1003-3033.2022.04.005
	LIU Yao, JIAO Shuangjian. Application of ST-GCN in unsafe action identification of construction workers[J]. China Safety Science Journal, 2022, 32(4): 30-35. doi: 10.16265/j.cnki.issn1003-3033.2022.04.005
[11]	FANG Weili, LOVE P E D, LUO Hanbin, et al. Computer vision for behaviour-based safety in construction: a review and future directions[J]. Advanced Engineering Informatics, 2020, 43: DOI: 10.1016/j.aei.2019.100980. doi: 10.1016/j.aei.2019.100980
[12]	KWON Seongkyung, SEO Jihwan, YUN Junyoung, et al. Driving behavior classification and sharing system using CNN-LSTM approaches and V2X communication[J]. Applied Sciences, 2021, 11(21): 8856-8878. doi: 10.3390/app11198856
[13]	QI Mengshi, WANG Yunhong, QIN Jie, et al. StagNet: an attentive semantic RNN for group activity and individual action recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(2): 549-565. doi: 10.1109/TCSVT.76
[14]	DING Lieyun, FANG Weili, LUO Hanbin, et al. A deep hybrid learning model to detect unsafe behavior: integrating convolution neural networks and long short-term memory[J]. Automation in Construction, 2018, 86: 118-124. doi: 10.1016/j.autcon.2017.11.002
[15]	温廷新, 王贵通, 孔祥博, 等. 基于迁移学习与残差网络的矿工不安全行为识别[J]. 中国安全科学学报, 2020, 30(3):41-46. doi: 10.16265/j.cnki.issn1003-3033.2020.03.007
	WEN Tingxin, WANG Guitong, KONG Xiangbo, et al. Identification of miners' unsafe behaviors based on transfer learning and residual network[J]. China Safety Science Journal, 2020, 30(3): 41-46. doi: 10.16265/j.cnki.issn1003-3033.2020.03.007
[16]	YAN Sijie, XIONG Yuanjun, LIN Dahua. Spatial temporal graph convolutional networks for skeleton-based action recognition[C]. Thirty-second AAAI Conference on Artificial Intelligence, 2018: 8561-8568.

序号	不安全工作行为	行为表现
1	伸懒腰	在5~7 s内出现双臂打开行为
2	瞌睡状态	在5~7 s时间段内频繁点头
3	低头入睡	保持低头状态持续5~7 s
4	歪头入睡	头部靠在椅背上保持静止5~7 s
5	半躺入睡	上半身躺在椅子里保持静止5~7 s

识别模型	Top-1准确率	Top-5准确率
ST-GCN^[16]	71.73	96.83
ATC-ST-GCN	93.65	100.00