Method of coupling identifying unsafe behaviors of underground personnel based on dual-model algorithm

doi:10.16265/j.cnki.issn1003-3033.2026.02.1372

Abstract

Abstract:

In order to prevent safety accidents caused by unsafe behaviors of underground personnel and to ensure their safety, by utilizing advanced machine vision and computer technologies, the traditional YOLOv5s algorithm and OpenPose algorithm target detection models were improved, and a dual-model coupled algorithm for identifying unsafe behaviors of underground personnel was proposed. Through statistical analysis of the most common unsafe behaviors in current underground coal mines, the unsafe behaviors of miners were classified, including item-related, action-related, and area-related unsafe behaviors. According to the characteristics of miners' unsafe behaviors, the improved YOLOv5s algorithm and the OpenPose algorithm were coupled for recognition, and training and verification were conducted on public datasets and self-built datasets. The results show that compared with the current mainstream methods, the dual-model coupled recognition method has a significant improvement in recognition accuracy on self-built datasets and public datasets, with an increase of 5% to 10%, and can quickly and effectively identify unsafe behaviors of underground personnel.

Key words: underground personnel, unsafe behaviors, dual-model algorithm, coupling identifying, object detection

CLC Number:

X936

TAN Bo, SUI Longkun, KE Wei, LIU Yan, ZHU Quanjie, HE Ning. Method of coupling identifying unsafe behaviors of underground personnel based on dual-model algorithm[J]. China Safety Science Journal, 2026, 36(2): 18-26.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Table 1

Fig.7

Fig.8

Fig.9

Table 2

References 26

[1]	李瑞群. 煤矿员工行为安全管控研究[J]. 中国安全科学学报, 2022, 32(增2):26-31.
	LI Ruiqun. Research on behavioral safety control of employees in coal mines[J]. China Safety Science Journal, 2022, 32(S2): 26-31. doi: 10.16265/j.cnki.issn1003-3033.2022.S2.0104
[2]	彭海兵, 李男男. 神东补连塔煤矿不安全行为管控体系建设[J]. 煤矿安全, 2021, 52(1):247-251.
	PENG Haibing, LI Nannan. Management and control system construction of unsafe behaviors in Shendong Bulianta coal mine[J]. Safety in Coal Mines, 2021, 52(1): 247-251.
[3]	李磊, 田水承. 煤矿工人不安全行为组合干预仿真研究[J]. 中国安全科学学报, 2016, 26(7):23-28.
	LI Lei, TIAN Shuicheng. Simulation research on combination intervention in coal miners' unsafe behaviors[J]. China Safety Science Journal, 2016, 26(7): 23-28.
[4]	CAO Xiangang, ZHANG Chiyu, WANG Peng, et al. Unsafe mining behavior identification method based on an improved ST-GCN[J]. Sustainability, 2023, 15(2): DOI:10.3390/su15021041.
[5]	王立兵, 任予鑫, 马昆, 等. 多源数据融合智能识别煤矿山场景特征AI模型[J]. 煤炭学报, 2023, 48(12):4617-4631.
	WANG Libing, REN Yuxin, MA Kun, et al. AI model for intelligent recognition of coal mine scene features through multi-source data fusion[J]. Journal of China Coal Society, 2023, 48(12): 4617-4631.
[6]	王均利, 李佳悦, 李秉天, 等. 基于深度学习的煤矿井下低光照人脸检测方法[J]. 工矿自动化, 2023, 49(11):145-150.
	WANG Junli, LI Jiayue, LI Bingtian, et al. Deep learning-based face detection method under low illumination conditions in coal mines[J]. Journal of Mine Automation, 2023, 49(11): 145-150.
[7]	杨赛烽, 高彬, 冯仕民, 等. 基于三维骨骼节点的人体行为识别方法[J]. 传感技术学报, 2018, 31(12):1834-1841.
	YANG Saifeng, GAO Bin, FENG Shimin, et al. Human behavior recognition method based on 3D skeleton nodes[J]. Chinese Journal of Sensors and Actuators, 2018, 31(12): 1834-1841.
[8]	鞠春雷, 聂方超, 刘文岗, 等. 基于长短期记忆网络的矿工不安全行为研究[J]. 煤矿安全, 2020, 51(9):260-264.
	JU Chunlei, NIE Fangchao, LIU Wengang, et al. Research on miners' unsafe behavior based on long and short term memory[J]. Safety in Coal Mines, 2020, 51(9): 260-264.
[9]	董昕宇, 师杰, 张国英. 基于参数轻量化的井下人体实时检测算法[J]. 工矿自动化, 2021, 47(6):71-78.
	DONG Xinyu, SHI Jie, ZHANG Guoying. Realtime detection algorithm of underground human body based on lightweight parameters[J]. Industry and Mine Automation, 2021, 47(6): 71-78.
[10]	LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot multiBox detector[C]. European Conference on Computer Vision. Cham: Springer International Publishing, 2016: 21-37.
[11]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified real-time object detection[C]. 2016 IEEE Conference on Computer Visionand Pattern Recognition (CVPR), 2016: 779-788.
[12]	李梦荷, 许宏吉, 石磊鑫, 等. 基于骨骼关键点检测的多人行为识别[J]. 计算机科学, 2021, 48(4):138-143. doi: 10.11896/jsjkx.200300042
	LI Menghe, XU Hongji, SHI Leixin, et al. Multi-person activity recognition based on bone key points detection[J]. Computer Science, 2021, 48(4): 138-143. doi: 10.11896/jsjkx.200300042
[13]	张磊, 李熙尉, 燕倩如, 等. 基于改进YOLOv5s的综采工作面人员检测算法[J]. 中国安全科学学报, 2023, 33(7):82-89. doi: 10.16265/j.cnki.issn1003-3033.2023.07.2226
	ZHANG Lei, LI Xiwei, YAN Qianru, et al. Personnel detection algorithm in fully mechanized coal face based on improved YOLOv5s[J]. China Safety Science Journal, 2023, 33(7): 82-89. doi: 10.16265/j.cnki.issn1003-3033.2023.07.2226
[14]	韩康, 李敬兆, 陶荣颖. 基于改进YOLOv7和ByteTrack的煤矿关键岗位人员不安全行为识别[J]. 工矿自动化, 2024, 50(3):82-91.
	HAN Kang, LI Jingzhao, TAO Rongying. Recognition of unsafe behaviors of key position personnel in coal mines based on improved YOLOv7 and ByteTrack[J]. Journal of Mine Automation, 2024, 50(3): 82-91.
[15]	阮进林, 高鹏, 孙源, 等. 基于轻量级OpenPose的井下人员行为检测算法研究[J]. 煤炭工程, 2024, 56(4):150-156.
	RUAN Jinlin, GAO Peng, SUN Yuan, et al. Detection algorithm of underground personnel behavior based on lightweight OpenPose[J]. Coal Engineering, 2024, 56(4): 150-156.
[16]	田水承, 杨鹏飞, 唐凯, 等. 煤矿高概率险兆事件界定及其研究意义[J]. 西安科技大学学报, 2020, 40(4):566-571.
	TIAN Shuicheng, YANG Pengfei, TANG Kai, et al. Definition and research significance of high probability near-miss in coal mine[J]. Journal of Xi'an University of Science and Technology, 2020, 40(4): 566-571.
[17]	阴东玲, 陈兆波, 曾建潮, 等. 煤矿作业人员不安全行为的影响因素分析[J]. 中国安全科学学报, 2015, 25(12):151-156.
	YIN Dongling, CHEN Zhaobo, ZENG Jianchao, et al. Analysis of factors affecting coal mine operators' unsafe acts[J]. China Safety Science Journal, 2015, 25(12): 151-156.
[18]	马跃, 傅贵, 杨卓明. 矿工不安全行为分类及控制对策研究[J]. 煤矿安全, 2014, 45(9):235-237,241.
	MA Yue, FU Gui, YANG Zhuoming. Classification and control countermeasure of miner unsafe behavior[J]. Safety in Coal Mines, 2014, 45(9): 235-237, 241.
[19]	YAO Wei, WANG Aiming, NIE Yifan, et al. Study on the recognition of coal miners' unsafe behavior and status in the hoist cage based on machine vision[J]. Sensors, 2023, 23(21): DOI: 10.3390/s23218794.
[20]	LI Lijuan, ZHANG Peng, YANG Shipin, et al. YOLOv5-SFE: an algorithm fusing spatio-temporal features for detecting and recognizing workers' operating behaviors[J]. Advanced Engineering Informatics, 2023, 56: DOI: 10.1016/j.aei.2023.101988.
[21]	FU Liang. Construction site safety helmet wearing detection method based on improved YOLOv5[J]. Journal of Physics: Conference Series, IOP Publishing, 2023, 2560(1): DOI:10.1088/1742-6596/2560/1/012042.
[22]	LIU Yiping, JIANG Benchi, HE Huan. et al. Helmet wearing detection algorithm based on improved YOLOv5[J]. Scientific Reports, 2024, 14: DOI: 10.1038/s41598-024-58800-6.
[23]	罗国富, 王源, 李浩, 等. 基于改进YOLOv5s的智能车间工人不安全行为实时检测方法[J]. 计算机集成制造系统, 2024, 30(5):1610-1619.
	LUO Guofu, WANG Yuan, LI Hao, et al. Unsafe behavior real time detection method of intelligent workshop workers based on improved YOLOv5s[J]. Computer Integrated Manufacturing Systems, 2024, 30(5): 1610-1619.
[24]	CAO Zhe, GINES M H, TOMAS S, et al. OpenPose: real-time multi-person 2D pose estimation using part affinity fields[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(1): 172-186. doi: 10.1109/TPAMI.34
[25]	王鸿, 陈明举, 熊兴中, 等. 基于OpenPose与AT-STGCN的电力作业人员行为识别技术[J]. 四川轻化工大学学报:自然科学版, 2023, 36(4):61-70.
	WANG Hong, CHEN Mingju, XIONG Xingzhong, et al. Behavior recognition technology of power operator based on OpenPose and AT-STGCN[J]. Journal of Sichuan University of Science & Engineering: Natural Science Edition, 2023, 36(4): 61-70.
[26]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.

不安全行为分类	矿井场景样本数目	非矿井场景样本数目
物品	4 300	4 900
动作	7 200	6 200
区域	3 000	5 400
总计	14 500	16 500

模型	数据集	mAP@0.5/%	mAP@0.5: 0.95/%	参数量/ 10⁷个	时间/ ms	FPS/ (帧·s^-1)
SSD	COCO数据集	63.5	41.3	4.79	52	50
SSD	自制数据集	74.5	44.8	3.96	49	53
YOLOv5s	COCO数据集	74.7	43.5	4.43	47	54
YOLOv5s	自制数据集	82.8	46.9	3.84	41	59
YOLOv7	COCO数据集	79.6	45.7	3.78	44	56
YOLOv7	自制数据集	86.4	48.5	3.54	39	57
YOLOv7-tiny	COCO数据集	85.2	45.3	3.88	44	59
YOLOv7-tiny	自制数据集	89.4	49.8	3.62	40	63
OpenPose-MobileNetV1	COCO数据集	90.2	51.3	3.47	42	64
OpenPose-MobileNetV1	自制数据集	93.4	54.4	2.85	41	68
OpenPose-LSTM	COCO数据集	91.2	52.7	3.13	39	60
OpenPose-LSTM	自制数据集	92.6	53.6	3.02	35	66
文中方法	COCO数据集	93.2	55.3	2.58	35	65
文中方法	自制数据集	96.3	58.1	2.76	33	67