基于计算机视觉的人员疲劳状态与不安全行为识别

doi:10.16265/j.cnki.issn1003-3033.2025.03.0749

摘要/Abstract

摘要：

以提高塔吊操作的安全性和效率为案例,提出一种综合识别疲劳状态和不安全行为的方法,以实时发现驾驶员可能存在的安全隐患。通过摄像头捕获实时视频流,并对视频进行分析和预处理,提取关键信息,用于识别后续的疲劳和不安全行为;在疲劳状态识别方面,采用基于眼睛和嘴部状态的分析方法,重点监测眼睛开闭状态、眨眼频率及哈欠次数等生理指标;在不安全行为识别方面,结合计算机视觉与深度学习技术,实时检测驾驶员的潜在危险操作,从而确保及时发现安全风险。结果表明:经过优化后的YOLOv5-高效通道关注(ECA)模型在疲劳状态和不安全行为识别中性能得到显著提升;模型在测试集上的精确率和召回率均超过90%,展现出良好的识别能力。

关键词: 计算机视觉, 疲劳状态, 不安全行为状态, 塔吊驾驶员, YOLOv5

Abstract:

Taking improving the safety and efficiency of tower crane operation as an example, a method of integrated identification of fatigue state and unsafe behavior was proposed in order to detect the potential safety hazards of drivers in real time. A live video stream was captured via a camera, and the video was analyzed and pre-processed to extract critical information for identifying subsequent fatigue and unsafe behavior. In terms of fatigue state recognition, the analysis method based on the state of eyes and mouth was used to monitor the physiological indicators such as the state of eyes opening and closing, the blink frequency and yawn frequency. In terms of unsafe behavior identification, computer vision and deep learning technology were combined to detect the potential dangerous operations of drivers in real time, thus ensuring timely detection of safety risks. The results show that the performance of the optimized YOLOv5-ECA(Efficient Channel Attention) model is significantly improved in fatigue state and unsafe behavior recognition. The accuracy rate and recall rate of the model on the test set are more than 90%, showing good recognition ability.

Key words: computer vision, fatigue state, unsafe behavior detection, tower crane driver, YOLOv5

中图分类号:

X912安全生理学

李华, 吴立舟, 钟兴润, 郭粮玮, 崔煜馨. 基于计算机视觉的人员疲劳状态与不安全行为识别[J]. 中国安全科学学报, 2025, 35(3): 28-35.

LI Hua, WU Lizhou, ZHONG Xingrun, GUO Liangwei, CUI Yuxin. Recognition of personnel fatigue state and unsafe behavior based on computer vision[J]. China Safety Science Journal, 2025, 35(3): 28-35.

图/表 11

图1

图2

图3

图4

表1

图5

图6

表2

图7

图8

图9

参考文献 13

[1]	SHAIK M. A systematic review on detection and prediction of driver drowsiness[J]. Transportation Research Interdisciplinary Perspectives, 2023, 21: DOI:10.1016/j.trip.2023.100864.
[2]	MEHMOOD L, LI Heng, UMER W, et al. Non-invasive detection of mental fatigue in construction equipment operators through geometric measurements of facial features[J]. Journal of Safety Research, 2024, 89: 234-250. doi: 10.1016/j.jsr.2024.01.013 pmid: 38858047
[3]	KUWAHARA A, NISHIKAWA K, HIRAKAWA R, et al. Eye fatigue estimation using blink detection based on eye aspect ratio mapping(EARM)[J]. Cognitive Robotics, 2022, 2: 50-59.
[4]	LIU Pengkun, CHI Hunglin, LI Xiao, et al. Effects of dataset characteristics on the performance of fatigue detection for crane operators using hybrid deep neural networks[J]. Automation in Construction, 2021, 132: DOI:10.1016/j.autcon.2021.103901.
[5]	孙世梅, 孙祖航, 冯子阳, 等. 基于行为安全“2-4”模型理论的建筑施工事故行为原因分析[J]. 中国安全科学学报, 2023, 33(11): 30-37. doi: 10.16265/j.cnki.issn1003-3033.2023.11.2450
	SUN Shimei, SUN Zuhang, FENG Ziyang, et al. Behavioral causes analysis of construction accidents based on behavior-based accident causation 24Model theory[J]. China Safety Science Journal, 2023, 33(11): 30-37. doi: 10.16265/j.cnki.issn1003-3033.2023.11.2450
[6]	蔡闯闯, 刘庆华. 基于多特征数据融合的疲劳驾驶检测研究[J]. 江苏科技大学学报:自然科学版, 2023, 37(5): 52-57.
	CAI Chuangchuang, LIU Qinghua. Research on fatigue driving detection based on multi-feature data fusion[J]. Journal of Jiangsu Normal University: Natural Science Edition, 2023, 37(5): 52-57.
[7]	辛嵩, 宋明达, 王泽明, 等. 针对特定驾驶员的疲劳驾驶检测方法[J]. 安全与环境学报, 2023, 23(1): 147-152.
	XIN Song, SONG Mingda, WANG Zeming, et al. Fatigue driving detection method for specific drivers[J]. Journal of Safety and Environment, 2023, 23(1): 147-152.
[8]	王红君, 白浩, 赵辉, 等. 基于计算机视觉的驾驶员疲劳状态检测预警技术[J]. 科学技术与工程, 2022, 22(12): 4887-4894.
	WANG Hongjun, BAI Hao, ZHAO Hui, et al. Driver fatigue state detection and early warning technology based on computer vision[J]. Science Technology and Engineering, 2022, 22(12): 4887-4894.
[9]	方浩杰, 董红召, 林少轩, 等. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报:工学版, 2023, 57(7): 1287-1296.
	FANG Haojie, DONG Hongzhao, LIN Shaoxuan, et al. Driver fatigue state detection method based on multi-feature fusion[J]. Journal of Zhejiang University: Engineering Science, 2023, 57(7): 1287-1296.
[10]	田水承, 郭谦, 张江江, 等. 基于蒙特卡罗法的建筑工人不安全行为风险评估[J]. 西安科技大学学报, 2023, 43(2): 373-379.
	TIAN Shuicheng, GUO Qian, ZHANG Jiangjiang, et al. Risk assessment of unsafe behaviors of construction workers based on Monte Carlo method[J]. Journal of Xi'an University of Science and Technology, 2023, 43(2): 373-379.
[11]	李华, 吴立舟, 薛曦澄, 等. 基于计算机视觉的高处临边作业安全巡检[J]. 中国安全科学学报, 2023, 33(9): 69-75. doi: 10.16265/j.cnki.issn1003-3033.2023.09.0202
	LI Hua, WU Lizhou, XUE Xicheng, et al. Computer vision based safety detection of high abutting edges[J]. China Safety Science Journal, 2023, 33(9): 69-75. doi: 10.16265/j.cnki.issn1003-3033.2023.09.0202
[12]	叶贵, 王妍, 任梦雪, 等. 体力疲劳对建筑工人不安全行为的影响效应研究[J]. 中国安全生产科学技术, 2023, 19(1): 122-127.
	YE Gui, WANG Yan, REN Mengxue, et al. Study on influence effect of physical fatigue on unsafe behavior of construction workers[J]. Journal of Safety Science and Technology, 2023, 19(1): 122-127.
[13]	史玉芳, 卢吉发. 基于SEM的建筑工人疲劳对不安全行为影响机理[J]. 西安科技大学学报, 2020, 40(4): 712-719.
	SHI Yufang, LU Jifa. Research on the impact of construction workers fatigue on unsafe behaviors based on SEM[J]. Journal of Xi'an University of Science and Technology, 2020, 40(4): 712-719.

Perclos取值	疲劳等级
[0,0.1)	清醒
[0.1,0.3)	临界状态
[0.3,0.4)	轻度疲劳
[0.4,0.5)	中度疲劳
[0.5,1.2)	严重疲劳

训练方法	P	R	mAP@ 0.5	mAP@ 0.5:0.95	模型内存/MB
YOLOv5	0.914	0.897	0.934	0.726	13.7
YOLOv5-CA	0.923	0.901	0.942	0.723	15.2
YOLOv5-CBAM	0.911	0.909	0.941	0.722	14.7
YOLOv5-ECA	0.923	0.905	0.940	0.723	13.7
YOLOv5-SE	0.904	0.904	0.934	0.715	14.7

[1]	张浩, 艾尔肯·亥木都拉. 基于改进YOLOv5的机械制造业企业安全风险管理模型[J]. 中国安全科学学报, 2025, 35(3): 52-59.
[2]	张泽辉, 张乾隆, 徐晓滨, 赵祖国, 王海泉, 李昊. 基于视觉的工人高处攀爬不安全行为识别模型[J]. 中国安全科学学报, 2025, 35(2): 144-151.
[3]	刘惠临, 方琼, 江宇, 魏华章, 王涛, 张树川. 基于YOLOv5s的轻量化森林火灾探测算法[J]. 中国安全科学学报, 2025, 35(1): 75-83.
[4]	赵江平, 王欣然, 吴立舟. 改进YOLOv5s的路面坑槽目标检测模型[J]. 中国安全科学学报, 2025, 35(1): 67-74.
[5]	雷斌, 闫浪浪, 余华, 温岩, 张亮, 李哲旭. YOLOv5监测算法下轨道交通应急预警感知系统构建[J]. 中国安全科学学报, 2024, 34(9): 225-233.
[6]	纪执安, 周云奕, 张玉媛, 郭新然, 石坤. 基于改进YOLOv5的工业现场不安全行为检测[J]. 中国安全科学学报, 2024, 34(7): 38-43.
[7]	吴立舟, 李华, 李典斌, 吴昱锦, 刘攀旺, 薛曦澄. 基于计算机视觉的塔吊危险区域入侵预警[J]. 中国安全科学学报, 2024, 34(7): 139-146.
[8]	张光远, 邓龙, 王亚伟, 孙自伟, 李莎, 陈诚. 基于脑电信号特征的高铁调度员疲劳状态识别[J]. 中国安全科学学报, 2024, 34(6): 235-246.
[9]	张术琳, 张亚楠, 田超, 严翔, 鲁义, 施式亮. 基于CNN的化工园区火灾火焰图像识别研究[J]. 中国安全科学学报, 2024, 34(1): 179-186.
[10]	王瑶涵, 宋泽阳, 张利冬. 基于卷积神经网络的安全标识分类算法研究[J]. 中国安全科学学报, 2023, 33(S1): 263-269.
[11]	李华, 吴立舟, 薛曦澄, 钟兴润. 基于计算机视觉的高处临边作业安全巡检[J]. 中国安全科学学报, 2023, 33(9): 69-75.
[12]	张磊, 李熙尉, 燕倩如, 王浩盛, 雷伟强. 基于改进YOLOv5s的综采工作面人员检测算法[J]. 中国安全科学学报, 2023, 33(7): 82-89.
[13]	王喆, 李享, 杨栋梁, 刘丹. 基于YOLOv5s的城镇森林交界域火灾探测模型[J]. 中国安全科学学报, 2023, 33(6): 152-158.
[14]	赵江平, 刘星星, 张想卓. 基于改进YOLOv5s的外脚手架隐患图像识别技术[J]. 中国安全科学学报, 2023, 33(12): 60-66.
[15]	张军, 尹柳, 巩欣飞, 徐赫桦. 基于改进YOLOv5的小目标烟雾检测算法[J]. 中国安全科学学报, 2023, 33(11): 75-81.