Research on DBN incorporating reinforcement learning for runway intrusion risk prediction

doi:10.16265/j.cnki.issn1003-3033.2024.07.0229

Abstract

Abstract:

In order to solve the problems of difficulty in quantifying the risk of airport runway incursion events, poor timeliness and low accuracy, and to enhance the capability of predicting runway incursion risks, a DBN model incorporating reinforcement learning for risk prediction was constructed. Firstly, causal inference theory was combined with grey relational analysis to analyze historical runway incursion events and identify the underlying risk factors. Secondly, Bayesian network(BN) theory was applied to explore the correlations among these factors and quantify these correlations using the Pearson linear correlation coefficient. This process helped in constructing a causation correlations network that effectively represented the propagation of risks associated with runway incursions. Then, the triangular fuzzy method and Hidden Markov Models (HMMs) were utilized to further refine and optimize the DBN parameter learning mechanism. Finally, the model's accuracy was validated using historical data. The results demonstrate that the proposed model's predictions of runway incursion risks closely align with the statistical values of historical data, achieving an accuracy rate of 84%, which represents a significant 10% improvement over Bayesian network predictions. Additionally, the use of mutual information to identify key nodes is found to effectively improve accuracy and discrimination compared to the degree value evaluation method.

Key words: reinforcement learning, dynamic Bayesian network (DBN), runway incursion, risk prediction, grey correlation analysis

CLC Number:

X949

WU Wei, WU Zexuan, WANG Xinglong, ZHU Longfei. Research on DBN incorporating reinforcement learning for runway intrusion risk prediction[J]. China Safety Science Journal, 2024, 34(7): 20-27.

Figures/Tables 10

Table 1

List of causal nodes

符号	名称	符号	名称	符号	名称
a₁	人的因素	a₂₂	管制人员培训工作不到位	b₇	降水
a₂	场面保障人员和车辆驾驶员差错	$a 2 3$	管制人员工作年限低	b₈	沙尘
a₃	管制人员差错	a₂₄	管制人员工作强度高	c₁	设备因素
a₄	飞行人员因素	a₂₅	管制人员人数少	c₂	监视设备故障
a₅	场面人员未及时发现危险	a₂₆	复诵错误	c₃	通信设备故障
a₆	场面人员未经许可进入跑道	a₂₇	飞行人员对机场不熟悉	c₄	地面标志有误
a₇	进入错误跑道	a₂₈	飞行人员未听指令进入跑道	c₅	场面灯光中断
a₈	场面人员和车辆驾驶员反应迟钝	a₂₉	飞行人员处置不及时	c₆	场面灯光故障
a₉	场面人员和车辆驾驶员业务素质差	a₃₀	飞行人员未及时发现错误	c₇	设备可靠性差
a₁₀	场面人员和车辆驾驶员对机场熟悉程度低	a₃₁	飞行人员陆空通话水平低	c₈	通信中断
a₁₁	场面人员和车辆驾驶员工作经验低	a₃₂	飞行人员反应时间长	c₉	标志模糊不清或不合理
a₁₂	场面人员和车辆驾驶员工作年限	a₃₃	飞行人员业务素质差	c₁₀	设备质量差
a₁₃	场面人员和车辆驾驶员培训工作不到位	a₃₄	飞行人员工作年限低	c₁₁	检修频率低
a₁₄	未发现飞行人员复诵错误并纠正	a₃₅	飞行员经验少	d₁	监管因素
a₁₅	未发现飞行人员、车辆等异常并及时处置	a₃₆	飞行人员培训不到位	d₂	监管频次低
a₁₆	管制员未按规定操作	b₁	环境因素	d₃	规章执行率低
a₁₇	管制指令不及时	b₂	地面湿滑	d₄	监控手段不足
a₁₈	管制人员陆空通话水平低	b₃	低能见度	d₅	监管人员不足
a₁₉	管制人员业务素质差	b₄	航班流量大	d₆	监管投入不足
a₂₀	管制人员工作经验不足	b₅	跑滑布局复杂
a₂₁	管制人员反应时间长	b₆	夜间

Table 1

Table 2

Eighteen risk values corresponding to a17

风险	$F 1$	$F 2$	$F 3$	$F 4$	$F 5$	$F 6$
取值	28.6	28.3	27.7	27.6	27.3	26.7
风险	$F 7$	$F 8$	$F 9$	$F 10$	$F 11$	$F 12$
取值	24.7	24.4	23.8	23.9	23.5	22.9
风险	$F 13$	$F 14$	$F 15$	$F 16$	$F 17$	$F 18$
取值	29.7	29.5	28.9	25.7	25.4	24.9

Table 2

Table 3

Fig.1

Fig.2

Fig.3

Table 4

Fig.4

Fig.5

Fig.6

References 19

[1]	United States Department of Transportation of Federal Aviation Aadministration. Runway safety statistics[R], 2022.
[2]	中国民用航空局航空安全办公室. 中国民航航空安全报告[R], 2021.
[3]	ROGERSON E C, LAMBERT J H. Prioritizing risks via several expert perspectives with application to runway safety[J]. Reliability Engineering and System Safety, 2012, 103(3):22-34.
[4]	JIJOK M, WESLEY L B, MAJOR B, et al. Statistical modelling of runway incursion occurrences in the United States - science direct[J]. Journal of Air Transport Management, 2017, 65(6):54-62.
[5]	霍志勤, 韩松臣. 跑道侵入的威胁与差错分析及控制研究[J]. 安全与环境学报, 2012, 12(4):192-196.
	HUO Zhiqin, HAN Songchen. Analysis on the threats and errors caused by the runway incursions and the corresponding controlling ways[J]. Journal of Safety and Environment, 2012, 12(4):192-196.
[6]	霍志勤, 韩松臣. 基于案例推理和规则推理融合的跑道侵入事件调查[J]. 中国安全科学学报, 2012, 22(7):53-58.
	HUO Zhiqin, HAN Songchen. Runway incursions investigation based on integration of rule-based reasoning and case-based reasoning[J]. China Safety Science Journal, 2012, 22(7):53-58.
[7]	夏正洪, 万健, 朱新平. 机场热点时空分布特征及运行风险评价[J]. 中国安全科学学报, 2018, 28(1):93-98. doi: 10.16265/j.cnki.issn1003-3033.2018.01.016
	XIA Zhenghong, WAN Jian, ZHU Xinping. Research on spatial and temporal distribution characteristics of airport hotspots and airport operational risk assessment[J]. China Safety Science Journal, 2018, 28(1):93-98. doi: 10.16265/j.cnki.issn1003-3033.2018.01.016
[8]	STROEVE S H, SOM P, DOORN B A V, et al. Strengthening air traffic safety management by moving from outcome-based towards risk-based evaluation of runway incursions[J]. Reliability Engineering & System Safety, 2016, 147(8):93-108.
[9]	王兴隆, 周督异. 基于复杂网络的飞行区冲突关键栅格区识别[J]. 中国安全生产科学技术, 2023, 19(8):32-38.
	WANG Xinglong, ZHOU Duyi. Identification of critical grid areas in flight area conflicts based on complex network[J]. Journal of Safety Science and Technology, 2023, 19(8):32-38.
[10]	罗军, 林雪宁. 基于模糊集和改进TOPSIS方法的跑道侵入风险评估[J]. 中国安全科学学报, 2012, 22(12):116-121.
	LUO Jun, LIN Xuening. Runway incursion risk assessment based on fuzzy sets theory and improved TOPSIS method[J]. China Safety Science Journal, 2012, 22(12):116-121.
[11]	罗军, 林雪宁, 闫永刚. 基于高斯贝叶斯网络模型的跑道侵入影响因素研究[J]. 安全与环境学报, 2012, 12(3):196-199.
	LUO Jun, LIN Xuening, YAN Yonggang. Analysis of runway incursion influence factors based on gaussian bayesian network model[J]. Journal of Safety and Environment, 2012, 12(3):196-199.
[12]	黄宝军, 房晓丹, 王洁宁. 基于多agent的跑道侵入风险分析建模及仿真[J]. 系统仿真学报, 2017, 29(4):910-917,925. doi: 10.16182/j.issn1004731x.joss.201704028
	HUANG Baojun, FANG Xiaodan, WANG Jiening. Risk simulation analysis of runway incursions based on multi agents[J]. Journal of System Simulation, 2017, 29(4):910-917,925. doi: 10.16182/j.issn1004731x.joss.201704028
[13]	焦卫东, 王瑞冬, 姚军强, 等. 跑道侵入与冲突检测的共形几何代数描述[J]. 航空学报, 2022, 43(5):483-492.
	JIAO Weidong, WANG Ruidong, YAO Junqiang, et al. Conformal geometric algebraic description of runway incursion and conflict detection[J]. Acta Aeronautica et Astronautica Sinica, 2022, 43(5):483-492.
[14]	王洁宁, 庾睿. 集成本体和BN的跑道侵入态势评估方法[J]. 计算机应用研究, 2019, 36(12):3773-3776.
	WANG Jiening, YU Rui. Situation assessment method of runway incursion based on combination of ontology and BN[J]. Application Research of Computers, 2019, 36(12):3773-3776.
[15]	袁乐平, 赵力梵, 张文东, 等. 基于复杂网络的民航跑道侵入事件分析[J]. 安全与环境学报, 2023, 23(5):1368-1375.
	YUAN Leping, ZHAO Lifan, ZHANG Wendong, et al. Analysis of civil aviation runway incursion incident based on complex network[J]. Journal of Safety and Environment, 2023, 23(5):1368-1375.
[16]	沈笑云, 张肖艳, 张思远. 基于ADS-B的跑道侵入冲突检测与告警研究[J]. 中国安全科学学报, 2019, 29(9):119-124. doi: 10.16265/j.cnki.issn1003-3033.2019.09.019
	SHEN Xiaoyun, ZHANG Xiaoyan, ZHANG Siyuan. Research on runway incursion conflict detection and alarm based on ADS-B[J]. China Safety Science Journal, 2019, 29(9):119-124. doi: 10.16265/j.cnki.issn1003-3033.2019.09.019
[17]	吴维, 罗欣然, 魏明. 基于SD模型的跑道侵入风控网络脆弱性分析[J]. 中国安全科学学报, 2021, 31(4):41-48. doi: 10.16265/j.cnki.issn1003-3033.2021.04.006
	WU Wei, LUO Xinran, WEI Ming. Vulnerability assessment of runway intrusion risk control network based on SD model[J]. China Safety Science Journal, 2021, 31(4):41-48. doi: 10.16265/j.cnki.issn1003-3033.2021.04.006
[18]	STAMP M. A revealing introduction to hidden markov models[J]. Leee Assp Magruine, 2004, 8(5):1-21.
[19]	乌兰, 刘全, 黄志刚, 等. 优势加权互信息最大化的最大熵分层强化学习[J]. 计算机学报, 2023, 46(10):2066-2083.
	WU Lan, LIU Quan, HUANG Zhigang, et al. Maximum entropy hierarchical reinforcement learning with advantage-weighted mutual information maximization[J]. Chinese Journal of Computers, 2023, 46(10):2066-2083.

节点状态	概率范围	模糊数集
极低	0.1	(0.1, 0.1, 0.2)
低	0.3	(0.2, 0.3, 0.4)
一般	0.5	(0.4, 0.5, 0.6)
高	0.7	(0.6, 0.7, 0.8)
极高	0.9	(0.8, 0.9, 0.9)

预测结果概率值	预测结果占比分布情况/%
预测结果概率值	BN	DBN	HMMs学习的DBN
[0,0.5)	6.4	5.5	2.0
[0.5,0.85]	19.2	14.2	13.8
(0.85,1]	74.4	80.3	84.2