Explainable prediction for hard landing of civil aircraft based on LightGBM-SHAP

doi:10.16265/j.cnki.issn1003-3033.2024.10.1123

Abstract

Abstract:

In order to prevent hard landing overrun events of civil aircraft, first, data including kinematics, system performance and other engineering parameters was collected from QAR. Then QAR data processing activities such as the airport segment clustering, sample balancing and statistical feature extraction were carried out. Subsequently, LightGBM model was used to predict the hard landing events of civil aircraft, and compared with extreme gradient boosting (XGBoost), decision tree (DT) and long short-term memory (LSTM) models. Finally, the shapley additive explanation (SHAP) algorithm was employed to identify the causal mechanisms of hard landing events and to analyze the impact of various flight parameters on the model's prediction results. The result demonstrates that the proposed model not only exhibits high accuracy and precision in predicting hard landing events (accuracy, correctness and recall reaching 99%, 92% and 88%, respectively), but also provides quantitative and visual explanation information for the decision-making process of hard landing prediction for specific flight segments.

Key words: lightweight gradient boosting machine (LightGBM), civil aircraft, hard landing, quick access recorder (QAR) data, machine learning, explainable

CLC Number:

X949

XIAO Guosong, LIU Jiachen, ZHANG Yuanshan, DONG Lei, CHEN Xi. Explainable prediction for hard landing of civil aircraft based on LightGBM-SHAP[J]. China Safety Science Journal, 2024, 34(10): 134-142.

Figures/Tables 16

Fig.1

Table 1

Table 2

K-means clustering algorithm for aircraft landing segments

输入:数据集 $D = {u 1, u 2, … u N}$ ,其中, $u n$ 表示第n个航段的着陆位置,航段(样本)数量为N
从样本集中随机选择一个航段作为初始质心 $β 1$ 令 $C 1 = {x 1}$ for $j = 2,3, …, N$ 计算航段 $u j$ 到每个质心 $μ i$ 的距离: $d j i = ‖ u j - μ i ‖ 2$ ; if最近距离 $a r g m i n i ∈ {1,2, …, d j i}$ > 4.8 km then 形成一个新的簇 $C s = {u j}$ ; else 将航段 $u j$ 分配到距离最近的簇中; end if end for for $i = 1,2, …, d i j$
计算簇中所有样本的均值: $β' i = 1 \| C i \| ∑ u ∈ C i u i$ ; 将质心 $β i$ 更新为 $β' i$ ; end for 直到所有航段都被分配到相应的簇

Table 2

Fig.2

Fig.3

Fig.4

Table 3

Table 4

Table 5

Model evaluation indicators and meanings

评价指标	计算公式	符号含义
A	$T P + T N T P + T N + F P + F N$	$T P$ 为硬着陆预测正确的数量; $T N$ 为正常着陆预测正确的数量; $F P$ 为正常着陆被错误预测为硬着陆的数量; $F N$ 为硬着陆被错误预测为正常着陆的数量;P_h为硬着陆预测结果的概率;P_n为正常着陆预测结果的概率
P	$T P T P + F P$
R	$T P T P + F N$
F₁分数	$2 × P × R P + R$
AUC	P (P_h> P_n)

Table 5

Table 6

Fig.5

Fig.6

Fig.7

Fig.8

Table 7

Fig.9

References 22

[1]	The Boeing Company. Statistical summary of commercial jet airplane accidents worldwide operations[EB/OL]. (2024-06-29). https://www.boeing.com/content/dam/boeing/boeingdotcom/company/about_bca/pdf/statsum.pdf.
[2]	WANG Lei, REN Yong, WU Changxu. Effects of flare operation on landing safety: a study based on ANOVA of real flight data[J]. Safety Science, 2018, 102: 14-25.
[3]	BLAJEV T, CURTIS W. Go-around decision-making and execution project[EB/OL]. (2017-03-01). https://flightsafety.org/wp-content/uploads/2017/03/Go-around-study_final.pdf.
[4]	刘汉辉, 高扬. 中国民航的安全文化建设(之三)[J]. 中国安全科学学报, 2002, 12(3): 12-17.
	LIU Hanhui, GAO Yang. Safety culture construction of civil aviation in China (part 3)[J]. China Safety Science Journal, 2002, 12(3): 12-17.
[5]	TONG Chao, YIN Xiang, LI Jun, et al. An innovative deep architecture for aircraft hard landing prediction based on time-series sensor data[J]. Applied Soft Computing, 2018, 73: 344-349.
[6]	ZHANG Haochi, ZHU Tongyu. Aircraft hard landing prediction using LSTM neural network[C]. the 2^nd International Symposium on Computer Science and Intelligent Control,2018:1-5.
[7]	GIL D, HERNANDEZ-SABATE A, ENCONNIERE J, et al. E-pilots: a system to predict hard landing during the approach phase of commercial flights[J]. IEEE Access, 2021, 10: 7489-7503.
[8]	LIU Yinfu, SUN Ruishan, HE Peng. Research on the pre-warning method of aircraft long landing based on the XGboost algorithm and operation characteristics clustering[J]. Aerospace, 2023, 10(5): DOI:10.3390/aerospace10050409.
[9]	董磊, 刘嘉琛, 陈曦, 等. 面向适航符合性的智能航电系统认证研究进展[J]. 航空工程进展, 2023, 14(3): 26-40.
	DONG Lei, LIU Jiachen, CHEN Xi, et al. Research progress of AI-based avionics system certification for airworthiness compliance[J]. Advances in Aeronautical Science and Engineering, 2023, 14(3): 26-40.
[10]	Society of Automotive Engineers. Artificial intelligence in aeronautical systems: statement of concerns: AIR6988[R], 2021.
[11]	European Union Aviation Safety Agency. Guidance for Level 1 & 2 machine learning applications[R], 2024.3.
[12]	LI Kun, XU Haocheng, LIN Xiao. Analysis and visualization of accidents severity based on LightGBM-TPE[J]. Chaos, Solitons & Fractals, 2022, 157: DOI: 10.1016/j.chaos.2022.111987.
[13]	SUN Deliang, WU Xiaoqing, WEN Haijia, et al. A LightGBM-based landslide susceptibility model considering the uncertainty of non-landslide samples[J]. Geomatics, Natural Hazards and Risk, 2023, 14(1): DOI: 10.1080/19475705.2023.2213807.
[14]	汪祖民, 王恺锋, 李艳志, 等. 基于LightGBM和SHAP的云南省森林火灾预测研究[J]. 消防科学与技术, 2023, 42(11): 1567-1571.
	WANG Zumin, WANG Kaifeng, LI Yanzhi, et al. Research on forest fire prediction in Yunnan province based on LightGBM and SHAP[J]. Fire Science and Technology, 2023, 42(11): 1567-1571.
[15]	张凯, 张科. 基于LightGBM算法的边坡稳定性预测研究[J]. 中国安全科学学报, 2022, 32(7): 113-120. doi: 10.16265/j.cnki.issn1003-3033.2022.07.1473
	ZHANG Kai, ZHANG Ke. Prediction study on slope stability based on LightGBM algorithm[J]. China Safety Science Journal, 2022, 32(7): 113-120. doi: 10.16265/j.cnki.issn1003-3033.2022.07.1473
[16]	KE Guolin, MENG Qi, FINLEY T, et al. LightGBM: a highly efficient gradient boosting decision tree[C]. 31^st International Conference on Neural Information Processing Systems, 2017: 1-9.
[17]	戢晓峰, 乔新, 普永明, 等. 地铁站周边建成环境对交通事故风险的影响机制[J]. 中国安全科学学报, 2022, 32(10): 162-170. doi: 10.16265/j.cnki.issn1003-3033.2022.10.2546
	JI Xiaofeng, QIAO Xin, PU Yongming, et al. Influence mechanism of built environment around subway station traffic accident risk[J]. China Safety Science Journal, 2022, 32(10): 162-170. doi: 10.16265/j.cnki.issn1003-3033.2022.10.2546
[18]	AGARWAL R, MELNICK L, FROSST N, et al. Neural additive models: interpretable machine learning with neural nets[J]. Advances in Neural Information Processing Systems, 2021, 34: 4699-4711.
[19]	BRYAN M. Sample flight data[EB/OL]. (2012-12-04). https://c3.ndc.nasa.gov/dashlink/projects/85/resources/.
[20]	BARDOU N, OWENS D. Hard landing, a case study for crews and maintenance personnel[EB/OL]. (2019-02-18). https://safetyfirst.airbus.com/hard-landing-a-case-study-for-crews-and-maintenance-personnel/.
[21]	宋英华, 江晨, 李墨潇, 等. 基于改进Smote-GBDT算法的岩爆预测模型[J]. 中国安全科学学报, 2023, 33(9): 25-32. doi: 10.16265/j.cnki.issn1003-3033.2023.09.0850
	SONG Yinghua, JIANG Chen, LI Moxiao, et al. Rockburst prediction model based on improved Smote-GBDT algorithm[J]. China Safety Science Journal, 2023, 33(9): 25-32. doi: 10.16265/j.cnki.issn1003-3033.2023.09.0850
[22]	LI Xu, SHANG Jiaxing, ZHENG Linjiang, et al. IMTCN: an interpretable flight safety analysis and prediction model based on multi-scale temporal convolutional networks[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 25(1): 289-302.

类型	参数	采样率/ (次·s^-1)
动力学参数	计算空速(Computed Airspeed, CAS)/kn	4
	垂直加速度(Vertical Acceleration, VRTG)/g	8
	下降率(Altitude Rate, ALTR)/ (ft·min^-1)	4
	偏航角(Drift Angle, DA)/(°)	4
	攻角(Angle Of Attack, AOA)/(°)	4
	气压矫正高度(Baro correct Altitude, BAL)/ft	4
	磁航向(Magnetic Heading, MH)/(°)	4
	横向加速度(Cross Track Acceleration, CTAC)/g	16
操纵参数	主驾驶杆位置(Control Column Position Capt, CCPC)/Counts	2
	最大选择高度(Selected Altitude, ALTS)/ft	1
	选择垂直速度(Selected Vertical Speed, VSPS)/(ft·min^-1)	1
	选择马赫数(Selected Mach, MNS)	1
	方向舵踏板位置(Rudder Pedal Position, RUDP)	2
	推力手柄角度(Power Lever Angle, PLA)/(°)	4
环境参数	风向(Wind Direction, WD)/(°)	4
环境参数	风速(Wind Speed, WS)/kn	4
配置参数	扰流板位置(Roll Spoiler, SPL)/(°)	1
	方向舵位置(Rudder Position, RUDD)/(°)	2
	襟翼位置(T.E. Flap Position, FLAP)	1
	副翼位置(Aileron position, AIL)/(°)	1
	俯仰安定面位置(Pitch Trim Position, PTRM)/(°)	1

参数	max	min	Mean	Median	VAR	STD
AIL	86.621 19	78.131 12	83.880 15	83.716 16	1.745 762	3.047 684
AOA	2.636 699 9	-4.790 007 6	-2.576 91	-2.812 48	1.409 389	1.986 378
CAS	120.937 5	108.25	116.678 3	116.812 5	2.474 43	6.122 802
CTAC	0.044 942	-0.028 3	0.012 701	0.017 586	0.020 799	0.000 433
MH	-60.483 4	-63.131 1	-61.945 6	-61.955 6	0.690 906	0.477 352

模型参数	默认值	搜索范围	优化值
学习率	0.1	[0.01, 0.1]	0.05
最大深度	—	[3, 5]	5
叶子节点个数	30	(15, 30, 50, 100)	30

样本平衡前/后	P/%	A/%	R/%	F₁分数	AUC
样本平衡前	92	5	15	0.07	0.58
样本平衡后	99	92	88	0.90	0.99

参数特征	类型	出现次数
max_RUDD	配置参数	28
min_PTRM	配置参数	26
max_ALTS	操纵参数	22
var_PTRM	配置参数	20
min_ALTS	操纵参数	20
min_RUDD	配置参数	18
var_DA	动力学参数	18
var_WD	环境参数	17
var_RUDP	操纵参数	17
std_MNS	操纵参数	16