Evaluation method of ATCO's task performance based on operation data

doi:10.16265/j.cnki.issn1003-3033.2020.03.002

Abstract

Abstract: In order to address the common problem of strong subjectivity and obvious differences in monitoring and evaluating ATCO's work performance by air traffic management organizations in China, an assessment index system was designed based on flight tracks, controlling airspaces, device operation data and instruction release data. Then, assessment procedure and calculation method of indexes were studied from aspects of flight safety, operational efficiency and controlling rules, and their value was quantified by utilizing fuzzy synthetic evaluation method. Finally, the evaluation system was released on Matlab platform and validated with simulative operation data in certain sector of China. The research shows that this objective data-based assessment method makes more integrated, quantitative and intelligent evaluation of ATCO's operational skills and decision-making capacity compared with traditional checklist assessment method, therefore it can help obtain evaluation results more quickly and accurately.

Key words: air traffic controller (ATCO), task performance, evaluation index, fuzzy synthetic evaluation, checklist

CLC Number:

X949

YANG Yue, HUANG Yigang, SONG Xiangbo, WANG Jianzhong. Evaluation method of ATCO's task performance based on operation data[J]. China Safety Science Journal, 2020, 30(3): 8-14.

References

[1] 梁曼,徐肖豪,邹少孚. 基于模糊BN和SHELLO模型的管制人为因素安全风险分析 [J]. 中国安全科学学报,2013,23(11): 126-131.
LIANG Man, XU Xiaohao, ZOU Shaofu. Air traffic control human factors safety risk analysis based on fuzzy Bayesian network and SHELLO model [J]. China Safety Science Journal, 2013, 23(11): 126-131.
[2] PAUL L E. The evaluation of conflicts in air traffic control simulation [R]. William J. Hughes Technical Center, 1989.
[3] HANSON M A, BORMAN W C, MOGILKA H J, et al. Computerized assessment of skill for a highly technical job [R]. Lawrence Erlbaum Associates, 1999.
[4] SINCLAIR G, SEILER R. Air traffic selection & training (AT-SAT) test success predictability and preparation [J]. Journal of Computational Information Systems, 2012, 8(23): 9 671-9 677.
[5] BROACH D, BYRNE C L, MANNING C A, et al. The validity of the air traffic selection and training (AT-SAT) test battery in operational use federal aviation administration [J]. Palaeontology, 2013, 48(6):1 409-1 417.
[6] ARMINEN I, KOSKELA I, PALUKKA H. Multimodal production of second pair parts in air traffic control training [J]. Journal of Pragmatics, 2014, 65(1): 46-62.
[7] BYRNE C L, BROACH D. An evaluation of the utility of AT-SAT for the placement of new controllers by option [R]. Federal Aviation Administration, 2014.
[8] Federal Aviation Administration. Air traffic technical training [EB/OL]. [2015-10-30]. https://www.faa.gov/document Library/media/Order/JO_3120.4P.pdf.
[9] OPRINS E, BURGGRAAFF E, VAN WEERDENBURG H. Design of a competence-based assessment system for air traffic control training [J]. The International Journal of Aviation Psychology, 2006, 16(3): 297-320.
[10] SUBOTIC B, SCHUSTER W, MAJUMDAR A, et al. Controller recovery from equipment failures in air traffic control: a framework for the quantitative assessment of the recovery context [J]. Reliability Engineering & System Safety, 2014, 132: 60-71.
[11] 张建平,刘丹,程延松,等. 空中交通管制员技能评估主成分分析方法 [J]. 航空计算技术,2014,44(2): 7-10.
ZHANG Jianping, LIU Dan, CHENG Yansong, et al. An assessment method on skill ability of air traffic controller based on principal component analysis [J]. Aeronautical Computing Technique,2014,44(2): 7-10.
[12] 王霞,孙石磊. 基于空中交通复杂度的管制员认知负荷测度 [J]. 安全与环境学报,2018,18(5): 1 881-1 885.
WANG Xia, SUN Shilei. On the approach to measuring the air traffic controllers' cognitive load on account of the complexity of the air traffic [J]. Journal of Safety and Environment, 2018,18(5): 1 881-1 885.
[13] 中国民用航空局. 民用航空空中交通管制培训管理规则 [L]. 2012-02-14.
[14] 中国民用航空局. 雷达管制基础模拟机培训大纲 [Z]. 2012-06-26.

[1]	LUO Zhenmin, WANG Dengfei, DING Xuhan. Accuracy analysis on evaluation indexes of dust suppressant effect [J]. China Safety Science Journal, 2022, 32(1): 195-200.
[2]	YANG Sanjun, JING Yaru, JIANG Runfa. Construction of evaluation index system for emergency rescue capabilities of mine rescue teams [J]. China Safety Science Journal, 2021, 31(8): 180-188.
[3]	JIANG Fuchuan, ZHOU Shi, WU Zengtong, ZHONG Jichao, ZHANG Shuhao, ZHANG Xuefang. Analysis of coal mine safety investment decision based on entropy weight-TOPSIS method [J]. China Safety Science Journal, 2021, 31(7): 24-29.
[4]	LI Yanfeng, YIN Jiaxiao, LIU Chaofeng, WANG Wei, YU Xiaochen, LI Zipeng. Assessment on seismic resilience of water supply network based on projection pursuit cluster [J]. China Safety Science Journal, 2020, 30(6): 152-157.
[5]	ZHANG Zhenjiang, ZHANG Yuzhao, WANG Xiaorong. Robustness analysis of railway express freight network [J]. China Safety Science Journal, 2020, 30(3): 150-156.
[6]	TIAN Fangyuan, TIAN Shuicheng, LI Hongxia, ZHENG Ya, SHAO Jiang. Psychological symptoms of public under COVID-19 outbreak and improvement of emergency management system [J]. China Safety Science Journal, 2020, 30(11): 30-36.
[7]	XIE Anna, YIN Panpan, WANG Zhiyuan. Influence factor analysis of railway passenger transportation safety based on SEM [J]. China Safety Science Journal, 2019, 29(S2): 30-34.
[8]	LI Junjie, YAO Xinyi, CHEN Dingjun, NI Shaoquan. Evaluation index selection of comprehensive inter-regional traffic emergency rescue plan [J]. China Safety Science Journal, 2018, 28(S2): 185-190.
[9]	CHEN Liuyu, LI Xijian, YIN Xin, BI Juan. Complex network theory based analysis of importance of urban gas pipeline network nodes [J]. China Safety Science Journal, 2018, 28(9): 74-80.
[10]	WANG Tingchun, ZHANG Yali, SUN Deqing, XU Yang, CHANG Qingtao. Evaluation of comprehensive safety capability of refining contractor [J]. China Safety Science Journal, 2018, 28(8): 155-160.
[11]	CAO Shanshan, WANG Peng, SHAN Xiaofang, YAO Yang. A review of progress in reliability evaluation on district heating system [J]. China Safety Science Journal, 2017, 27(12): 49-55.
[12]	. Construction and Optimization of Performance Evaluation Index System for BPVSI Institute [J]. China Safety Science Journal, 2013, 23(9): 159-.
[13]	. Integrated Evaluation of Public Security Risk Measurement Based on "Bow-string-arrow" Model [J]. China Safety Science Journal, 2013, 23(7): 133-.
[14]	. Research on Ship Pilot Reliability Evaluation System [J]. China Safety Science Journal, 2013, 23(5): 76-.
[15]	. A Fuzzy Comprehensive Evaluation Method for Ecosystem Health Status in Mining Area [J]. China Safety Science Journal, 2009, 19(12): 154-.