Research on optimization of disaster triplet information extraction based on BERT

doi:10.16265/j.cnki.issn1003-3033.2022.02.016

Abstract

Abstract:

In order to quickly and precisely extract triplet information from text on online social media, NLP technology was utilized to study the application and algorithm optimization of the information extraction. Then, BERT pre-trained language model was applied in a case of triplet information extraction of geological disasters. Considering the model's problems of "low-rank bottlenecks" caused by its own MHA mechanism, the key-size was increased to optimize the model. The results show that the proposed method can significantly improve fault tolerance and accuracy of disaster information extraction, including disaster type, occurrence location, occurrence time, from news reports. And it can be used to analyze spatial distribution and trend of disasters, and then provide scientific analysis and decision-making support for disaster emergency management, such as preparation of emergency plan, optimal allocation of emergency resources, regional monitoring and early warning, etc.

Key words: natural language processing (NLP), bidirectional encoder representation from transformers(BERT), low-rank bottleneck, multi-head attention(MHA), disaster information

SONG Dunjiang, YANG Lin, ZHONG Shaobo. Research on optimization of disaster triplet information extraction based on BERT[J]. China Safety Science Journal, 2022, 32(2): 115-120.

Figures/Tables 8

Fig.1

Fig.2

Tab.1

Tab.2

Tab.3

Fig.3

Tab.4

Fig.4

References 15

[1]	YU Kaixian, LUNG Peiyau, ZHAO Tingting, et al. Automatic extraction of protein-protein interactions using grammatical relationship graph[J]. BMC Medical Informatics & Decision Making, 2018, 18: 35-43.
[2]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. Advances in Neural Information Processing Systems, 2017: 5998-6008.
[3]	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251.
	ZHOU Feiyan, JIN Linpeng, DONG Jun. Summary of research on convolutional neural networks[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251.
[4]	杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(增2):1-6, 26.
	YANG Li, WU Yuqian, WANG Junli, et al. Summarization of recurrent neural network research[J]. Computer Applications, 2018, 38(S2):1-6, 26.
[5]	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[Z], 2018: arXiv preprint arXiv: 1810.04805.
[6]	LAN Zhenzhong, CHEN Mingda, GOODMAN S, et al. ALBERT: a Lite BERT for self-supervised learning of language representations[Z], 2019: arXiv preprint arXiv: 1909.11942.
[7]	宋敦江, 赵作权, 钟少波. 2005-2007年四川地震的时空演化特征分析[J]. 科技导报 2008, 26(22): 38-43.
	SONG Dunjiang, ZHAO Zuoquan, ZHONG Shaobo. Spatiotemporal evolution of earthquakes in sichuan province between 2005 and 2007[J]. Science & Technology Review, 2008, 26(22): 38-43.
[8]	ZHONG Shaobo, XUE Yong, CAO Chunxiang, et al. Explore disease mapping of hepatitis B using geostatistical analysis techniques[C]. International Conference on Computational Science, Springer, 2005: 464-471.
[9]	熊智, 钟少波, 宋敦江, 等. 城市轨道交通客流量时间序列分段拟合方法[J]. 中国安全科学学报, 2018, 28(11): 39-45.
	XIONG Zhi, ZHONG Shaobo, SONG Dunjiang, et al. A method of fitting urban rail transit passenger flow time series[J]. China Safety Science Journal, 2018, 28(11): 39-45.
[10]	杨永胜, 钟少波, 余致辰, 等. 一种城市燃气管网泄漏风险动态计算模型[J]. 中国安全科学学报, 2018, 28(1): 167-172. doi: 10.16265/j.cnki.issn1003-3033.2018.01.028
	YANG Yongsheng, ZHONG Shaobo, YU Zhichen, et al. A model for dynamic risk calculation of urban gas pipeline[J]. China Safety Science Journal, 2018, 28(1): 167-172. doi: 10.16265/j.cnki.issn1003-3033.2018.01.028
[11]	蒋琴, 钟少波, 朱伟. 京津冀地区森林火灾综合风险评估[J]. 中国安全科学学报, 2020, 30(10): 119-125. doi: 10.16265/j.cnki.issn 1003-3033.2020.10.017
	JIANG Qin, ZHONG Shaobo, ZHU Wei. Research on comprehensive risk assessment of forest fire in Beijing-Tianjin-Hebei region[J]. China Safety Science Journal, 2020, 30(10): 119-125. doi: 10.16265/j.cnki.issn 1003-3033.2020.10.017
[12]	HUANG Chao, ZHONG Shaobo, LI Xin, et al. Emergency case retrieval based on fuzzy sets and text mining[J]. Advances in Intelligent Systems and Computing, 2014, 277:911-919.
[13]	BHOJANAPALLI S, YUN C, RAWAT A S, et al. Low-rank bottleneck in multi-head attention models[C]. International Conference on Machine Learning, 2020: 864-873.
[14]	LIU Yinhan, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[Z], 2019: arXiv preprint arXiv: 1907.11692.
[15]	何炎祥, 罗楚威, 胡彬尧. 基于CRF和规则相结合的地理命名实体识别方法[J]. 计算机应用与软件, 2015, 32(1):179-185, 202.
	HE Yanxiang, LUO Chuwei, HU Binyao. Geographically named entity recognition method based on the combination of CRF and rules[J]. Computer Applications and Software, 2015, 32(1):179-185, 202.

序号	类型	举例
1	地震	{"text": "宜宾珙县发生5.6级地震,网友:我开车都感觉到在动", "id": "e86f7a200339", "event_list": [{"event_type": "灾害/意外-地震", "trigger": "地震", "trigger_start_index": 13, "arguments": [{"argument_start_index": 0, "role": "地点", "argument": "宜宾珙县",{"argument_start_index": 9, "role": "震级", "argument": "5.6级"}], "class": "灾害/意外"}]}
2	坍塌	{"text": "青岛地铁一施工段发生坍塌初步确定5人被困", "id": "e62884f833cb4", "event_list": [{"event_type": "灾害/意外-坍/垮塌", "trigger": "坍塌", "trigger_start_index": 10, "arguments": [{"argument_start_index": 0, "role": "坍塌主体", "argument": "青岛地铁一施工段"}], "class": "灾害/意外"}]}
3	洪灾	{"text": "2019年8月6日,湖北十堰山洪已致8人死亡,应急管理部派工作组救援工作", "id": "118d7ee0a6053", "event_list": [{"event_type": "灾害/意外-洪灾", "trigger": "山洪", "trigger_start_index": 14, "arguments": [{"argument_start_index": 10, "role": "地点", "argument": "湖北十堰"},{"argument_start_index": 0, "role": "时间", "argument": "2019年8月6日"},]}

软/硬件	版本/配置
操作系统	Ubuntu
CPU	I7-6700HQ@2.60Hz
GPU	GTX-1080 Ti(8GB)
内存	32Gb
Python	3.7.4
Tensorflow	1.14.0
Keras	2.3.1

模型	序号	key_size	Epoch	P/%	R/%	F₁/%
Small	1	12	10	56.4	23.9	33.7
Small	2	12	20	68.3	49.1	56.6
Small+	3	128	10	69.4	30.3	42.9
Small+	4	128	20	73.3	55.3	64.4
Tiny	5	12	10	67.3	17.8	29.9
Tiny	6	12	20	68.9	42.6	52.3
Tiny+	7	118	10	59.4	21.1	35.8
Tiny+	8	118	20	71.5	44.4	54.8

模型	文本长度l	正确数量
Tiny+	0 < l ≤32	54
	32 < l ≤64	53
	64 < l ≤96	61
	96 < l ≤128	63
Small+	0 < l ≤32	59
	32 < l ≤64	66
	64 < l ≤96	74
	96 < l ≤128	73