基于BERT的灾害三元组信息抽取优化研究

doi:10.16265/j.cnki.issn1003-3033.2022.02.016

中国安全科学学报 ›› 2022, Vol. 32 ›› Issue (2): 115-120.doi: 10.16265/j.cnki.issn1003-3033.2022.02.016

基于BERT的灾害三元组信息抽取优化研究

宋敦江¹(), 杨霖², 钟少波³^,^**()

¹ 中国科学院科技战略咨询研究院,北京 100190
² 太原学院计算机科学与技术系,山西太原 030032
³ 北京城市系统工程研究中心城市运行研究部,北京 100035

收稿日期:2021-11-22 修回日期:2022-01-15 出版日期:2022-08-18 发布日期:2022-08-28
通讯作者: 钟少波
作者简介:
宋敦江 (1979—),男,湖北黄石人,博士,副研究员,主要从事可持续发展战略研究。E-mail: songdj@casisd.cn。
基金资助:
国家重点研发计划项目(2019YFF0301300); 国家自然科学基金资助(41471338); 中国科学院科技战略咨询研究院重点培育项目; 北京市科技计划课题(Z191100001419002)

Research on optimization of disaster triplet information extraction based on BERT

SONG Dunjiang¹(), YANG Lin², ZHONG Shaobo³^,^**()

¹ Institute of Science and Development, Chinese Academy of Sciences, Beijing 100190, China
² Department of Computer Science and Technology, Taiyuan University, Taiyuan Shanxi 030032, China
³ Urban Operation Research Department, Beijing Research Center of Urban Systems Engineering, Beijing 100035, China

Received:2021-11-22 Revised:2022-01-15 Online:2022-08-18 Published:2022-08-28
Contact: ZHONG Shaobo

摘要/Abstract

摘要：

为从网络媒体文本中快速、准确提取灾害三元组信息,利用自然语言处理(NLP)技术,研究灾害三元组信息抽取应用及其算法优化。通过双向编码器表示(BERT)预训练语言模型,应用于地质灾害三元组信息提取的实例中,针对模型由于底层多头注意力(MHA)机制会导致“低秩瓶颈”问题,对此,通过增大模型key-size对其进行优化。结果表明:所提方法能够显著提升从新闻报道等文本中提取地质灾害种类、发生地点、发生时间等关键信息的容错率及精准率;可得到对地质等灾害空间分布情况和趋势的分析,进而为预案编制、应急资源优化配置、区域监测预警等灾害应急管理工作提供科学分析和决策信息支持。

关键词: 然语言处理(NLP), 双向编码器表示(BERT), 低秩瓶颈, 多头注意力(MHA), 灾害信息

Abstract:

In order to quickly and precisely extract triplet information from text on online social media, NLP technology was utilized to study the application and algorithm optimization of the information extraction. Then, BERT pre-trained language model was applied in a case of triplet information extraction of geological disasters. Considering the model's problems of "low-rank bottlenecks" caused by its own MHA mechanism, the key-size was increased to optimize the model. The results show that the proposed method can significantly improve fault tolerance and accuracy of disaster information extraction, including disaster type, occurrence location, occurrence time, from news reports. And it can be used to analyze spatial distribution and trend of disasters, and then provide scientific analysis and decision-making support for disaster emergency management, such as preparation of emergency plan, optimal allocation of emergency resources, regional monitoring and early warning, etc.

Key words: natural language processing (NLP), bidirectional encoder representation from transformers(BERT), low-rank bottleneck, multi-head attention(MHA), disaster information

宋敦江, 杨霖, 钟少波. 基于BERT的灾害三元组信息抽取优化研究[J]. 中国安全科学学报, 2022, 32(2): 115-120.

SONG Dunjiang, YANG Lin, ZHONG Shaobo. Research on optimization of disaster triplet information extraction based on BERT[J]. China Safety Science Journal, 2022, 32(2): 115-120.

图/表 8

图1

图2

表1

表2

表3

图3

表4

图4

参考文献 15

[1]	YU Kaixian, LUNG Peiyau, ZHAO Tingting, et al. Automatic extraction of protein-protein interactions using grammatical relationship graph[J]. BMC Medical Informatics & Decision Making, 2018, 18: 35-43.
[2]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. Advances in Neural Information Processing Systems, 2017: 5998-6008.
[3]	周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6):1229-1251.
	ZHOU Feiyan, JIN Linpeng, DONG Jun. Summary of research on convolutional neural networks[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251.
[4]	杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(增2):1-6, 26.
	YANG Li, WU Yuqian, WANG Junli, et al. Summarization of recurrent neural network research[J]. Computer Applications, 2018, 38(S2):1-6, 26.
[5]	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[Z], 2018: arXiv preprint arXiv: 1810.04805.
[6]	LAN Zhenzhong, CHEN Mingda, GOODMAN S, et al. ALBERT: a Lite BERT for self-supervised learning of language representations[Z], 2019: arXiv preprint arXiv: 1909.11942.
[7]	宋敦江, 赵作权, 钟少波. 2005-2007年四川地震的时空演化特征分析[J]. 科技导报 2008, 26(22): 38-43.
	SONG Dunjiang, ZHAO Zuoquan, ZHONG Shaobo. Spatiotemporal evolution of earthquakes in sichuan province between 2005 and 2007[J]. Science & Technology Review, 2008, 26(22): 38-43.
[8]	ZHONG Shaobo, XUE Yong, CAO Chunxiang, et al. Explore disease mapping of hepatitis B using geostatistical analysis techniques[C]. International Conference on Computational Science, Springer, 2005: 464-471.
[9]	熊智, 钟少波, 宋敦江, 等. 城市轨道交通客流量时间序列分段拟合方法[J]. 中国安全科学学报, 2018, 28(11): 39-45.
	XIONG Zhi, ZHONG Shaobo, SONG Dunjiang, et al. A method of fitting urban rail transit passenger flow time series[J]. China Safety Science Journal, 2018, 28(11): 39-45.
[10]	杨永胜, 钟少波, 余致辰, 等. 一种城市燃气管网泄漏风险动态计算模型[J]. 中国安全科学学报, 2018, 28(1): 167-172. doi: 10.16265/j.cnki.issn1003-3033.2018.01.028
	YANG Yongsheng, ZHONG Shaobo, YU Zhichen, et al. A model for dynamic risk calculation of urban gas pipeline[J]. China Safety Science Journal, 2018, 28(1): 167-172. doi: 10.16265/j.cnki.issn1003-3033.2018.01.028
[11]	蒋琴, 钟少波, 朱伟. 京津冀地区森林火灾综合风险评估[J]. 中国安全科学学报, 2020, 30(10): 119-125. doi: 10.16265/j.cnki.issn 1003-3033.2020.10.017
	JIANG Qin, ZHONG Shaobo, ZHU Wei. Research on comprehensive risk assessment of forest fire in Beijing-Tianjin-Hebei region[J]. China Safety Science Journal, 2020, 30(10): 119-125. doi: 10.16265/j.cnki.issn 1003-3033.2020.10.017
[12]	HUANG Chao, ZHONG Shaobo, LI Xin, et al. Emergency case retrieval based on fuzzy sets and text mining[J]. Advances in Intelligent Systems and Computing, 2014, 277:911-919.
[13]	BHOJANAPALLI S, YUN C, RAWAT A S, et al. Low-rank bottleneck in multi-head attention models[C]. International Conference on Machine Learning, 2020: 864-873.
[14]	LIU Yinhan, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[Z], 2019: arXiv preprint arXiv: 1907.11692.
[15]	何炎祥, 罗楚威, 胡彬尧. 基于CRF和规则相结合的地理命名实体识别方法[J]. 计算机应用与软件, 2015, 32(1):179-185, 202.
	HE Yanxiang, LUO Chuwei, HU Binyao. Geographically named entity recognition method based on the combination of CRF and rules[J]. Computer Applications and Software, 2015, 32(1):179-185, 202.

序号	类型	举例
1	地震	{"text": "宜宾珙县发生5.6级地震,网友:我开车都感觉到在动", "id": "e86f7a200339", "event_list": [{"event_type": "灾害/意外-地震", "trigger": "地震", "trigger_start_index": 13, "arguments": [{"argument_start_index": 0, "role": "地点", "argument": "宜宾珙县",{"argument_start_index": 9, "role": "震级", "argument": "5.6级"}], "class": "灾害/意外"}]}
2	坍塌	{"text": "青岛地铁一施工段发生坍塌初步确定5人被困", "id": "e62884f833cb4", "event_list": [{"event_type": "灾害/意外-坍/垮塌", "trigger": "坍塌", "trigger_start_index": 10, "arguments": [{"argument_start_index": 0, "role": "坍塌主体", "argument": "青岛地铁一施工段"}], "class": "灾害/意外"}]}
3	洪灾	{"text": "2019年8月6日,湖北十堰山洪已致8人死亡,应急管理部派工作组救援工作", "id": "118d7ee0a6053", "event_list": [{"event_type": "灾害/意外-洪灾", "trigger": "山洪", "trigger_start_index": 14, "arguments": [{"argument_start_index": 10, "role": "地点", "argument": "湖北十堰"},{"argument_start_index": 0, "role": "时间", "argument": "2019年8月6日"},]}

软/硬件	版本/配置
操作系统	Ubuntu
CPU	I7-6700HQ@2.60Hz
GPU	GTX-1080 Ti(8GB)
内存	32Gb
Python	3.7.4
Tensorflow	1.14.0
Keras	2.3.1

模型	序号	key_size	Epoch	P/%	R/%	F₁/%
Small	1	12	10	56.4	23.9	33.7
Small	2	12	20	68.3	49.1	56.6
Small+	3	128	10	69.4	30.3	42.9
Small+	4	128	20	73.3	55.3	64.4
Tiny	5	12	10	67.3	17.8	29.9
Tiny	6	12	20	68.9	42.6	52.3
Tiny+	7	118	10	59.4	21.1	35.8
Tiny+	8	118	20	71.5	44.4	54.8

模型	文本长度l	正确数量
Tiny+	0 < l ≤32	54
	32 < l ≤64	53
	64 < l ≤96	61
	96 < l ≤128	63
Small+	0 < l ≤32	59
	32 < l ≤64	66
	64 < l ≤96	74
	96 < l ≤128	73

基于BERT的灾害三元组信息抽取优化研究

Research on optimization of disaster triplet information extraction based on BERT

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 15

相关文章 0

编辑推荐

Metrics

本文评价