关键词: 燃气管网, 燃气知识双向变换器(Gas-kBERT)模型, 自然语言处理(NLP), 知识注入, 双向变换器(BERT)模型

Abstract:

In order to enhance emergency management in the field of gas pipeline networks, Gas-kBERT model was proposed. The model incorporated data from the gas pipeline network field expanded by Chat Generative Pre-Trained Transformer,(ChatGPT)and Chinese Gas Language Understanding Subject-Predicate-Object(CGLU-Spo) and related corpora were constructed in this field. By altering the model's masking (MASK) mechanism, domain knowledge was successfully injected into the model. Considering the professionalism and specificity of the gas pipeline network field, Gas-kBERT was pre-trained on various scales and contents of corpora and fine-tuned on named entity recognition and classification tasks within this field. Experimental results demonstrated that, compared to the general BERT model, Gas-kBERT exhibited significant performance improvements in F₁-score in text mining tasks in the gas pipeline network field. Specifically, in the named entity recognition task, the F₁-score was increased by 29.55%, and in the text classification task, the F₁-score improvement reached up to 83.33%. This study proves that the Gas-kBERT model performs exceptionally well in text mining tasks in the gas pipeline network field.

Key words: gas pipeline networks, gas knowledge bidirectional encoder representations from transformers(Gas-kBERT)model, natural language processing(NLP), knowledge injection, bidirectional encoder representations from transformers (BERT)