中国安全科学学报 ›› 2024, Vol. 34 ›› Issue (7): 219-228.doi: 10.16265/j.cnki.issn1003-3033.2024.07.0133

• 防灾减灾技术与工程 • 上一篇    下一篇

基于暴雨灾害短视频的多模态情感特征研究

晋良海1,2,3(), 王抒情1,2, 王昕煜1,2   

  1. 1 三峡大学 水电工程施工与管理湖北省重点实验室,湖北 宜昌 443002
    2 三峡大学 水利与环境学院,湖北 宜昌 443002
    3 三峡大学 安全生产标准化评审中心,湖北 宜昌 443002
  • 收稿日期:2024-01-02 修回日期:2024-04-12 出版日期:2024-07-28
  • 作者简介:

    晋良海 (1973—),男,四川简阳人,博士,教授,博士生导师,主要从事安全工效学、建设项目运筹管理等方面的研究。E-mail:

  • 基金资助:
    教育部人文社科基金资助(21YJA630038); 中国长江三峡集团有限公司企业科研项目(202103551)

Research on multimodal emotion characteristics based on short video of rainstorm disaster

JIN Lianghai1,2,3(), WANG Shuqing1,2, WANG Xinyu1,2   

  1. 1 Hubei Provincial Key Laboratory of Construction and Management of Hydropower Engineering, China Three Gorges University, Yichang Hubei 443002, China
    2 College of Hydraulic & Environmental Engineering, China Three Gorges University, Yichang Hubei 443002, China
    3 Safety Production Standardization Evaluation Center, China Three Gorges University, Yichang Hubei 443002, China
  • Received:2024-01-02 Revised:2024-04-12 Published:2024-07-28

摘要:

为提高灾害应对效率,将“河北暴雨”“黑龙江暴雨”作为典型跨区域研究案例,收集短视频的文本-图像-音频多模态数据;面对海量的非结构化数据,运用深度学习技术,构建多模态情感智能分类模型,实现短视频情感特征的提取、跨模态融合与智能情感分类;并综合利用时空大数据,在时空维度上深度挖掘与分析暴雨灾害短视频多模态情感特征。结果表明:该模型准确率达85%以上,能有效实现短视频多模态情感智能分类任务。在时间维度上,网民情感波动与暴雨灾害周期大致相符,可作为判断灾情严重程度和舆情走向的依据;媒体及政府的干预对暴雨灾害情感演化具有重要作用。在空间维度上,消极情感随灾情转移呈现“低—高—低”变化趋势,且消极情感的共鸣和扩散效应呈现明显的地域性特征,需重视灾区、我国部分东部地区及类似灾害频发的非灾区舆情引导工作。

关键词: 暴雨灾害, 短视频, 多模态情感特征, 深度学习, 情感分类

Abstract:

To improve the efficiency of disaster response, the "Hebei rainstorm" and "Heilongjiang rainstorm" were adopted as illustrative cross-regional research cases, and text-image-audio multimodal data were collected from short videos. In the face of massive unstructured data, deep learning technology was employed to realize the extraction of multimodal emotional features, cross-modal integration and intelligent sentiment classification in short videos. By comprehensively using spatial and temporal big data, the multimodal emotional characteristics of short video of rainstorm disaster were deeply mined and analyzed in the spatial and temporal dimension. The results indicate that the model's accuracy exceeds 85%, efficiently fulfilling the objectives set for short video analysis. From the temporal perspective, the emotional fluctuations of netizens broadly align with the cycle of rainstorm disasters, providing a basis for assessing disaster severity and public opinion trends. Furthermore, the intervention of media and government entities plays a significant role in shaping the emotional evolution surrounding rainstorm disasters. In terms of spatial dimensions, negative emotions exhibit a "low-high-low" trend as disasters shift locations, and the resonance and diffusion of these emotions display distinct regional characteristics. Therefore, it is imperative to prioritize public opinion guidance in disaster-stricken areas, as well as in some eastern regions of China and non-disaster areas experiencing similar phenomena.

Key words: rainstorm disaster, short video, multimodal emotion characteristics, deep learning, emotional classification

中图分类号: