用于自动驾驶测试的车辆危险切入策略研究

doi:10.16265/j.cnki.issn1003-3033.2025.01.0540

中国安全科学学报 ›› 2025, Vol. 35 ›› Issue (1): 112-119.doi: 10.16265/j.cnki.issn1003-3033.2025.01.0540

用于自动驾驶测试的车辆危险切入策略研究

周扬¹^,²(), 陈运星²^,³^,^**(), 吴玲¹

¹ 西安航空学院车辆工程学院,陕西西安 710077
² 湖北文理学院纯电动汽车动力系统设计与测试湖北省重点实验室,湖北襄阳 441053
³ 湖北文理学院汽车与交通工程学院,湖北襄阳 441053

收稿日期:2024-08-11 修回日期:2024-10-20 出版日期:2025-01-28
通信作者:
**陈运星(1987—),男,湖北荆门人,博士,副教授,主要从事驾驶行为感知、智能驾驶技术等方面的研究。E-mail:chenyunxing@chd.edu.cn。
作者简介:
周扬 (1989—),男,陕西汉中人,博士,副教授,主要从事人-车-路系统安全、自动驾驶测试等方面的研究。E-mail:297399014@qq.com。
陈运星副教授
吴玲副教授
基金资助:
国家自然科学基金资助(51908054); 陕西省科技厅自然科学基础研究计划项目(2024JC-YBMS-301); 湖北省技术创新计划科技重大项目(2024BAA011); 纯电动汽车动力系统设计与测试湖北省重点实验室开放基金资助(ZDSYS202310)

Research on vehicle hazardous cut-in strategy used in autonomous driving test

ZHOU Yang¹^,²(), CHEN Yunxing²^,³^,^**(), WU Ling¹

¹ School of Vehicle Engineering, Xi'an Aeronautical Institute, Xi'an Shaanxi 710077, China
² Hubei Key Laboratory of Power System Design and Test for Electrical Vehicle, Hubei University of Arts and Science, Xiangyang Hubei 441053, China
³ School of Automotive and Traffic Engineering, Hubei University of Arts and Science, Xiangyang Hubei 441053, China

Received:2024-08-11 Revised:2024-10-20 Published:2025-01-28

摘要/Abstract

摘要：

为提高车辆切入测试场景中交通车的交互能力,提出一种基于深度强化学习方法的车辆危险切入策略设计方法。首先,基于可扩展多智能体强化学习培训学校(SMARTS)仿真平台构建仿真环境;然后,采用双延迟深度确定性策略梯度算法(TD3)训练智能体危险切入随机选定的目标车辆,将该算法与近端策略优化算法(PPO)和深度确定性策略梯度算法(DDPG)进行对比,在7种不同车辆密度的场景中测试训练后的模型;最后,构建多智能体测试环境,将所训练模型用于智能驾驶策略的验证。结果表明: 模型在训练中的危险切入成功率达80.35%,优于2种对比方法;在模型测试中,除2 700辆/h测试场景外,该模型在另外3个未在训练中使用的测试场景均达到80%以上的危险切入成功率,显示出良好的泛化能力。同时,切入时刻与目标车的碰撞时间值显示95%集中在0~6 s,取值在(0,2]、(2,4]和(4,6]s的占比分别为60%、30%和5%,可覆盖具有不同碰撞风险的测试工况。在智能驾驶策略验证中,采用所训练模型控制的交通车能主动切入至待测车辆前方,使待测车辆面临追尾风险,有助于发现智能驾驶策略的安全隐患。

关键词: 自动驾驶, 车辆危险切入, 虚拟测试, 危险场景, 强化学习

Abstract:

To improve the interaction ability of traffic vehicles in the cut-in scenario, a method for constructing a vehicle hazardous cut-in strategy based on deep reinforcement learning was proposed. Firstly, a simulated environment was built based on scalable multi-agent reinforcement learning training school(SMARTS) simulation platform. Then, twin delayed deep deterministic policy gradients (TD3) algorithm was adopted to train an agent to cut in a randomly chosen target vehicle hazardously. The algorithm was compared with proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms. The trained model was tested in seven different scenarios with varying traffic densities. Finally, a multi-agent testing environment was built, and the trained model was applied to validate intelligent driving strategies. The results show that the success rate of hazardous cut-ins reaches 80.35% in model training with TD3 algorithm, outperforming both comparative methods. In model testing, except for the 2 700 vehicle/h test scenario, the model achieves a hazardous cut-in success rate of over 80% in the other three test scenarios that were not used in training, demonstrating good generalization ability. Meanwhile, the time to collision values between the ego vehicle and the target vehicle at the moment of lane changes are concentrated within the range of 0 to 6 seconds, with 95% falling within this bracket. The proportions of time to collision values in the intervals of (0,2], (2,4], (4,6]s are 60%, 30%, and 5% respectively, covering test conditions with different collision risk. In the validation of intelligent driving strategies, the traffic vehicle controlled by the trained model can actively perform cut-ins in front of the test vehicles, exposing it to the risk of a rear-end collision and helping in identifying safety vulnerabilities in intelligent driving strategies.

Key words: autonomous driving, vehicle hazardous cut-in, virtual tests, hazardous scenarios, reinforcement learning

中图分类号:

周扬, 陈运星, 吴玲. 用于自动驾驶测试的车辆危险切入策略研究[J]. 中国安全科学学报, 2025, 35(1): 112-119.

ZHOU Yang, CHEN Yunxing, WU Ling. Research on vehicle hazardous cut-in strategy used in autonomous driving test[J]. China Safety Science Journal, 2025, 35(1): 112-119.

图/表 11

图1

表1

状态表示

类型	特征描述
自车	纵向车速、横向车速、朝向、车道偏移距离
附近车辆(i为车辆数)	距离 $d i$ 、纵向距离 $d i x$ 、横向距离 $d i y$
目标点与目标车(T为目标车,m为目标点)	与目标点的距离为 $d m$ ,自车与目标车的纵向距离 $d T x$ ,横向距离 $d T y$ ,相对车速 $Δ v, 碰撞时间 o = d x T 2 + d y T 2 Δ v$

表1

图2

表2

图3

图4

表3

图5

图6

图7

图8

参考文献 17

[1]	朱冰, 张培兴, 赵健, 等. 基于场景的自动驾驶汽车虚拟测试研究进展[J]. 中国公路学报, 2019, 32(6): 1-19. doi: 10.19721/j.cnki.1001-7372.2019.06.001
	ZHU Bing, ZHANG Peixing, ZHAO Jian, et al. Review of scenario-based virtual validation methods for automated vehicle[J]. China Journal of Highway and Transport, 2019, 32(6): 1-19. doi: 10.19721/j.cnki.1001-7372.2019.06.001
[2]	邓伟文, 李江坤, 任秉韬, 等. 面向自动驾驶的仿真场景自动生成方法综述[J]. 中国公路学报, 2022, 35(1): 316-333. doi: 10.19721/j.cnki.1001-7372.2022.01.027
	DENG Weiwen, LI Jiangkun, REN Bingtao, et al. A survey on automatic simulation scenario generation methods for autonomous driving[J]. China Journal of Highway and Transport, 2022, 35(1): 316-333. doi: 10.19721/j.cnki.1001-7372.2022.01.027
[3]	徐向阳, 胡文浩, 董红磊, 等. 自动驾驶汽车测试场景构建关键技术综述[J]. 汽车工程, 2021, 43(4): 610-619.
	XU Xiangyang, HU Wenhao, DONG Honglei, et al. Review of key technologies for autonomous vehicle test scenario construction[J]. Automotive Engineering, 2021, 43(4): 610-619.
[4]	郭柏苍, 雒国凤, 金立生, 等. 面向自动驾驶虚拟测试的变道切入场景库构建方法[J]. 吉林大学学报:工学版, 2023, 53(11): 3130-3140.
	GUO Baicang, LUO Guofeng, JIN Lisheng, et al. Construction method of cut-in scenario library for automatic driving virtual tests[J]. Journal of Jilin University: Engineering and Technology Edition, 2023, 53(11): 3130-3140.
[5]	赵祥模, 赵玉钰, 景首才, 等. 面向自动驾驶测试的危险变道场景泛化生成[J]. 自动化学报, 2023, 49 (10): 2211-2223.
	ZHAO Xiangmo, ZHAO Yuyu, JING Shoucai, et al. Generalization generation of hazardous lane-changing scenarios for automated vehicle testing[J]. Acta Automatica Sinica, 2023, 49 (10): 2211-2223.
[6]	朱冰, 范天昕, 赵健, 等. 基于危险边界搜索的自动驾驶系统加速测试方法[J]. 吉林大学学报:工学版, 2023, 53(3): 704-712.
	ZHU Bing, FAN Tianxin, ZHAO Jian, et al. Accelerate test method of automated driving system based on hazardous boundary search[J]. Journal of Jilin University: Engineering and Technology Edition, 2023, 53(3): 704-712.
[7]	FENG Shuo, YAN Xintao, SUN Haowei, et al. Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment[J]. Nature Communications, 2021, 12(1): DOI: 10.1038/s41467-021-21007-8.
[8]	FENG Shuo, SUN Haowei, YAN Xintao, et al. Dense reinforcement learning for safety validation of autonomous vehicles[J]. Nature, 2023, 615(7953): 620-627.
[9]	SUN Haowei, FENG Shuo, YAN Xintao, et al. Corner case generation and analysis for safety assessment of autonomous vehicles[J]. Transportation Research Record, 2021, 2675(11): 587-600.
[10]	李江坤, 邓伟文, 任秉韬, 等. 基于场景动力学和强化学习的自动驾驶边缘测试场景生成方法[J]. 汽车工程, 2022, 44 (7): 976-986.
	LI Jiangkun, DENG Weiwen, REN Bingtao, et al. Automatic driving edge scene generation method based on scene dynamics and reinforcement learning[J]. Automotive Engineering, 2022, 44(7): 976-986.
[11]	ZHOU Ming, LUO Jun, VILLELLA J, et al. Smarts: an open-source scalable multi-agent reinforcement training school for autonomous driving[C]. Conference on Robot Learning, 2021: 264-285.
[12]	罗崎瑞, 张道文, 周华, 等. 面向智能汽车预期功能安全的驾驶场景评价[J]. 中国安全科学学报, 2022, 32(8):140-145. doi: 10.16265/j.cnki.issn1003-3033.2022.08.1768
	LUO Qirui, ZHANG Daowen, ZHOU Hua, et al. Evaluation on driving scenarios for safety of intended functionality of intelligent vehicles[J]. China Safety Science Journal, 2022, 32(8):140-145. doi: 10.16265/j.cnki.issn1003-3033.2022.08.1768
[13]	胡祥旺, 倪颖, 孙剑. 车联网环境下匝道汇入区瓶颈换道优化[J]. 同济大学学报:自然科学版, 2023, 51(9):1424-1432.
	HU Xiangwang, NI Ying, SUN Jian. Freeway merging area lane changing advisory optimization under connected vehicles environment[J]. Journal of Tongji University: Natural Science, 2023, 51(9): 1424-1432.
[14]	周文帅, 朱宇, 赵祥模, 等. 面向高速公路车辆切入场景的自动驾驶测试用例生成方法[J]. 汽车技术, 2021(1): 11-18.
	ZHOU Wenshuai, ZHU Yu, ZHAO Xiangmo, et al. Vehicle cut-in test case generation methods for testing of autonomous driving on highway[J]. Automobile Technology, 2021(1): 11-18.
[15]	FUJIMOTO S, HOOF H, MEGER D. Addressing function approximation error in actor-critic methods[C]. International Conference on Machine Learning (ICML), 2018: 1587-1596.
[16]	ZHOU Yang, CHEN Yunxing. Learning to drive in the NGSIM simulator using proximal policy optimization[J]. Journal of Advanced Transportation, 2023, 2023: DOI: 10.1155/2023/4127486.
[17]	CHEN Baiming, CHEN Xiang, WU Qiong, et al. Adversarial evaluation of autonomous vehicles in lane-change scenarios[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 10 333-10 342.

参数	取值	参数	取值	参数	取值
学习率	3×10^-4	经验回放池大小	1×10⁶	软更新率	0.005
批大小	100	折扣因子	0.99	总步数	5×10⁵

用于自动驾驶测试的车辆危险切入策略研究

Research on vehicle hazardous cut-in strategy used in autonomous driving test

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 17

相关文章 10

编辑推荐

Metrics

本文评价

[1]	赵尧华, 陈延展, 郑亮, 李树凯. 面向自动驾驶安全测试的碰撞场景构建与仿真分析[J]. 中国安全科学学报, 2024, 34(7): 211-218.
[2]	吴维, 吴泽萱, 王兴隆, 祝龙飞. 融合强化学习的DBN跑道侵入风险预测[J]. 中国安全科学学报, 2024, 34(7): 20-27.
[3]	赵长啸, 李道俊, 孙亦轩, 景鹏, 田毅. 基于深度强化学习的综合航电系统安全性优化方法[J]. 中国安全科学学报, 2024, 34(7): 123-131.
[4]	魏明, 孙雅茹, 孙博, 王盛杰. 基于深度强化学习的无人机线路及航迹协同规划[J]. 中国安全科学学报, 2023, 33(8): 68-76.
[5]	王长帅, 徐铖铖, 邵永成, 佟昊, 彭畅. 基于生存分析的自动驾驶接管时间分析[J]. 中国安全科学学报, 2023, 33(8): 142-148.
[6]	张晖, 倪定安, 曾科, 丁乃侃, 吴超仲. 人机共驾环境下驾驶疲劳研究综述[J]. 中国安全科学学报, 2023, 33(3): 204-211.
[7]	罗崎瑞, 张道文, 周华, 庞劭荣, 李哓艳, 王朝健. 面向智能汽车预期功能安全的驾驶场景评价*[J]. 中国安全科学学报, 2022, 32(8): 140-145.
[8]	郭子慧, 郭伟伟, 谭墍元. 驾驶员接管自动驾驶车辆的眼动特性和行为分析[J]. 中国安全科学学报, 2022, 32(1): 65-71.
[9]	龙时丹. 面向高速铁路的全自动运行系统应用[J]. 中国安全科学学报, 2020, 30(S1): 92-97.
[10]	周华, 张巧玉, 母尧尧, 谭正平, 孙庆, 张道文. 十字路口乘用车与二轮车典型危险场景聚类及推演[J]. 中国安全科学学报, 2020, 30(4): 100-107.

算法	危险切入成功率/%
TD3	80.35
DDPG	29.57
PPO	68.85