Abstract:

To improve the interaction ability of traffic vehicles in the cut-in scenario, a method for constructing a vehicle hazardous cut-in strategy based on deep reinforcement learning was proposed. Firstly, a simulated environment was built based on scalable multi-agent reinforcement learning training school(SMARTS) simulation platform. Then, twin delayed deep deterministic policy gradients (TD3) algorithm was adopted to train an agent to cut in a randomly chosen target vehicle hazardously. The algorithm was compared with proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms. The trained model was tested in seven different scenarios with varying traffic densities. Finally, a multi-agent testing environment was built, and the trained model was applied to validate intelligent driving strategies. The results show that the success rate of hazardous cut-ins reaches 80.35% in model training with TD3 algorithm, outperforming both comparative methods. In model testing, except for the 2 700 vehicle/h test scenario, the model achieves a hazardous cut-in success rate of over 80% in the other three test scenarios that were not used in training, demonstrating good generalization ability. Meanwhile, the time to collision values between the ego vehicle and the target vehicle at the moment of lane changes are concentrated within the range of 0 to 6 seconds, with 95% falling within this bracket. The proportions of time to collision values in the intervals of (0,2], (2,4], (4,6]s are 60%, 30%, and 5% respectively, covering test conditions with different collision risk. In the validation of intelligent driving strategies, the traffic vehicle controlled by the trained model can actively perform cut-ins in front of the test vehicles, exposing it to the risk of a rear-end collision and helping in identifying safety vulnerabilities in intelligent driving strategies.

Key words: autonomous driving, vehicle hazardous cut-in, virtual tests, hazardous scenarios, reinforcement learning