Automatic Overtaking on Two-way Roads with Vehicle Interactions Based on Proximal Policy Optimization

Sie sind hier: Homepage > Suche

Automatic Overtaking on Two-way Roads with Vehicle Interactions Based on Proximal Policy Optimization

Chen, Xiaochang / Wei, Jieqiang / Renl, Xiaoqiang / Johansson, Karl H. / Wang, Xiaofan

Overtaking the lead vehicle on two-way roads in the presence of several oncoming vehicles is a complex task for autonomous vehicles. In this paper, we formulate the overtaking behavior of an ego vehicle based on a deep reinforcement learning (DRL) method. First, a two-way urban road is created, wherein the ego vehicle aims to reach the destination safely and efficiently while considering multiple traffic participants. We use different intelligent driver model (IDM) parameters to account for different drivers' habits. Furthermore, we introduce different responses of other vehicles when the ego vehicle takes overtaking maneuver. Then, a hierarchical control framework is proposed to manage vehicles on the road, which supervises vehicle behaviors at the high layer and controls the motion at the lower layer. The DRL method named Proximal Policy Optimization is applied to derive the high-level decision-making policies. A self-attention mechanism is further introduced to improve the performance of our algorithm. Finally, the overtaking maneuvers of the ego vehicle in different training timesteps are analyzed and how the responses of other vehicles affect the ego one's overtaking behavior is investigated. Simulation results show that our approach can achieve good performance to deal with the two-way road autonomous overtaking task. Supplementary video is available at https://youtu.be/jPEGjM7cBuk.