Highlight • A velocity control method with considering the following vehicle for car-following behavior is proposed. • Microscopic trajectory data are applied to construct the three-vehicles mode and to simulate future mixed traffic flow scenarios. • Soft actor-critic algorithm is adopted as the velocity control strategy and the reward function is designed for balancing the safety degree in the three vehicles. • Safety and efficiency improvements are validated by comparing with the naturalistic trajectory data.

Abstract Car-following behavior is a common driving behavior. It is necessary to consider the following vehicle in the car-following model of autonomous vehicle (AV) under the background of the vehicle-to-vehicle transportation system. In this study, a safe velocity control method for AV based on reinforcement learning with considering the following vehicle is proposed. First, the mixed driving environment of AVs and human-driven vehicles is constructed, and the trajectories of the leading and following vehicles are extracted from the naturalistic High D driving dataset. Next, the soft actor-critic (SAC) algorithm is used as the velocity control algorithm, in which the agent is AV, the action is acceleration, and the state is the relative distance and relative speed between the AV and the leading and following vehicles. Then, a reward function based on state and corresponding action is designed to guide AV to choose acceleration without collision between the leading and following vehicles. Furthermore, AVs are gradually able to learn to avoid collisions between the leading and following vehicles after training the model. The test result of the trained model shows that the SAC agent can achieve complete collision avoidance, resulting in zero collision. Finally, the driving performance of the SAC agent and that of human driving are compared and analyzed for safety and efficiency. The results of this study are expected to improve the safety of the car-following process..