A Lightweight Simulation Framework for Learning Control Policies for Autonomous Vehicles in Real-World Traffic Condition

Abstract

We present a new simulation framework for learning control policies for autonomous vehicles (AVs) based on real-world vehicle data and maps. The framework we propose consists of three major components: a) creating a detailed lane-level map (i.e., a high definition (HD) map) for a region of interest, b) generating an environment on the HD-map using the sensor outputs (e.g., GPS, radar) from vehicles driving in the same region, and c) learning a control policy based on the realistic environment constructed. We created the lane-level HD-maps using open street maps (OSM) and aerial imagery, from which we extracted the lane-level marking and edge features. The extracted image features are then utilized to calculate higher level attributes (e.g., curvature, heading, cross-sections etc.) for each point in the HD-map. The data acquired from vehicle sensors is combined with the constructed map to create a realistic environment. Based on the constructed environment, we learned a policy to control the vehicle laterally using a reinforcement learning algorithm and longitudinally using proportional-integral-derivative (PID) controller. Our experimental results show that the proposed framework works well, offering a flexible and scalable solution for learning control policies for AVs in realistic environments.