Dynamic 3D Environment Perception Using Monocular Vision and Semantic Segmentation

Abstract

This paper presents a complete system for traffic environment perception based on a single color image source. The acquired color images are segmented into road and obstacle areas using a U-Net style Convolutional Neural Network (CNN). The segmented image is mapped into a bird’s-eye view using the automatically calibrated camera parameters. Obstacle scans are extracted from the bird’s-eye view image, highlighting the contact points between the obstacles and the road. The scans are postprocessed to increase the connectivity between obstacle parts. The processed scans are used to generate the measurement likelihood values for all observable cells of a dynamic occupancy grid, taking into consideration the expected measurement errors with respect to the distance from the camera. A particle-based occupancy grid is used to track the environment at cell level, and then the occupied cells are grouped into individual objects. The system is able to estimate stable cuboids with measured width, length, orientation and speed for the moving individual objects such as vehicles and pedestrians, and also to identify generic occupied areas for the continuous structures such as fences or barriers.