DNNs for multi-map semantic segmentation

Abstract

Modern vehicles include a vast number of intellectual functions such as lane-keeping assist (LKA), vehicle, pedestrian and obstacle recognition (FCW, PPS) which are implemented in the advanced driver assistance system (ADAS). These functions allow a vehicle to localize itself correctly within the road lane and to increase the overall system safety. It is also critical for vehicle motion and planning the target trajectory. Previously, algorithms implementing ADAS functions were based on classical computer vision approaches (e.g. edge detection, morphology, Hough transform), which did well only on a rather simple road scene. Modern state-of-the-art systems are based on semantic segmentation networks, it is the unquestionable trend. With a more thorough study of the road scene segmentation issues we face the problem that the existing benchmark suites such as MOTS, KITTI as well as recent DNNs for the road/lane semantic segmentation employ only mutually exclusive classes i.e. in this case, a pixel can belong to a single class only. But if we recognize the road scene, a pixel can easily refer to several classes, e.g. to ego-lane and crosswalk. The classical approach with mutually exclusive classes will give preference to only one class in this case and we will get an ego-lane consisting of two components. As a result, it may be difficult to restore the ego-lane at the stage of post-processing, see Figure 1. To overcome this problem, in the paper we propose the approach with multiple segmentation maps as an output of the DNN architecture, as well as a multi-map loss function. In this case, each pixel is referred to several classes at the same time (depending on the number of layers) and we don’t have the restriction to use mutually exclusive classes. The DNN classifier for each segmentation map has a separate activation branch and loss function.