DrsNet: Dual-resolution semantic segmentation with rare class-oriented superpixel prior

Abstract

Rare-class objects in natural scene images that are usually small and less frequent often convey more important information for scene understanding than the common ones. However, they are often overlooked in scene labeling studies due to two main reasons, low occurrence frequency and limited spatial coverage. Many methods have been proposed to enhance overall semantic labeling performance, but only a few consider rare-class objects. In this work, we present a deep semantic labeling framework with special consideration of rare classes via three techniques. First, a novel dual-resolution coarse-to-fine superpixel representation is developed, where fine and coarse superpixels are applied to rare classes and background areas respectively. This unique dual representation allows seamless incorporation of shape features into integrated global and local convolutional neural network (CNN) models. Second, shape information is directly involved during the CNN feature learning for both frequent and rare classes from the re-balanced training data, and also explicitly involved in data inference. Third, the proposed framework incorporates both shape information and the CNN architecture into semantic labeling through a fusion of probabilistic multi-class likelihood. Experimental results demonstrate competitive semantic labeling performance on two standard datasets both qualitatively and quantitatively, especially for rare-class objects.