Joint CRF and Locality-Consistent Dictionary Learning for Semantic Segmentation

Abstract

Semantic image segmentation can be accomplished by assigning a proper object category label to each meaningful region of an image. Beyond the original bottom-up models, the use of top-down categorization information has been applied to semantic segmentation to improve performance. An excellent example of such a top-down scheme is to integrate a Conditional Random Field (CRF) model with sparse dictionary learning. However, the existing solutions merely consider the discrimination of dictionaries to obtain better sparse codes, without considering the inherent data locality characteristics. In this paper, we explore such characteristics and propose a novel semantic segmentation framework based on an innovative CRF model with locality-consistent dictionary learning. In particular, we propose two new locality-consistent dictionary learning strategies by capturing the local consistencies in the feature space and the label space. In addition, we develop a joint dictionary and a CRF model parameter learning algorithm to seamlessly integrate the proposed locality-consistent dictionary learning strategies into the CRF model. Extensive experiments are conducted with two popular databases of different traits (i.e., Graz-02 and PASCAL-CONTEXT). The simulation results confirm the efficiency of the proposed scheme, especially when training data are limited.