Woo-sung Shim

Semantic video segmentation using both appearance and geometric information

The segmentation is the first step and core technology for semantic understanding of the video. Many tasks in the computer vision such as tracking, recognition and 3D reconstruction, etc. rely on the segmentation result as preprocessing. However, the video segmentation has been known to be a very complicated and hard problem. The objects in the video change their colors and shapes according to the surrounding illumination, the camera position, or the object motion. The color, motion, or depth has been utilized individually as a key clue for the segmentation in many researches. However, every object in the image is composed of several features such as color, texture, depth and motion. That is why single-feature based segmentation method often fails. Humans can segment the objects in video with ease because the human visual system enables to consider color, texture, depth and motion at the same time. In this paper, we propose the video segmentation algorithm which is motivated by the human visual system. The algorithm performs the video segmentation task by simultaneously utilizing the color histogram of the color, the optical flow of the motion, and the homography of the structure. Our results show that the proposed algorithm outperforms other appearance based segmentation method in terms of semantic quality of the segmentation [15]. The proposed segmentation method will serve as a basis for better high-level tasks such as recognition, tracking [3],[4] and video understanding [1].

Superpixel Coherency and Uncertainty Models for Semantic Segmentation

We present an efficient semantic segmentation algorithm based on contextual information which is constructed using super pixel-level cues. Although several semantic segmentation algorithms employing super pixel-level cues have been proposed and significant technical advances have been achieved recently, these algorithms still suffer from inaccurate super pixel estimation, recognition failure, time complexity and so on. To address problems, we propose novel super pixel coherency and uncertainty models which measure coherency of super pixel regions and uncertainty of the super pixel-wise preference, respectively. Also, we incorporate two super pixel models in an efficient inference method for the conditional random field (CRF) model. We evaluate the proposed algorithm based on MSRC and PASCAL datasets, and compare it with state-of-the-art algorithms quantitatively and qualitatively. We conclude that the proposed algorithm outperforms previous algorithms in terms of accuracy with reasonable time complexity.