Ilkoo Ahn

Depth-Based Disocclusion Filling for Virtual View Synthesis

Free-viewpoint rendering (FVR) has become a popular topic in 3D research. A promising technology in FVR is to generate virtual views using a single texture image and the corresponding depth image. A critical problem that occurs when generating virtual views is that the regions covered by the foreground objects in the original view may be disoccluded in the synthesized views. In this paper, a depth based disocclusion filling algorithm using patch based texture synthesis is proposed. In contrast to the existing patch based virtual view synthesis methods, the filling priority is driven by the robust structure tensor and the epipolar directional term. Moreover, the best-matched patch is searched in the background regions and finally the best-matched patch is chosen by considering the color similarity and some factors such as epipolar line and the magnitude of data term. Superiority of the proposed method over the existing methods is proved by comparing the experimental results.

A Novel Depth-Based Virtual View Synthesis Method for Free Viewpoint Video

Free-viewpoint rendering (FVR) has become a popular topic in 3-D research. A promising technology in FVR is to generate virtual views using a single texture image and the corresponding depth image. A critical problem that occurs when generating virtual views is that the regions covered by the foreground objects in the original view may be disoccluded in the synthesized views. In this paper, a depth based disocclusion filling algorithm using patch-based texture synthesis is proposed. In contrast to the existing patch-based virtual view synthesis methods, the filling priority is driven by the robust structure tensor that efficiently reflects the overall structure of an image part and a new confidence term that produces fine synthesis results even near the foreground boundaries. Moreover, the best-matched patch is searched in the background regions and finally it is chosen through a new patch distance measure. Significant superiority of the proposed method over the state-of-the-art methods is presented by comparing the experimental results.

An Intelligent Display Scheme of Soccer Video on Mobile Devices

The rapid progress of the multimedia signal processing has contributed to the extensive use of multimedia devices with small liquid crystal display (LCD) panel. With the proliferation of mobile devices with a small display, the video sequences captured for normal viewing on standard TV or HDTV may give the small-display viewers uncomfortable experience in understanding what is happening in a scene. For instance, in a soccer video sequence taken by a long-shot camera technique, the tiny objects (e.g., soccer ball and players) may not be clearly visible on the small LCD panel. Thus, an intelligent display technique is needed for small-display viewers. To this end, one of the key technologies is to determine region of interest (ROI), whenever ROI display is considered more effective. In this paper, a fully automatic and computationally efficient method is proposed for intelligent display of soccer video on small multimedia capable mobile devices. In this work, instead of taking visual saliency into account, we adopt a domain-specific approach to exploit the attributes of soccer video. The proposed scheme includes three key steps: ground color learning, shot classification, and ROI determination. Experimental results show that the proposed scheme offers useful tools for intelligent video display for small mobile devices.

Ground Color Customization of Soccer Videos

In this paper, we present a method for customizing the ground color in outdoor sports video to provide TV viewers with a better viewing experience. This relates to content personalization and the issue is becoming critical with the advent of mobile TV and interactive TV such as IPTV. In the case of watching outdoor sports video such as soccer video, it is sometimes observed that the ground color is not satisfied by viewers. In this work, the proposed algorithm is focused on customizing the ground color to deliver viewers better viewing experiences. The algorithm comprises three modules; ground detection, long-shot detection, and ground color customization. We customize the ground color by considering the difference between ground colors from both input video and target video. Experimental results show that the proposed scheme offers useful tools to provide a more comfortable viewing environment and is amenable to realtime performance even in a software based implementation.

Face and Hair Region Labeling Using Semi-Supervised Spectral Clustering-Based Multiple Segmentations

The multiple segmentation (MS) scheme is considered to be a way to get a better spatial support for various shaped objects in image segmentation. The MS scheme assumes that the segmented regions (i.e., segments) can be treated as hypotheses for object support rather than mere partitionings of the image. As for attaining each segmentation in the MS scheme, one of the most popular methods is to employ spectral clustering (SC). When applied to image segmentation tasks, SC groups a set of pixels or small regions into unique segments. While it has been popularly used in image segmentation, it often fails to deal with images containing objects with complex boundaries. To split the image as close to the object boundaries as possible, some prior knowledge can be used to guide the clustering algorithm toward appropriate partitioning of the data. In semisupervised clustering, prior knowledge is often formulated as pairwise constraints. In this paper, we propose an MS technique combined with constrained SC to build a face and hair region labeler. To put it concretely, pairwise constraints modified to fit the problem of labeling face regions are added to SC and multiple segments are generated by the constrained SC. Then, the labeling is conducted by estimating the likelihoods for each segment to belong to the target object classes. Experiments are conducted on three datasets and the results show that the proposed scheme offers useful tools for labeling the face images.

Intelligent video display to raise quality of experience on mobile devices

Mobile devices have been transformed from voice communication tools to advanced tools for consuming multimedia contents. The extensive use of such mobile devices entails watching multimedia contents on small LCD panels. However, the most of video sequences are captured for normal viewing on standard TV or HDTV, but for cost reasons, merely resized and delivered without additional editing. This may give the small-display-viewers uncomfortable experiences in understanding what is happening in a scene. For instance, in a soccer video sequence taken by a long-shot camera technique, the tiny objects (e.g., soccer ball and players) may not be clearly viewed on the small LCD panel. Thus, an intelligent display technique needs to be developed to provide small-display-viewers with better experience. To this end, one of the key technologies is to determine region of interest (ROI) and display the magnified ROI on the screen, where ROI is a part of the scene that viewers pay more attention to than other regions. In this paper, which is an extension from our prior work, we focus on soccer video display for mobile devices, and a fully automatic and computationally efficient method is proposed. Instead of taking generic approaches utilizing visually salient features, we take domain-specific approach to exploit the attributes of the soccer video. The proposed scheme consists of two stages: shot classification, and ROI determination. The experimental results show that the proposed scheme offers useful tools for intelligent video display for multimedia mobile devices.