Du-sik Park

Depth Map Coding Based on Synthesized View Distortion Function

This paper presents an efficient depth map coding method based on a newly defined rendering view distortion function. As compared to the conventional depth map coding in which distortion is measured by only investigating the coding error in depth map, the proposed scheme focuses on virtually synthesized view quality by involving co-located color information. In detail, the proposed distortion function estimates rendered view quality, where area-based scheme is provided in order to mimic the warping/view-rendering process accurately. Moreover, the coding performance of the proposed distortion metric is even improved by involving the additional SKIP mode derived by co-located color coding information. The simulation results show the proposed scheme could achieve approximately 30% bit-rate saving for depth data, and about 10% bit-rate saving for overall multi-view data.

Rotating your face using multi-task deep neural network

Face recognition under viewpoint and illumination changes is a difficult problem, so many researchers have tried to solve this problem by producing the pose- and illumination- invariant feature. Zhu et al. [26] changed all arbitrary pose and illumination images to the frontal view image to use for the invariant feature. In this scheme, preserving identity while rotating pose image is a crucial issue. This paper proposes a new deep architecture based on a novel type of multitask learning, which can achieve superior performance in rotating to a target-pose face image from an arbitrary pose and illumination image while preserving identity. The target pose can be controlled by the users intention. This novel type of multi-task model significantly improves identity preservation over the single task model. By using all the synthesized controlled pose images, called Controlled Pose Image (CPI), for the pose-illumination-invariant feature and voting among the multiple face recognition results, we clearly outperform the state-of-the-art algorithms by more than 4~6% on the MultiPIE dataset.

A novel 2D-to-3D conversion technique based on relative height-depth cue

We present a simple depth estimation framework for 2D-to-3D media conversion. The perceptual depth information from monocular image is estimated by the optimal use of relative height cue, which is one of well-known depth recovery cues. The height depth cue is very common in photographic images. We propose a novel line tracing method and depth refinement filter as core of our depth estimation framework. The line tracing algorithm traces strong edge positions to generate an initial staircase depth map. The initial depth map is further improved by a recursive depth refinement filter. We present visual results from depth estimation and stereo image generation.

Optimal projector configuration design for 300-Mpixel multi-projection 3D display

To achieve an immersive natural 3D experience on a large screen, a 300-Mpixel multi-projection 3D display that has a 100-inch screen and a 40° viewing angle has been developed. To increase the number of rays emanating from each pixel to 300 in the horizontal direction, three hundred projectors were used. The projector configuration is an important issue in generating a high-quality 3D image, the luminance characteristics were analyzed and the design was optimized to minimize the variation in the brightness of projected images. The rows of the projector arrays were repeatedly changed according to a predetermined row interval and the projectors were arranged in an equi-angular pitch toward the constant central point. As a result, we acquired very smooth motion parallax images without discontinuity. There is no limit of viewing distance, so natural 3D images can be viewed from 2 m to over 20 m.

Image indexing using weighted color histogram

Image indexing is the process of image retrieval from databases of images or videos based on their contents. Specifically histogram-based algorithms are considered to be effective for color image indexing. We suggest a new method of color space quantization in the CIELUV color space, named weighted LUV quantization. With this method, each bin in the LUV space has a different weighting factor, which is applied to the histogram intersection. The weighted LUV histogram intersection provides the advantage of perceptual uniformity of the CIELUV color space. An additional advantage is the consideration of perceptual sensitivity to more saturated colors by the use of a weighting factor.

A Fast and Efficient Multi-View Depth Image Coding Method Based on Temporal and Inter-View Correlations of Texture Images

Multi-view video plus depth format, which includes texture and depth images, has been recently introduced as a video representation to support depth perception of scenes and efficient view generation at arbitrary positions. Especially, a depth image has been one of the significantly important issues for successful services of 3-D video applications. In this paper, we introduce a fast and efficient multi-view depth image coding method using the texture images. The proposed method determines to skip some blocks of the depth image at the early stage without a normal encoding process including rate-distortion optimization, based on temporal and inter-view correlations between the previously encoded texture images. The skipped blocks are predicted from the neighboring depth images. Experimental results demonstrate that the proposed method not only achieves the drastically high coding performance but also reduces the complexity of the encoder.

3D user interface combining gaze and hand gestures for large-scale display

In this paper, we present a novel attentive and immersive user interface based on gaze and hand gestures for interactive large-scale displays. The combination of gaze and hand gestures provide more interesting and immersive ways to manipulate 3D information.

2D-to-3D conversion by using visual attention analysis

This paper proposes a novel 2D-to-3D conversion system based on visual attention analysis. The system was able to generate stereoscopic video from monocular video in a robust manner with no human intervention. According to our experiment, visual attention information can be used to provide rich 3D experience even when depth cues from monocular view are not enough. Using the algorithm introduced in the paper, 3D display users can watch 2D media in 3D. In addition, the algorithm can be embedded into 3D displays in order to deliver better viewing experience with more immersive feeling. Using visual attention information to give a 3D effect is first tried in this research as far as we know.

Motion information sharing mode for depth video coding

Multi-view plus depth (MVD) video is getting more popular as a new video format for 3DTV and Free viewpoint TV. By using multi-view and depth video, it is possible to efficiently render virtual views in decoder site. In this paper, we propose a motion information sharing mode for efficient depth video coding. Because the depth video is correlated with the color video especially in object boundary areas, the motion information of the color video can be shared with the depth video. Each ma-croblock selects a mode with minimum rate-distortion cost, and the proposed mode can be used as a candidate mode. Since the proposed mode is especially selected in object boundary areas, we can obtain more coding gain. Experimental results showed coding gain up to 0.8dB, but the results depend on the quality of depth video.

Plane segmentation based intra prediction for depth map coding

This paper presents an efficient intra prediction scheme for depth map coding in multi-view plus depth (MVD) system. As compared to the conventional intra prediction algorithm, the proposed method segments the current block into k regions, and applies different prediction scheme for each segmented region, which results in higher prediction accuracy by avoiding wrong estimation across different regions. Moreover, we also provide the efficient lossless coding scheme for region segmentation information based on unique depth characteristics, by which the proposed scheme is competitive w.r.t. R-D cost. The simulation results show the superior performance of the proposed scheme comparing to the H.264/AVC intra prediction and other edge-based intra scheme for depth map.