Jong-sul Min

A High Quality Depth Map Upsampling Method Robust to Misalignment of Depth and Color Boundaries

In recent years, fusion camera systems that consist of color cameras and Time-of-Flight (TOF) depth sensors have been popularly used due to its depth sensing capability at real-time frame rates. However, captured depth maps are limited in low resolution compared to the corresponding color images due to physical limitation of the TOF depth sensor. Most approaches to enhancing the resolution of captured depth maps depend on the implicit assumption that when neighboring pixels in the color image have similar values, they are also similar in depth. Although many algorithms have been proposed, they still yield erroneous results, especially when region boundaries in the depth map and the color image are not aligned. We therefore propose a novel kernel regression framework to generate the high quality depth map. Our proposed filter is based on the vector pointing similar pixels that represents the unit vector toward similar neighbors in the local region. The vectors are used to detect misaligned regions between color edges and depth edges. Unlike conventional kernel regression methods, our method properly handles misaligned regions by introducing the numerical analysis of the local structure into the kernel regression framework. Experimental comparisons with other data fusion techniques prove the superiority of the proposed algorithm.

Trilateral filter construction for depth map upsampling

In recent years, fusion camera systems that consist of color cameras and Time-of-Flight (TOF) depth sensors have been popularly used due to its depth sensing capability at real-time frame rates. However, captured depth maps are limited in low resolution compared to the corresponding color images due to physical limitation of the TOF depth sensor. Although many algorithms have been proposed, they still yield erroneous results, especially when boundaries of the depth map and the color image are not aligned. We therefore propose a novel kernel regression framework to generate the high quality depth map. Our proposed filter is based on the vector pointing homogeneous pixels that represents the unit vector toward similar neighbors in the local region. The vectors are used to detect misaligned regions between color edges and depth edges. Experimental comparisons with other data fusion techniques prove the superiority of the proposed algorithm.

Photometric and geometric rectification for stereoscopic images

Stereoscopic images are captured by two cameras at different positions. In general, the two images often have geometric distortions including vertical misalignment, rotation and keystone as well as photometric distortions such as luminance or color differences. Even in case of a carefully designed parallel stereo camera configuration, the captured images pairs may have the distortions that cause uncomfortable 3D experiences to users. In this paper, we develop an algorithm to correct the captured image pairs to give a comfortable stereoscopic experience to users. The algorithm provides a practical method for compensating the photometric and geometrical distortions.

AAM-based face reorientation using a single camera

In a video conference, eye-contact between conferees is an important issue because it makes them feel immersive and friendly. An eye-contact feeling does not mean only parallel gaze angle between conferees. For a real eye-contact feeling, it is necessary that the conferees face is also perpendicular to the camera so that users feel a face-to-face conversation to the conferee. In this paper, we present a new method to reorient a face for the real eye-contact using a generic 3D face model based on a single camera.