Hong-Chang Shin

Real-time view synthesis system with multi-texture structure of GPU

Glassless 3D display requires multiple images taken from different viewpoints to show a scene. Thus, generating such a large number of viewpoint images effectively is emerging as a key technique in 3D video technology. Image-based view synthesis is a technique of generating required virtual viewpoint images using a limited number of views and depth maps. In this paper, we propose an algorithm to compute virtual views much faster by using multi-texture image structure of graphics processing unit(GPU). We demonstrate the effectiveness of our algorithm for fast view synthesis through a variety of experiments with real data.

View interpolation using a simple block matching and guided image filtering

In this paper, we introduce a simple and practical solution for intermediate view interpolation using a simple block matching and a guided image filtering. Proposed method consists of two major blocks. The first block is a disparity estimation block. We focused on disparity refinement step rather than the other steps of the framework for stereo matching method. The initial disparity map is estimated using hierarchical block matching. It provides more accurate and robust result compared to pixel-wise block matching method. Then the estimated initial disparity map is refined by a guided image filtering. We use an efficient approach to find the median value when edge-ware weighted disparity layers are accumulated. Refined disparity map are used for intermediate view interpolation. With these data, block-based view interpolation is performed in order to generate virtual multi-view videos.

Depth adjustment of ROI using scalable multiplexing in 3D display

In this paper, we presents a depth adjustment technique that controls depth of region of interest using scalable multiplexing in 3D display. Autostereoscopic displays offer different 3D images depending on the viewing direction. In order to display various viewpoint images on the autostereoscopic display at the same time, we usually divide the space of display panel spatially. This is called spatial multiplexing. However, this spatial multiplexing causes degradation of each viewpoint image resolution. Each viewpoint source is resized with the ratio of the inverse proportion to the number of view due to limited resolution of 3D display. When the images are downscaled with a certain ratio and, the downscaling ratio of the object of interest is bigger than that of the others, the object of interest is enlarged more than the other region without any loss. Moreover, enlarged object region covers the other regions belong to the background of image. In this case, the covered area can offer enough space to make the object that has binocular disparity without any interpolation of holes. It can also control the objects depth in the 3D scene.

Multi-view Image Generation using Grid-mesh based Image Domain Warping and Occlusion Region Information

In this paper, we propose an algorithm that generates multi-view images by grid-mesh based image domain warping using occlusion mask and various image features obtained from the stereoscopic images. In the proposed algorithm, we first extract image saliency map, line segments and disparity saliency map from stereo images and then get them through a process that improves the quality of extracted features. This process is accomplished in two steps. In the first step, reliability of disparity saliency map on object boundary regions is enhanced by using occlusion information. And in the second step, we enhance the quality of image features in terms of temporal consistency by using temporal consistency information for stereo images. With these enhanced features, multi-view images are generated by grid-mesh based image domain warping technique. Experimental results show that the proposed algorithm performs better than existing algorithms in terms of visual quality.

Full-parallax virtual view image synthesis using image-based rendering for light-field content generation

Light-field content is required to provide full-parallax 3D view with dense angular resolution. However, it is very hard to directly capture such dense full-parallax view images using a camera system because it requires specialised micro-lens arrays or a heavy camera-array system. Therefore, we present an algorithm to synthesise full-parallax virtual view images using image-based rendering appropriate for light-field content generation. The proposed algorithm consists of four-directional image warping, view image blending using the nearest view image priority selection and the sum of the weighted inverse Euclidean distance, and hole filling. Experimental results show that dense full-parallax virtual view images can be generated from sparse full-parallax view images with fewer image artefacts. Finally, it is confirmed that the proposed full-parallax view synthesis algorithm can be used for light-field content generation without a dense camera array system.

Hierarchical bilateral filtering based disparity estimation for view synthesis

In this paper, we introduce a high efficient and practical disparity estimation using hierarchical bilateral filtering for real-time view synthesis. The proposed method is based on hierarchical stereo matching with hardware-efficient bilateral filtering. Hardware-efficient bilateral filtering is different from the exact bilateral filter. The purpose of the method is to design an edge-preserving filter that can be efficiently parallelized on hardware. The proposed hierarchical bilateral filtering based disparity estimation is essentially a coarse-to-fine use of stereo matching with bilateral filtering. It works as follows: firstly, the hierarchical image pyramid are constructed; the multi-scale algorithm then starts by applying a local stereo matching to the downsampled images at the coarsest level of the hierarchy. After the local stereo matching, the estimated disparity map is refined with the bilateral filtering. And then the refined disparity map will be adaptively upsampled to the next finer level. The upsampled disparity map used as a prior of the corresponding local stereo matching at the next level, and filtered and so on. The method we propose is essentially a combination of hierarchical stereo matching and hardware-efficient bilateral filtering. As a result, visual comparison using real-world stereoscopic video clips shows that the method gives better results than one of state-of-art methods in terms of robustness and computation time.

Scalable multiplexing for depth adjustment of ROI in 3D display

In this paper, we present a depth adjustment technique that controls the depth of regions of interest by utilizing scalable multiplexing for multi-view autostereoscopic 3D displays. The most popular used approach to adjust depth of a scene is changing the baseline distance. The baseline distance can be changed by generating virtual views, using view interpolation or extrapolation. But these kinds of approaches change the depth of objects, but the size of objects. As a matter of fact, the size of an object is one of the factors that give 3D depth perception. The proposed scalable multiplexing changes the size of an object as well as the disparity of a scene. The proposed method changes the size of an object with varying the downscaling ratio during multiplexing stage. Not only can it enlarge the size of an object without any quality loss, but it can also adjust depth of an object by shifting it. As a result, we confirmed that it is possible to adjust the depth and change size of the ROI by enlarging the ROI gradually and shifting it pixel by pixel in 3D display.