Che-Han Chang

Content-Aware Display Adaptation and Interactive Editing for Stereoscopic Images

We propose a content-aware stereoscopic image display adaptation method which simultaneously resizes a binocular image to the target resolution and adapts its depth to the comfort zone of the display while preserving the perceived shapes of prominent objects. This method does not require depth information or dense correspondences. Given the specification of the target display and a sparse set of correspondences, our method efficiently deforms the input stereoscopic images for display adaptation by solving a least-squares energy minimization problem. This can be used to adjust stereoscopic images to fit displays with different real estates, aspect ratios and comfort zones. In addition, with slight modifications to the energy function, our method allows users to interactively adjust the sizes, locations and depths of the selected objects, giving users aesthetic control for depth perception. User studies show that the method is effective at editing depth and reducing occurrences of diplopia and distortions.

Shape-Preserving Half-Projective Warps for Image Stitching

This paper proposes a novel parametric warp which is a spatial combination of a projective transformation and a similarity transformation. Given the projective transformation relating two input images, based on an analysis of the projective transformation, our method smoothly extrapolates the projective transformation of the overlapping regions into the non-overlapping regions and the resultant warp gradually changes from projective to similarity across the image. The proposed warp has the strengths of both projective and similarity warps. It provides good alignment accuracy as projective warps while preserving the perspective of individual image as similarity warps. It can also be combined with more advanced local-warp-based alignment methods such as the as-projective-as-possible warp for better alignment accuracy. With the proposed warp, the field of view can be extended by stitching images with less projective distortion (stretched shapes and enlarged sizes).

A line-structure-preserving approach to image resizing

This paper proposes a content-aware image resizing method which simultaneously preserves both salient image features and important line structure properties: parallelism, collinearity and orientation. When there are prominent line structures in the image, image resizing methods without explicitly taking these properties into account could produce line structure distortions in their results. Since the human visual system is very sensitive to line structures, such distortions often become noticeable and disturbing. Our method couples mesh deformations for image resizing with similarity transforms for line features. Mesh deformations are used to control content preservation while similarity transforms are analyzed in the Hough space to maintain line structure properties. Our method strikes a good balance between preserving content and maintaining line structure properties. Experiments show the proposed method often outperforms methods without taking line structures into account, especially for scenes with prominent line structures.

Real-Time Human Movement Retrieval and Assessment With Kinect Sensor

The difficulty of vision-based posture estimation is greatly decreased with the aid of commercial depth camera, such as Microsoft Kinect. However, there is still much to do to bridge the results of human posture estimation and the understanding of human movements. Human movement assessment is an important technique for exercise learning in the field of healthcare. In this paper, we propose an action tutor system which enables the user to interactively retrieve a learning exemplar of the target action movement and to immediately acquire motion instructions while learning it in front of the Kinect. The proposed system is composed of two stages. In the retrieval stage, nonlinear time warping algorithms are designed to retrieve video segments similar to the query movement roughly performed by the user. In the learning stage, the user learns according to the selected video exemplar, and the motion assessment including both static and dynamic differences is presented to the user in a more effective and organized way, helping him/her to perform the action movement correctly. The experiments are conducted on the videos of ten action types, and the results show that the proposed human action descriptor is representative for action video retrieval and the tutor system can effectively help the user while learning action movements.

Rectangling Stereographic Projection for Wide-Angle Image Visualization

This paper proposes a new projection model for mapping a hemisphere to a plane. Such a model can be useful for viewing wide-angle images. Our model consists of two steps. In the first step, the hemisphere is projected onto a swung surface constructed by a circular profile and a rounded rectangular trajectory. The second step maps the projected image on the swung surface onto the image plane through the perspective projection. We also propose a method for automatically determining proper parameters for the projection model based on image content. The proposed model has several advantages. It is simple, efficient and easy to control. Most importantly, it makes a better compromise between distortion minimization and line preserving than popular projection models, such as stereographic and Pannini projections. Experiments and analysis demonstrate the effectiveness of our model.

Spatially-Varying Image Warps for Scene Alignment

This paper proposes a method to align a set of images captured from multiple view points. Traditional methods using image warps parameterized by global transformations suffer from the problem of misalignment due to parallax effects induced by camera motions between images and depth variations of the scene. Our method parameterizes warps using mesh deformation and achieves spatially-varying transformations to alleviate the misalignment problem. The proposed method has two stages: a hybrid image alignment stage which combines direct-based methods and feature-based methods, followed by a shape-preserving aggregation stage which further refines the result. Experiments show that our method achieves better alignment and provides visually pleasing image summaries for scenes.

CLKN: Cascaded Lucas-Kanade Networks for Image Alignment

This paper proposes a data-driven approach for image alignment. Our main contribution is a novel network architecture that combines the strengths of convolutional neural networks (CNNs) and the Lucas-Kanade algorithm. The main component of this architecture is a Lucas-Kanade layer that performs the inverse compositional algorithm on convolutional feature maps. To train our network, we develop a cascaded feature learning method that incorporates the coarse-to-fine strategy into the training process. This method learns a pyramid representation of convolutional features in a cascaded manner and yields a cascaded network that performs coarse-to-fine alignment on the feature pyramids. We apply our model to the task of homography estimation, and perform training and evaluation on a large labeled dataset generated from the MS-COCO dataset. Experimental results show that the proposed approach significantly outperforms the other methods.

Action tutor: real-time exemplar-based sequential movement assessment with kinect sensor

With the aid of depth camera, such as Microsoft Kinect, the difficulty of vision-based posture estimation is greatly decreased, and human action analysis has achieved a wide range of applications. However, there is still much to do to develop effective movement assessment technique, which bridges the results of human posture estimation and the understanding of human action performance. In this work, we propose an action tutor system which enables the user to interactively retrieve the learning exemplar of the target action movement and to immediately acquire motion instructions while learning it in front of the Kinect. In the retrieval stage, non-linear time warping algorithms are designed to retrieve video segments similar to the query movement roughly performed by the user. In the learning stage, the user learns according to the selected video exemplar, and the motion assessment including both static and dynamic differences is presented to the user in a more effective and organized way, helping him/her to perform the action movement correctly.

Content-aware display adaptation and editing for stereoscopic images

We propose a content-aware stereoscopic image display adaptation method which simultaneously resizes a binocular image to the target resolution and adapts its depth to the comfort zone of the display while preserving the perceived shapes of prominent objects [Chang et al. 2011]. This method does not require depth information or dense correspondences. Given the specification of the target display and a sparse set of correspondences, our method efficiently deforms the input stereoscopic images for display adaptation by solving a least-squares energy minimization problem. This can be used to adjust stereoscopic images to fit displays with different real estates, aspect ratios, and comfort zones. In addition, with slight modifications to the energy function, our method allows users to interactively adjust the sizes, locations, and depths of the selected objects, giving users aesthetic control for depth perception.