Danping Liao

Manifold alignment based color transfer for multiview image stitching

In multiview image stitching, color transfer removes all color inconsistences between different views under different illumination conditions and camera settings to make the stitching more seamless or visually acceptable. This paper presents a manifold alignment method to perform color transfer by exploring manifold structures of partially overlapped source and target images. Manifold alignment projects a pair of source and target images into a common embedding space in which not only the local geometries of color distribution in the respective images are preserved, but also the corresponding pixels in overlapped area across two images are pairwise aligned. Under this new space, color transfer can be considered as a matching problem between different manifolds, i.e. the color of each target pixel is replaced by the color of a source pixel that is nearest to this target pixel in this new space. Compared with other techniques in the literature, the proposed method makes full use of both the correspondences in overlapped area and the intrinsic color structures in the whole stitching scene so that a favorable performance is achieved.

A Manifold Alignment Approach for Hyperspectral Image Visualization With Natural Color

The trichromatic visualization of hundreds of bands in a hyperspectral image (HSI) has been an active research topic. The visualized image shall convey as much information as possible from the original data and facilitate easy image interpretation. However, most existing methods display HSIs in false color, which contradicts with user experience and expectation. In this paper, we propose a new framework for visualizing an HSI with natural color by the fusion of an HSI and a high-resolution color image via manifold alignment. Manifold alignment projects several data sets to a shared embedding space where the matching points between them are pairwise aligned. The embedding space bridges the gap between the high-dimensional spectral space of the HSI and the RGB space of the color image, making it possible to transfer natural color and spatial information in the color image to the HSI. In this way, a visualized image with natural color distribution and fine spatial details can be generated. Another advantage of the proposed method is its flexible data setting for various scenarios. As our approach only needs to search a limited number of matching pixel pairs that present the same object, the HSI and the color image can be captured from the same or semantically similar sites. Moreover, the learned projection function from the hyperspectral data space to the RGB space can be directly applied to other HSIs acquired by the same sensor to achieve a quick overview. Our method is also able to visualize user-specified bands as natural color images, which is very helpful for users to scan bands of interest.

Noise reduction of hyperspectral imagery based on nonlocal tensor factorization

Noise reduction for hyperspectral imagery (HSI) is an indispensable step before further processes such as object detection and classification. In this paper, we propose a noise reduction method for HSI based on non-local strategy and tensor factorization. Based on the observation that natural images are always locally self-repetitive, we divide the whole HSI into small sub-blocks and cluster similar blocks into groups. Since similar blocks share the same underlying structure, the redundancy can be utilized to remove noise of the blocks jointly. We stack the similar blocks to construct a fourth-order tensor from each group. Noise is reduced by finding the lower dimensional approximation of each of the fourth-order tensors via Tucker factorization. The experimental results indicate that the proposed method has a good quality of restoring the true signal from the noisy observation.

Visualization of Hyperspectral Imaging Data Based on Manifold Alignment

Tristimulus display of the abundant information contained in a hyper spectral image is a challenging task. Previous visualization approaches focused on preserving as much information as possible in the reduced spectral space, but ended up with displaying hyper spectral images as false color images, which contradicts with human experience and expectation. This paper proposes a new framework to tackle this problem. It is based on the fusion of a hyper spectral image and a high-resolution color image via manifold alignment technique. Manifold learning is an important tool for dimension reduction. Manifold alignment projects a pair of two data sets into a common embedding space so that the pairs of corresponding points in these two data sets are pair wise aligned in this new space. Hyper spectral image and high-resolution color image have strong complementary properties due to the high spectral resolution in the former and the high spatial resolution in the latter. The embedding space produced by manifold alignment bridges a gap between the high dimensional spectral space of hyper spectral image and RGB space of color image, making it possible to transfer the natural color and spatial information of a high-resolution color image to a hyper spectral image to generate a visualized image with natural color distribution and finer details.

Semisupervised manifold learning for color transfer between multiview images

In multiview image stitching, the colors of images in a scene might vary when images are taken under different illumination or camera settings. A common way to produce a seamless stitched image is to transform the colors of a target image to match that of a source image. In this paper we present a color transfer method based on two premises: first, pixels in the generated image should have similar colors with their corresponding pixels in the source image. Second, pixels with similar colors should still have similar colors after color transfer. Our method can be considered as a semisupervised manifold learning approach, where the corresponding pixels of the input images serve as the labeled data. Our goal is to learn a final image which not only shares the same colors with the source image but also has the same image structure with the target image. While manifold learning methods aim to find an embedded space to represent the data with minimum structure loss, the proposed method further constrains the solution space using the labeled data. This paper introduces a parametric linear method and a nonparametric nonlinear method to tackle different types of color changes. Experimental results show the effectiveness of our methods both quantitatively and qualitatively.

Visualization of hyperspectral imagery based on manifold learning

Displaying the abundant information contained in a hyperspectral image is a challenging task. Previous visualization approach focused only on preserving the structure in the original images. They ended up with presenting pseudo-color images and stopped short of adjusting the color of the images to retrieve more desirable visual effects. In this paper, a new visualization algorithm is proposed. It can be modeled as a two stage approach. At the first stage, Laplacian Eigenmaps algorithm is applied to reduce the dimension of the hyperspectral image. In this way we obtain a three dimensional image with pseudo-color. At the second stage, we transfer the natural color of a panchromatic image to the image obtained by the first step via manifold alignment. Experimental results show that the visualized image not only retains the structure of the hyperspectral image but also possesses natural colors.

Spectral Image Visualization Using Generative Adversarial Networks

Spectral images captured by satellites and radio-telescopes are analyzed to obtain information about geological compositions distributions, distant asters as well as undersea terrain. Spectral images usually contain tens to hundreds of continuous narrow spectral bands and are widely used in various fields. But the vast majority of those image signals are beyond the visible range, which calls for special visualization technique. The visualizations of spectral images shall convey as much information as possible from the original signal and facilitate image interpretation. However, most of the existing visualizatio methods display spectral images in false colors, which contradict with humans experience and expectation. In this paper, we present a novel visualization generative adversarial network (GAN) to display spectral images in natural colors. To achieve our goal, we propose a loss function which consists of an adversarial loss and a structure loss. The adversarial loss pushes our solution to the natural image distribution using a discriminator network that is trained to differentiate between false-color images and natural-color images. We also use a cycle loss as the structure constraint to guarantee structure consistency. Experimental results show that our method is able to generate structure-preserved and natural-looking visualizations.

Bound analysis of natural gradient descent in stochastic optimization setting

Natural gradient descent is a metric aware optimization algorithm which utilizes an underlying Riemannian parameter space, and has successfully improved performance in statistical asymptotic and experimental point of view. In this paper, we investigate the bound property of natural gradient descent in stochastic optimization setting. The bound property is analyzed in both direct and indirect ways. Substituting natural gradient for vanilla gradient is considered as the direct analysis. In this way, we analyze the bound of natural gradient descent method by convergence analysis technique. Afterwards, the bound is analyzed in an indirect way by introducing mirror gradient according to its equivalence to natural gradient. Employing mirror gradient in bound analysis makes the procedure of parameter update more intuitive. We finally present experimental results to support our theoretical findings.