Lidan Miao

A Maximum Entropy Approach to Unsupervised Mixed-Pixel Decomposition

Due to the wide existence of mixed pixels, the derivation of constituent components (endmembers) and their fractional proportions (abundances) at the subpixel scale has been given a lot of attention. The entire process is often referred to as mixed-pixel decomposition or spectral unmixing. Although various algorithms have been proposed to solve this problem, two potential issues still need to be further investigated. First, assuming the endmembers are known, the abundance estimation is commonly performed by employing a least-squares error criterion, which, however, makes the estimation sensitive to noise and outliers. Second, the mathematical intractability of the abundance non-negative constraint results in computationally expensive numerical approaches. In this paper, we propose an unsupervised decomposition method based on the classic maximum entropy principle, termed the gradient descent maximum entropy (GDME), aiming at robust and effective estimates. We address the importance of the maximum entropy principle for mixed-pixel decomposition from a geometric point of view and demonstrate that when the given data present strong noise or when the endmember signatures are close to each other, the proposed method has the potential of providing more accurate estimates than the popular least-squares methods (e.g., fully constrained least squares). We apply the proposed GDME to the subject of unmixing multispectral and hyperspectral data. The experimental results obtained from both simulated and real images show the effectiveness of the proposed method

Binary Tree-based Generic Demosaicking Algorithm for Multispectral Filter Arrays

In this paper, we extend the idea of using mosaicked color filter array (CFA) in color imaging, which has been widely adopted in the digital color camera industry, to the use of multispectral filter array (MSFA) in multispectral imaging. The filter array technique can help reduce the cost, achieve exact registration, and improve the robustness of the imaging system. However, the extension from CFA to MSFA is not straightforward. First, most CFAs only deal with a few bands (3 or 4) within the narrow visual spectral region, while the design of MSFA needs to handle the arrangement of multiple bands (more than 3) across a much wider spectral range. Second, most existing CFA demosaicking algorithms assume the fixed Bayer CFA and are confined to properties only existed in the color domain. Therefore, they cannot be directly applied to multispectral demosaicking. The main challenges faced in multispectral demosaicking is how to design a generic algorithm that can handle the more diversified MSFA patterns, and how to improve performance with a coarser spatial resolution and a less degree of spectral correlation. In this paper, we present a binary tree based generic demosaicking method. Two metrics are used to evaluate the generic algorithm, including the root mean-square error (RMSE) for reconstruction performance and the classification accuracy for target discrimination performance. Experimental results show that the demosaicked images present low RMSE (less than 7) and comparable classification performance as original images. These results support that MSFA technique can be applied to multispectral imaging with unique advantages

The design and evaluation of a generic method for generating mosaicked multispectral filter arrays

The technology of color filter arrays (CFA) has been widely used in the digital camera industry since it provides several advantages like low cost, exact registration, and strong robustness. The same motivations also drive the design of multispectral filter arrays (MSFA), in which more than three spectral bands are used. Although considerable research has been reported to optimally reconstruct the full-color image using various demosaicking algorithms, studies on the intrinsic properties of these filter arrays as well as the underlying design principles have been very limited. Given a set of representative spectral bands, the design of an MSFA involves two issues: the selection of tessellation mechanisms and the arrangement/layout of different spectral bands. We develop a generic MSFA generation method starting from a checkerboard pattern. We show, through case studies, that most of the CFAs currently used by the industry can be derived as special cases of MSFAs generated using the generic algorithm. The performance of different MSFAs are evaluated based on their intrinsic properties, namely, the spatial uniformity and the spectral consistency. We design two metrics, static coefficient and consistency coefficient, to measure these two parameters, respectively. The experimental results demonstrate that the generic algorithm can generate optimal or near-optimal MSFAs in both the rectangular and the hexagonal domains

A generic method for generating multispectral filter arrays

The technology of color filter arrays (CFA) has been widely used in the digital camera industry since it provides several advantages like low cost, exact registration, and strong robustness. The same motivations also drive the design of multi-spectral filter arrays (MSFA), in which more than three color bands are used (e.g. visible and infrared). Although considerable research has been reported to optimally reconstruct the full-color image using various interpolation algorithms, studies on the intrinsic properties of these filter arrays as well as the underlying design principles have been very limited. In this paper, we identify the properties a CFA should possess and extend the design philosophy to MSFA. Based on these discussions, we develop a generic MSFA generation method starting from a checkerboard pattern with both rectangular and hexagonal tessellations. By manipulating this pattern through a combination of decomposition and subsampling steps, we can generate MSFAs that satisfy all the design requirements. We show, through case studies, that most of the CFAs currently used by the industry can be derived as special cases. To evaluate the performance of MSFAs, we design a metric, referred as the static coefficient (SC), to measure the uniformity of MSFAs.

Unsupervised Learning with Mini Free Energy

In this paper, we present an unsupervised learning with mini free energy for early breast cancer detection. Although an early malignant tumor must be small in size, the abnormal cells reveal themselves physiologically by emitting spontaneously thermal radiation due to the rapid cell growth, the so-called angiogenesis effect. This forms the underlying principle of Thermal Infrared (TIR) imaging in breast cancer study. Thermal breast scanning has been employed for a number of years, which however is limited to a single infrared band. In this research, we deploy two satellite-grade dual-color (at middle wavelength IR (3 - 5&mgr;m) and long wavelength IR (8 - 12&mgr;m)) IR imaging cameras equipped with smart subpixel automatic target detection algorithms. According to physics, the radiation of high/low temperature bodies will shift toward a shorter/longer IR wavelength band. Thus, the measured vector data x per pixel can be used to invert the matrix-vector equation x=As pixel-by-pixel independently, known as a single pixel blind sources separation (BSS). We impose the universal constraint of equilibrium physics governing the blackbody Planck radiation distribution, i.e., the minimum Helmholtz free energy, H = E - ToS. To stabilize the solution of Lagrange constrained neural network (LCNN) proposed by Szu et al., we incorporate the second order approximation of free energy, which corresponds to the second order constraint in the method of multipliers. For the subpixel target, we assume the constant ground state energy Eo can be determined by those normal neighborhood tissue, and then the excited state can be computed by means of Taylor series expansion in terms of the pixel I/O data. We propose an adaptive method to determine the neighborhood to find the free energy locally. The proposed methods enhance both the sensitivity and the accuracy of traditional breast cancer diagnosis techniques. It can be used as a first line supplement to traditional mammography to reduce the unwanted X-rays during the chemotherapy recovery. More important, the single pixel BSS method renders information on the tumor stage and tumor degree during the recovery process, which is not available using the popular independent component analysis (ICA) techniques.

Unsupervised Decomposition of Mixed Pixels Using the Maximum Entropy Principle

Due to the wide existence of mixed pixels, the derivation of constituent components (endmembers) and their proportions (abundances) at subpixel scales has become an important research topic. In this paper, we propose a novel unsupervised decomposition method based on the classical maximum entropy principle, termed uMaxEnt. The algorithm integrates a global least square error-based endmember detection and a per-pixel maximum entropy learning to find the most possible proportions. We apply the proposed method to the subject of spectral unmixing. The experimental results obtained from both simulated and real hyper-spectral data demonstrate the effectiveness of the uMaxEnt method

A Constrained Non-Negative Matrix Factorization Approach to Unmix Highly Mixed Hyperspectral Data

This paper presents a blind source separation method to unmix highly mixed hyperspectral data, i.e., each pixel is a mixture of responses from multiple materials and no pure pixels are present in the image due to large sampling distance. The algorithm introduces a minimum volume constraint to the standard non-negative matrix factorization (NMF) formulation, referred to as the minimum volume constrained NMF (MVC-NMF). MVC-NMF explores two important facts: first, the spectral data are non-negative; second, the constituent materials occupy the vertices of a simplex, and the simplex volume determined by the actual materials is the minimum among all possible simplexes that circumscribe the data scatter space. The experimental results based on both synthetic mixtures and a real image scene demonstrate that the proposed method outperforms several state-of-the-art approaches.

A Generic Binary Tree-Based Progressive Demosaicking Method for Multispectral Filter Array

The technique of multispectral filter array (MSFA) is a multispectral extension of the widely deployed color filter array (CFA), which uses single chip sensors and subsequent interpolation strategies to produce full color images. However, multispectral demosaicking presents unique challenges to traditional CFA demosaicking algorithms which cannot be directly extended to deal with various filter arrays with different numbers of spectral bands or different spatial patterns within each band. In addition, the more spectral bands involved, the more sparse each band samples the image plane, and the less information we can utilize when performing the interpolation. In this paper, we study a generic MSFA demosaicking method, which follows a binary tree structure to determine the order according to which different spectral bands and different pixel locations within each band are progressively interpolated, making the edge information utilized more effectively. The proposed method is demonstrated to outperform three traditional interpolation techniques as well as three advanced CFA demosaicking methods using two performance measures, the root mean square error (RMSE) for reconstruction fidelity and the classification accuracy for target recognition performance.

Generic MSFA mosaicking and demosaicking for multispectral cameras

In this paper, we investigate the potential application of the multispectral filter array (MSFA) techniques in multispectral imaging for reasons like low cost, exact registration, and strong robustness. In both human and many animal visual systems, different types of photoreceptors are organized into mosaic patterns. This behavior has been emulated in the industry to develop the so-called color filter array (CFA) in the manufacture of digital color cameras. In this way, only one color component is measured at each pixel, and the sensed image is a mosaic of different color bands. We extend this idea to multispectral imaging by developing generic mosaicking and demosaicking algorithms. The binary tree-driven MSFA design process guarantees that the pixel distributions of different spectral bands are uniform and highly correlated. These spatial features facilitate the design of the generic demosaicking algorithm based on the same binary tree, which considers three interrelated issues: band selection, pixel selection and interpolation. We evaluate the reconstructed images from two aspects: better reconstruction and better target classification. The experimental results demonstrate that the mosaicking and demosaicking process preserves the image quality effectively, which further supports that the MSFA technique is a feasible solution for multispectral cameras.

A Blind Source Separation Perspective on Image Restoration

This paper re-investigates the physical image formation process leading to a new interpretation of the classic image restoration problem from a blind source separation (BSS) perspective. The observed distorted image is considered as a linear combination of a set of shifted version of the point spread function (PSF) with the weight coefficients determined by the actual image. The new interpretation brings two immediate benefits to the practice of image restoration. First, we can utilize the rich set of BSS methods to solve the blind image restoration problem. Second, the new formulation in terms of matrix product has the equivalent merit as the conventional matrix-vector notation in theoretical study of restoration algorithms. We develop a smoothness and block-decorrelation constrained nonnegative matrix factorization method (termed CNMF) to blindly recover both the PSF and the actual image. The experimental results compared to one of the state-of-the-art methods demonstrate the merit of the proposed approach.