Andrea Garzelli

Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis

This paper compares two general and formal solutions to the problem of fusion of multispectral images with high-resolution panchromatic observations. The former exploits the undecimated discrete wavelet transform, which is an octave bandpass representation achieved from a conventional discrete wavelet transform by omitting all decimators and upsampling the wavelet filter bank. The latter relies on the generalized Laplacian pyramid, which is another oversampled structure obtained by recursively subtracting from an image an expanded decimated lowpass version. Both the methods selectively perform spatial-frequencies spectrum substitution from an image to another. In both schemes, context dependency is exploited by thresholding the local correlation coefficient between the images to be merged, to avoid injection of spatial details that are not likely to occur in the target image. Unlike other multiscale fusion schemes, both the present decompositions are not critically subsampled, thus avoiding possible impairments in the fused images, due to missing cancellation of aliasing terms. Results are presented and discussed on SPOT data.

A global quality measurement of pan-sharpened multispectral imagery

This letter focuses on quality assessment of fusion of multispectral (MS) images with high-resolution panchromatic (Pan) observations. A new quality index suitable for MS imagery having four spectral bands is defined from the theory of hypercomplex numbers, or quaternions. Both spectral and radiometric distortion measurements are encapsulated in a unique measurement, simultaneously accounting for local mean bias, changes in contrast, and loss of correlation of individual bands, together with spectral distortion. Results are presented and discussed on very high-resolution QuickBird data, through comparisons between state-of-the-art and advanced MS+Pan merge algorithms.

Remote sensing image fusion using the curvelet transform

This paper presents an image fusion method suitable for pan-sharpening of multispectral (MS) bands, based on nonseparable multiresolution analysis (MRA). The low-resolution MS bands are resampled to the fine scale of the panchromatic (Pan) image and sharpened by injecting highpass directional details extracted from the high-resolution Pan image by means of the curvelet transform (CT). CT is a nonseparable MRA, whose basis functions are directional edges with progressively increasing resolution. The advantage of CT with respect to conventional separable MRA, either decimated or not, is twofold. Firstly, directional detail coefficients matching image edges may be preliminarily soft-thresholded to achieve a noise reduction that is better than that obtained in the separable wavelet domain. Secondly, modeling of the relationships between high-resolution detail coefficients of the MS bands and of the Pan image is more fitting, being accomplished in the directional multiresolution domain. Experiments are carried out on very-high-resolution MS+Pan images acquired by the QuickBird and Ikonos satellite systems. Fusion simulations on spatially degraded data, whose original MS bands are available for reference, show that the proposed curvelet-based fusion method performs slightly better than the state-of-the art. Fusion tests at the full scale reveal that an accurate and reliable Pan-sharpening, little affected by local inaccuracies even in the presence of complex and detailed urban landscapes, is achieved by the proposed method.

Multispectral and Panchromatic Data Fusion Assessment Without Reference

This paper introduces a novel approach for evaluating the quality of pansharpened multispectral (MS) imagery without resorting to reference originals. Hence, evaluations are feasible at the highest spatial resolution of the panchromatic (PAN) sensor. Wang and Bovik’s image quality index (QI) provides a statistical similarity measurement between two monochrome images. The QI values between any couple of MS bands are calculated before and after fusion and used to define a measurement of spectral distortion. Analogously, QI values between each MS band and the PAN image are calculated before and after fusion to yield a measurement of spatial distortion. The rationale is that such QI values should be unchanged after fusion, i.e., when the spectral information is translated from the coarse scale of the MS data to the fine scale of the PAN image. Experimental results, carried out on very high-resolution Ikonos data and simulated Pleiades data, demonstrate that the results provided by the proposed approach are consistent and in trend with analysis performed on spatially degraded data. However, the proposed method requires no reference originals and is therefore usable in all practical cases.

A Critical Comparison Among Pansharpening Algorithms

Pansharpening aims at fusing a multispectral and a panchromatic image, featuring the result of the processing with the spectral resolution of the former and the spatial resolution of the latter. In the last decades, many algorithms addressing this task have been presented in the literature. However, the lack of universally recognized evaluation criteria, available image data sets for benchmarking, and standardized implementations of the algorithms makes a thorough evaluation and comparison of the different pansharpening techniques difficult to achieve. In this paper, the authors attempt to fill this gap by providing a critical description and extensive comparisons of some of the main state-of-the-art pansharpening methods. In greater details, several pansharpening algorithms belonging to the component substitution or multiresolution analysis families are considered. Such techniques are evaluated through the two main protocols for the assessment of pansharpening results, i.e., based on the full- and reduced-resolution validations. Five data sets acquired by different satellites allow for a detailed comparison of the algorithms, characterization of their performances with respect to the different instruments, and consistency of the two validation procedures. In addition, the implementation of all the pansharpening techniques considered in this paper and the framework used for running the simulations, comprising the two validation procedures and the main assessment indexes, are collected in a MATLAB toolbox that is made available to the community.

MTF-tailored multiscale fusion of high-resolution MS and pan imagery

This work presents a multiresolution framework for merging a multispectral image having an arbitrary number of bands with a higher-resolution panchromatic observation. The fusion method relies on the generalized Laplacian pyramid (GLP), which is a multiscale, oversampled structure. The goal is to selectively perform injection of spatial frequencies from an image to another with the constraint of thoroughly retaining the spectral information of the coarser data. The novel idea is that a model of the modulation transfer functions (MTF) of the multispectral scanner is exploited to design the GLP reduction filter. Thus, the interband structure model (IBSM), which is calculated at the coarser scale, where both MS and PAN data are available, can be extended to the finer scale, without the drawback of the poor enhancement occurring when MTFs are assumed to be ideal filters. Experiments carried out on QuickBird data demonstrate that a superior spatial enhancement, besides the spectral quality typical of injection methods, is achieved by means of the MTF-adjusted fusion.

Optimal MMSE Pan Sharpening of Very High Resolution Multispectral Images

In this paper, we propose an optimum algorithm, in the minimum mean-square-error (mmse) sense, for panchromatic (Pan) sharpening of very high resolution multispectral (MS) images. The solution minimizes the squared error between the original MS image and the fusion result obtained by spatially enhancing a degraded version of the MS image through a degraded version, by the same scale factor, of the Pan image. The fusion result is also optimal at full scale under the assumption of invariance of the fusion parameters across spatial scales. The following two versions of the algorithm are presented: a local mmse (lmmse) solution and a fast implementation which globally optimizes the fusion parameters with a moderate performance loss with respect to the lmmse version. We show that the proposed method is computationally practical, even in the case of local optimization, and it outperforms the best state-of-the-art Pan-sharpening algorithms, as resulted from the IEEE Data Fusion Contest 2006, on true Ikonos and QuickBird data and on simulated Pleiades data.

A Theoretical Analysis of the Effects of Aliasing and Misregistration on Pansharpened Imagery

In this paper, the characteristics of multispectral (MS) and panchromatic (P) image fusion methods are investigated. Depending on the way spatial details are extracted from P, pansharpening methods can be broadly labeled into two main classes, corresponding to methods based on either component substitution (CS) or multiresolution analysis (MRA). Theoretical investigations and experimental results evidence that CS-based fusion is far less sensitive than MRA-based fusion to: 1) registration errors, i.e., spatial misalignments between MS and P images, possibly originated by cartographic projection and resampling of individual data sets; 2) aliasing occurring in MS bands and stemming from modulation transfer functions (MTF) of MS channels that are excessively broad for the sampling step. In order to assess the sensitiveness of methods, aliasing is simulated at degraded spatial scale by means of several MTF-shaped digital filters. Analogously, simulated misalignments, carried out at both full and degraded scale, evidence the quality-shift tradeoff of the two classes. MRA yields a slightly superior quality in the absence of aliasing/misalignments, but is more penalized than CS, whenever either aliasing or shifts between MS and P occur. Conversely, CS generally produces a slightly lower quality, but is intrinsically more aliasing/shift tolerant.

Hypercomplex Quality Assessment of Multi/Hyperspectral Images

This letter presents a novel image quality index which extends the Universal Image Quality Index for monochrome images to multispectral and hyperspectral images through hypercomplex numbers. The proposed index is based on the computation of the hypercomplex correlation coefficient between the reference and tested images, which jointly measures spectral and spatial distortions. Experimental results, both from true and simulated images, are presented on spaceborne and airborne visible/infrared images. The results prove accurate measurements of inter- and intraband distortions even when anomalous pixel values are concentrated on few bands.

Analysis of the Effects of Pansharpening in Change Detection on VHR Images

In this letter, we investigate the effects of pansharpening (PS) applied to multispectral (MS) multitemporal images in change-detection (CD) applications. Although CD maps computed from pansharpened data show an enhanced spatial resolution, they can suffer from errors due to artifacts induced by the fusion process. The rationale of our analysis consists in understanding to which extent such artifacts can affect spatially enhanced CD maps. To this end, a quantitative analysis is performed which is based on a novel strategy that exploits similarity measures to rank PS methods according to their impact on CD performance. Many multiresolution fusion algorithms are considered, and CD results obtained from original MS and from spatially enhanced data are compared.