Frosti Palsson

A New Pansharpening Algorithm Based on Total Variation

In this letter, we present a new method for the pansharpening of multispectral satellite imagery. Pansharpening is the process of synthesizing a high spatial resolution multispectral image from a low spatial resolution multispectral image and a high-resolution panchromatic (PAN) image. The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model. This model is based on the assumptions that a linear combination of the bands of the pansharpened image gives the PAN image and that a decimation of the pansharpened image gives the original multispectral image. Experimental results are based on two real datasets and the quantitative quality of the pansharpened images is evaluated using a number of spatial and spectral metrics, some of which have been recently proposed and do not need a reference image. The proposed method compares favorably to other well-known methods for pansharpening and produces images of excellent spatial and spectral quality.

Model-Based Fusion of Multi- and Hyperspectral Images Using PCA and Wavelets

In remote sensing, due to cost and complexity issues, multispectral (MS) and hyperspectral (HS) sensors have significantly lower spatial resolution than panchromatic (PAN) images. Recently, the problem of fusing coregistered MS and HS images has gained some attention. In this paper, we propose a novel method for fusion of MS/HS and PAN images and of MS and HS images. MS and, more so, HS images contain spectral redundancy, which makes the dimensionality reduction of the data via principal component (PC) analysis very effective. The fusion is performed in the lower dimensional PC subspace; thus, we only need to estimate the first few PCs, instead of every spectral reflectance band, and without compromising the spectral and spatial quality. The benefits of the approach are substantially lower computational requirements and very high tolerance to noise in the observed data. Examples are presented using WorldView 2 data and a simulated data set based on a real HS image, with and without added noise.

Classification of Pansharpened Urban Satellite Images

The classification of high resolution urban remote sensing imagery is addressed with the focus on classification of imagery that has been pansharpened by a number of different pansharpening methods. The pansharpening process introduces some spectral and spatial distortions in the resulting fused multispectral image, the amount of which highly varies depending on which pansharpening technique is used. In the majority of the pansharpening techniques that have been proposed, there is a compromise between the spatial enhancement and the spectral consistency. Here we study the effects of the spectral and spatial distortions on the accuracy in classification of pansharpened imagery. We also study the performance in terms of accuracy of the various pansharpening techniques during classification with spatial information, obtained using mathematical morphology (MM). MM is used to derive local spatial information from the panchromatic data. Random Forests (RF) and Support Vector Machines (SVM) will be used as classifiers. Experiments are done for three different datasets that have been obtained by two different imaging sensors, IKONOS and QuickBird. These sensors deliver multispectral images that have four bands, R, G, B and near infrared (NIR). To further study the contribution of the NIR band, experiments are done using both the RGB bands and all four bands, respectively.

Quantitative Quality Evaluation of Pansharpened Imagery: Consistency Versus Synthesis

Pansharpening is the process of fusing a high-resolution panchromatic image and a low-spatial-resolution multispectral image to yield a high-spatial-resolution multispectral image. This is a typical ill-posed inverse problem, and in the past two decades, many methods have been proposed to solve it. Still, there is no general consensus on the best way to quantitatively evaluate the spectral and spatial quality of the fused image. In this paper, we compare the two most widely used and accepted methods for quality evaluation. The first method is the verification of the synthesis property which states that the fused image should be as identical as possible to the multispectral image that the sensor would observe at a higher resolution. This is impossible to verify unless the observed images are spatially degraded so that the original observed multispectral image can be used as reference. The second method is to use metrics that do not use a reference, such as the quality no reference (QNR) metrics. However, there is another property, i.e., the consistency property, which states that the fused image reduced to the resolution of the original multispectral image should be as identical to the original image as possible. This has generally been considered a necessary condition that does not have to imply correct fusion. Using real WorldView-2 and QuickBird data and a total of 18 component substitution and multiresolution analysis methods, we demonstrate that the consistency property can indeed be used to give reliable assessment of the relative performance of pansharpening methods and is superior to using the QNR metrics.

Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network

In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when compared with conventional methods. This is especially true when the HS image is corrupted by additive noise.

SAR image denoising using total variation based regularization with sure-based optimization of the regularization parameter

Images obtained using Synthetic Aperture Radar (SAR) are corrupted by speckle. Speckle noise results from the chaotic interference of backscattered electromagnetic waves and makes the analysis, interpretation and classification of SAR images difficult. In this paper, we present a denoising algorithm based on Total Variation (TV) regularization. While this kind of denoising algorithm is not new, we propose to select the regularization parameter by minimizing the estimate of the mean square error (MSE) between the denoised image and the clean image. We do not have to know the clean image because we use a statistically unbiased MSE estimate - Steins Unbiased Risk Estimate (SURE), that depends on the observed image and the estimated image. However, since it is difficult to derive SURE analytically for this kind of problem, we estimate SURE using stochastic methods. We present results using both a simulated image and real SAR image.

MTF-Based Deblurring Using a Wiener Filter for CS and MRA Pansharpening Methods

Pansharpening is the fusion of low-resolution multispectral (MS) images and high-resolution panchromatic (PAN) images to yield a high-resolution MS image. The component substitution (CS) and multiresolution analysis (MRA) methods are usually computationally efficient, making them able to handle large datasets. However, these methods often produce images that suffer from spectral and spatial distortions. The CS and MRA methods can be described using general injection schemes where details extracted from the PAN image, modulated by a band-dependent gain constant, are added to the MS image, which has been interpolated to the size of the PAN image. In this paper, we propose a simple modification of these schemes where the interpolated MS image is replaced by its deblurred version, where the deblurring kernel is matched to the modulation transfer function (MTF) of the MS sensor. This can significantly enhance the quality of the fused image. Using two real datasets and one simulated dataset, our experimental results show that using the proposed preprocessing method can significantly increase both the spectral and spatial quality of the fused image according to quantitative quality metrics.

Hyperspectral image denoising using a sparse low rank model and dual-tree complex wavelet transform

Hyperspectral images (HSI) are often corrupted by noise making their analysis and interpretation difficult. In this paper we develop a sparse low rank model for HSI, which is useful for denoising. The two key benefits of the model for denoising are dimensionality reduction via noisy principal component analysis (nPCA) and the exploitation of sparse-ness in the dual-tree complex wavelet transform (CWT) coefficients of the loading matrix associated with the principal components (PCs). We present denoising examples of both synthetic and real data and compare our method to a PCA based 2-dimensional (2D) bivariate shrinkage method.

Pansharpening via sparsity optimization using overcomplete transforms

In this paper we consider pansharpening of multispectral satellite imagery based on solving an under-determined inverse problem regularized by the ℓ1-norm of the coefficients of overcomplete multi-scale transforms which all are tight-frame systems. There are two main approaches in sparsity promoting ℓ1-norm regularization, the analysis and the synthesis approach. We perform a number of experiments using two real and well known datasets where the focus is the comparison of the two approaches. One dataset includes a high resolution reference image while the other needs to be degraded prior to pansharpening in order to use the original multispectral image as the reference. Experiments are performed for a range of values for the regularization parameter, where each resulting pansharpened image is evaluated using three quality metrics. The behavior of those metrics as a function of the regularization parameter is compared for the analysis and synthesis formulations and it is shown that analysis gives better results.

A new pansharpening method using an explicit image formation model regularized via Total Variation

In this paper we present a new method for the pansharpening of multi-spectral satellite imagery. This method is based on a simple explicit image formation model which leads to an ill posed problem that needs to be regularized for best results. We use both Tikhonov (ridge regression) and Total Variation (TV) regularization. We develop the solutions to these two problems and then we address the problem of selecting the optimal regularization parameter λ. We find the value of λ that minimizes Steins unbiased risk estimate (SURE). For ridge regression this leads to an analytical expression for SURE while for the TV regularized solution we use Monte Carlo SURE where the estimate is obtained by stochastic means. Finally, we present experiment results where we use quality metrics to evaluate the spectral and spatial quality of the resulting pansharpened image.