Prakash P. Gajjar

New Learning Based Super-Resolution: Use of DWT and IGMRF Prior

In this paper, we propose a new learning-based approach for super-resolving an image captured at low spatial resolution. Given the low spatial resolution test image and a database consisting of low and high spatial resolution images, we obtain super-resolution for the test image. We first obtain an initial high-resolution (HR) estimate by learning the high-frequency details from the available database. A new discrete wavelet transform (DWT) based approach is proposed for learning that uses a set of low-resolution (LR) images and their corresponding HR versions. Since the super-resolution is an ill-posed problem, we obtain the final solution using a regularization framework. The LR image is modeled as the aliased and noisy version of the corresponding HR image, and the aliasing matrix entries are estimated using the test image and the initial HR estimate. The prior model for the super-resolved image is chosen as an Inhomogeneous Gaussian Markov random field (IGMRF) and the model parameters are estimated using the same initial HR estimate. A maximum a posteriori (MAP) estimation is used to arrive at the cost function which is minimized using a simple gradient descent approach. We demonstrate the effectiveness of the proposed approach by conducting the experiments on gray scale as well as on color images. The method is compared with the standard interpolation technique and also with existing learning-based approaches. The proposed approach can be used in applications such as wildlife sensor networks, remote surveillance where the memory, the transmission bandwidth, and the camera cost are the main constraints.

An Edge Preserving Multiresolution Fusion: Use of Contourlet Transform and MRF Prior

In this paper, we propose a new approach for multiresolution fusion using contourlet transform (CT). The method is based on modeling the low spatial resolution (LR) and high spectral resolution multispectral (MS) image as the degraded and noisy version of their high spatial resolution version. Since this is an ill-posed problem, it requires regularization in order to obtain the final solution. In this paper, we first obtain the initial estimate of the fused image from the available MS image and the panchromatic (Pan) image by using the CT domain learning. Since CT provides better directional edges, the initial estimate has better edge details. Using the initial estimate, we obtain the degradation that accounts for the aliasing between the LR MS image and fused image. Regularization is carried out by modeling the texture of the final fused image as a homogeneous Markov random field (MRF) prior, where the MRF parameter is estimated using the initial estimate. The use of MRF prior on the final fused image takes care of the spatial dependencies among the pixels. A simple gradient-based optimization technique is used to obtain the final fused image. Although we use homogeneous MRF, the proposed approach preserves the edges in the final fused image by retaining the edges from the initial estimate and by carrying out the optimization on nonedge pixels only. Therefore, the advantage of the proposed method lies in preserving the discontinuities without using the discontinuity preserving prior, thus avoiding the use of computationally taxing optimization techniques for regularization purposes. In addition, the proposed method causes minimum spectral distortion since it learns the texture using contourlet coefficients and does not use actual Pan image pixel intensities. We demonstrate the effectiveness of our approach by conducting the experiments using subsampled and nonsubsampled CT on different data sets captured using Ikonos-2, Quickbird, and Worldview-2 satellites.

Multiresolution image fusion using edge-preserving filters

Abstract. We propose two approaches of multiresolution image fusion using multistage guided filter and difference of Gaussians (DoGs). In a multiresolution image fusion problem, the given multispectral (MS) and panchromatic (Pan) images have high spectral and high spatial resolutions, respectively. One can obtain the fused image using these two images by injecting the missing high frequency details from the Pan image into the MS image. The quality of the final fused image will then depend on the method used for high frequency details extraction and also on the technique for injecting these details into the MS image. Specifically, we have chosen the guided filter and DoGs for detail extraction since these are more versatile in applications involving feature extraction, denoising, and so on. The detail extraction process in the fusion approach using a guided filter exploits the relationship between the Pan and MS images by utilizing one of them as a guidance image while extracting details from the other. The final fused image is obtained by adding the extracted high frequency details to the corresponding MS image. This way, the spatial distortion of the MS image is reduced by consistently combining the details obtained using both MS and Pan images. In the fusion method using DoGs, the high frequency details are extracted in the first and second levels by subtracting the blurred images of the original Pan. The extracted details at both DoGs are added to the MS image to obtain the final fused image. Advantages and disadvantages of each method are discussed and the comparison of the results is shown between the two. The results are also compared with the traditional and the state-of-the-art methods using the images captured using different satellites such as Quickbird, Ikonos-2, and Worldview-2. The quantitative assessment is evaluated using the conventional measures as well as using a relatively new index, i.e., quality with no reference which does not require a reference image. The results and measures clearly show that there is promising improvement in the quality of the fused image using the proposed approaches.

Multiresolution image fusion using compressive sensing and graph cuts

Multiresolution fusion refers to the enhancement of low spatial resolution (LR) of multispectral (MS) images to that of panchromatic (Pan) image without compromising on the spectral details. Many of the present day methods for multiresolution fusion require that the Pan and MS images are registered. In this paper we propose a new approach for multiresolution fusion which is based on the theory of compressive sensing and graph cuts. We first estimate a close approximation to the fused image by using the sparseness in the given Pan and MS images. Assuming that they have the same sparseness, the initial estimate of the fused image is obtained as the linear combination of the Pan blocks. The weights in the linear combination are estimated using the l1 minimization by making use of MS and the down sampled Pan image. The final solution is obtained by using a model based approach. The low resolution MS image is modeled as the degraded and noisy version of the fused image in which the degradation matrix entries are estimated by using the initial estimate and the MS image. Since the MS fusion is an ill-posed inverse problem, we use a regularization based approach to obtain the final solution. A truncated quadratic smoothness prior is used for the preservation of the discontinuities in the fused image. A suitable energy function is then formed which consists of data fitting term and the prior term and is minimized using a graph cuts based approach in order to obtain the fused image. The advantage of the proposed method is that it does not require the registration of Pan and MS data. The spectral characteristics are well preserved in the fused image since we are not directly operating on the Pan digital numbers. Effectiveness of the proposed method is illustrated by conducting experiments on synthetic as well as on real satellite images. Quantitative comparison of the proposed method in terms of Erreur Relative Globale Adimensionnelle de Synthase (ERGAS), Correlation Coefficient (CC), Relative Average Spectral Error (RASE) and Spectral Aangle Mapper (SAM) with the state of the art approaches indicate superiority of our approach.

Single Frame Super-Resolution: A New Learning Based Approach and Use of IGMRF Prior

In this paper, we propose a new learning based approach for super-resolving an image captured at low spatial resolution. Given the low spatial resolution test image and a training set consisting of low and high spatial resolution images, all captured using the same camera, we obtain super-resolution for the test image. We propose a new wavelet based learning technique that learns the high frequency details for the test image from the training set and thus obtain an initial high resolution estimate. Since super-resolution is an ill-posed problem we solve it using regularization framework. We model the low resolution image as the aliased and noisy version of the corresponding high resolution image and estimate the aliasing matrix using the test image and the initial high resolution (HR) estimate. The super-resolved image is modeled as an inhomogeneous Gaussian Markov Random Field (IGMRF) and the IGMRF prior model parameters are estimated using the initial HR estimate. Finally, the cost function formed is minimized using simple gradient descent approach. We demonstrate the effectiveness of the proposed approach by conducting experiments on gray scale as well as on color images. The method is compared with another existing learning-based approach which uses training set consisting of HR images only and employs autoregressive (AR) and wavelet priors. The advantage of the our approach when compared to motion-based methods is that there is no need of multiple observations and also registration. The proposed approach can be used in applications such as wildlife sensor network where memory, transmission bandwidth and camera cost are main constraints.

Multiresolution fusion using contourlet transform based edge learning

In this paper, we propose a new approach for multi-resolution fusion of remotely sensed images based on the contourlet transform based learning of high frequency edges. We obtain a high spatial resolution (HR) and high spectral resolution multi-spectral (MS) image using the available high spectral but low spatial resolution MS image and the Panchromatic (Pan) image. Since we need to predict the missing high resolution pixels in each of the MS images the problem is posed in a restoration framework and is solved using maximum a posteriori (MAP) approach. Towards this end, we first obtain an initial approximation to the HR fused image by learning the edges from the Pan image using the contourlet transform. A low resolution model is used for the MS image formation and the texture of the fused image is modeled as a homogeneous Markov random field (MRF) prior. We then optimize the cost function which is formed using the data fitting term and the prior term and obtain the fused image, in which the edges correspond to those in the initial HR approximation. The procedure is repeated for each of the MS images. The advantage of the proposed method lies in the use of simple gradient based optimization for regularization purposes while preserving the discontinuities. This in turn reduces the computational complexity since it avoids the use of computationally taxing optimization methods for discontinuity preservation. Also, the proposed method has minimum spectral distortion as we are not using the actual Pan digital numbers, instead learn the texture using contourlet coefficients. We demonstrate the effectiveness of our approach by conducting experiments on real satellite data captured by Quickbird satellite.

Pan-sharpening: Use of difference of Gaussians

In this paper, we propose a fast method for pan-sharpening based on difference of Gaussians (DoGs). The Panchromatic (Pan) and the multi-spectral (MS) images are used to obtain a pan-sharpened image having both high spectral and spatial resolutions. The method is based on two level DoG on the Pan image. First, the Pan image is convolved with Gaussian kernel to obtain a blurred version and the high frequency details are extracted as the first level DoGs by subtracting the blurred image from the original. In order to get the second level DoG, same steps are repeated on the blurred Pan image. The extracted details at both DoGs are added to MS image to obtain the final pan-sharpened image. Experiments have been conducted with different values of standard deviation of Gaussian blur with images captured from different satellite sensors such as Ikonos-2, Quickbird and Worlview-2. A relatively new quality measure with no reference (QNR) index along with the other traditional measures are evaluated to check the efficacy of the proposed algorithm. The subjective and the quantitative assessment show that the proposed technique performs better, fast and less complex when compared to recently proposed state of the art techniques.

A fast approach for edge preserving super-resolution

In this paper we propose a fast approach for edge preserving super-resolution (SR) based on learning of contourlet coefficients. Given a low resolution test image, we first obtain an initial HR estimate i.e., a close approximation to SR image by learning the contourlet coefficients from a training database consisting of low resolution (LR) and high resolution (HR) images. The final SR image is obtained by using a regularization framework in which both the SR and the LR images are modeled as separate homogeneous Markov Random Fields (MRFs). The LR image formation process is modeled as a decimated and noisy version of the SR image and the final cost function is minimized by using a gradient descent method. Novelty of our approach lies in preserving the edges in the final SR image while using a non edge preserving MRF prior. This is definitely advantageous since it avoids the use of discontinuity preserving prior and hence the computationally taxing optimization methods. The edges in the final SR correspond to those learned from the initial HR estimate. The use of MRF on the low resolution image imposes an additional constraint on the final solution and hence we expect a better solution. In addition, we use the initial HR image for estimating the decimation matrix entries as well as for learning the corresponding MRF parameter. We show the effectiveness of the proposed approach by conducting the experiments on images captured using a real camera.

Pan-sharpening based on Non-subsampled Contourlet Transform detail extraction

In this paper, we propose a new pan-sharpening method using Non-subsampled Contourlet Transform. The panchromatic (Pan) and multi-spectral (MS) images provided by many satellites have high spatial and high spectral resolutions, respectively. The pan-sharpened image which has high spatial and spectral resolutions is obtained by using these images. Since the NSCT is shift invariant and it has better directional decomposition capability compared to contourlet transform, we use it to extract high frequency information from the available Pan image. First, two level NSCT decomposition is performed on the Pan image which has high spatial resolution. The required high frequency details are obtained by using the coarser subband available after the two level NSCT decomposition of the Pan image. The coarser sub-band is subtracted from the original Pan image to obtain these details. These extracted details are then added to MS image such that the original spectral signature is preserved in the final fused image. The experiments have been conducted on images captured from different satellite sensors such as IKonos-2, Worlview-2 and Quickbird. The traditional quantitative measures along with quality with no reference (QNR) index are evaluated to check the potential of the proposed method. The proposed approach performs better compared to the recently proposed state of the art methods such as additive wavelet luminance proportional (AWLP) method and context based decision (CBD) method.

Zoom based super-resolution: a fast approach using particle swarm optimization

Given a set of images captured using different integer zoom settings of a camera, we propose a fast approach to obtain super-resolution (SR) for the least zoomed image at a resolution of the most zoomed image. We first obtain SR approximation to the superresolved image using a learning based approach that uses training database consisting of low-resolution (LR) and their high-resolution (HR) images. We model the LR observations as the aliased and noisy versions of their HR parts and estimate the decimation using the learned SR approximation and the available least zoomed observation. A discontinuity preserving Markov random field (MRF) is used as a prior and its parameters are estimated using the SR approximation. Finally Maximum a posteriori (MAP)-MRF formulation is used and the final cost function is optimized using particle swarm optimization (PSO) technique which is computationally efficient compared to simulated annealing. The proposed method can be used in multiresolution fusion for remotely sensed images where the available HR panchromatic image can be used to obtain HR multispectral images. Another interesting application is in immersive navigation for walk through application. Here one can change zoom setting without compromising on the spatial resolution.