Sanjit K. Mitra

Multisensor image fusion using the wavelet transform

Abstract The goal of image fusion is to integrate complementary information from multisensor data such that the new images are more suitable for the purpose of human visual perception and computer-processing tasks such as segmentation, feature extraction, and object recognition. This paper presents an image fusion scheme which is based on the wavelet transform. The wavelet transforms of the input images are appropriately combined, and the new image is obtained by taking the inverse wavelet transform of the fused wavelet coefficients. An area-based maximum selection rule and a consistency verification step are used for feature selection. The proposed scheme performs better than the Laplacian pyramid-based methods due to the compactness, directional selectivity, and orthogonality of the wavelet transform. A performance measure using specially generated test images is suggested and is used in the evaluation of different fusion methods, and in comparing the merits of different wavelet transform kernels. Extensive experimental results including the fusion of multifocus images, Landsat and Spot images, Landsat and Seasat SAR images, IR and visible images, and MRI and PET images are presented in the paper.

A new efficient approach for the removal of impulse noise from highly corrupted images

A new framework for removing impulse noise from images is presented in which the nature of the filtering operation is conditioned on a state variable defined as the output of a classifier that operates on the differences between the input pixel and the remaining rank-ordered pixels in a sliding window. As part of this framework, several algorithms are examined, each of which is applicable to fixed and random-valued impulse noise models. First, a simple two-state approach is described in which the algorithm switches between the output of an identity filter and a rank-ordered mean (ROM) filter. The technique achieves an excellent tradeoff between noise suppression and detail preservation with little increase in computational complexity over the simple median filter. For a small additional cost in memory, this simple strategy is easily generalized into a multistate approach using weighted combinations of the identity and ROM filter in which the weighting coefficients can be optimized using image training data. Extensive simulations indicate that these methods perform significantly better in terms of noise suppression and detail preservation than a number of existing nonlinear techniques with as much as 40% impulse noise corruption. Moreover, the method can effectively restore images corrupted with Gaussian noise and mixed Gaussian and impulse noise. Finally, the method is shown to be extremely robust with respect to the training data and the percentage of impulse noise.

A contour-based approach to multisensor image registration

Image registration is concerned with the establishment of correspondence between images of the same scene. One challenging problem in this area is the registration of multispectral/multisensor images. In general, such images have different gray level characteristics, and simple techniques such as those based on area correlations cannot be applied directly. On the other hand, contours representing region boundaries are preserved in most cases. The authors present two contour-based methods which use region boundaries and other strong edges as matching primitives. The first contour matching algorithm is based on the chain-code correlation and other shape similarity criteria such as invariant moments. Closed contours and the salient segments along the open contours are matched separately. This method works well for image pairs in which the contour information is well preserved, such as the optical images from Landsat and Spot satellites. For the registration of the optical images with synthetic aperture radar (SAR) images, the authors propose an elastic contour matching scheme based on the active contour model. Using the contours from the optical image as the initial condition, accurate contour locations in the SAR image are obtained by applying the active contour model. Both contour matching methods are automatic and computationally quite efficient. Experimental results with various kinds of image data have verified the robustness of the algorithms, which have outperformed manual registration in terms of root mean square error at the control points.

Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard

This paper addresses the problem of encoder optimization in a macroblock-based multimode video compression system. An efficient solution is proposed in which, for a given image region, the optimum combination of macroblock modes and the associated mode parameters are jointly selected so as to minimize the overall distortion for a given bit-rate budget. Conditions for optimizing the encoder operation are derived within a rate-constrained product code framework using a Lagrangian formulation. The instantaneous rate of the encoder is controlled by a single Lagrange multiplier that makes the method amenable to mobile wireless networks with time-varying capacity. When rate and distortion dependencies are introduced between adjacent blocks (as is the case when the motion vectors are differentially encoded and/or overlapped block motion compensation is employed), the ensuing encoder complexity is surmounted using dynamic programming. Due to the generic nature of the algorithm, it can be successfully applied to the problem of encoder control in numerous video coding standards, including H.261, MPEG-1, and MPEG-2. Moreover, the strategy is especially relevant for very low bit rate coding over wireless communication channels where the low dimensionality of the images associated with these bit rates makes real-time implementation very feasible. Accordingly, in this paper, the method is successfully applied to the emerging H.263 video coding standard with excellent results at rates as low as 8.0 Kb per second. Direct comparisons with the H.263 test model, TMN5, demonstrate that gains in peak signal-to-noise ratios (PSNR) are achievable over a wide range of rates.

Interpolated finite impulse response filters

A new approach to implement computationally efficient finite impulse response (FIR) digital filters is presented. The filter structure is a cascade of two sections. The first section generates a sparse set of impulse response samples and the other section generates the remaining samples by using interpolation. The method can be used to implement most practical FIR filters with significant savings in the number of arithmetic operations. Typically 1/2 to 1/8 of the number of multipliers and adders of conventional FIR filters are required in the implementation. The saving is achieved both in the linear phase and the non-linear phase cases. In addition, the new implementation gives smaller coefficient sensitivities and better roundoff noise properties than conventional implementations.

Handbook for Digital Signal Processing

From the Publisher: Digital signal processing (DSP) revolutionized the electronics industry. Its flexibility, cost effectiveness, grammability, precision and broad range of applications - including applications in telecommunications, consumer electronics, radar, sonar, and more - have made it the methodology of choice over analog signal processing. Over the past two decades, advances in DSP technology have been so rapid and so massive that, until now, no single volume has offered comprehensive theoretical coverage of this fascinating field along with practical DSP applications. Handbook for Digital Signal Processing is the definitive source of detailed information on all important topics in modern digital signal processing. The only up-to-date handbook of its kind, it fills the needs of practicing engineers and designers of hardware, systems, and software. Written and edited by internationally known authorities on DSP, Handbook for Digital Signal Processing is supplemented with hundreds of informative tables and illustrations. For professional engineers, designers, and researchers in electronics and telecommunications, this book will be an indispensable reference, now and for years to come.

The digital all-pass filter: a versatile signal processing building block

The properties of digital all-pass filters are reviewed and a broad overview of the diversity of applications in digital filtering is provided. Starting with the definition and basic properties of a scalar all-pass function, a variety of structures satisfying the all-pass property are assembled, with emphasis placed on the concept of structural losslessness. Applications are then outlined in notch filtering, complementary filtering and filter banks, multirate filtering, spectrum and group-delay equalization, and Hilbert transformations. In all cases, the structural losslessness property induces very robust performance in the face of multiplier coefficient quantization. Finally, the state-space manifestations of the all-pass property are explored, and it is shown that many all-pass filter structures are devoid of limit cycle behavior and feature very low roundoff noise gain. >

A unified rate-distortion analysis framework for transform coding

In our previous work, we have developed a rate-distortion (R-D) modeling framework for H.263 video coding by introducing the new concepts of characteristic rate curves and rate curve decomposition. In this paper, we further show it is a unified R-D analysis framework for all typical image/video transform coding systems, such as embedded zero-tree wavelet (EZW), set partitioning in hierarchical trees (SPIHT) and JPEG image coding; MPEG-2, H.263, and MPEG-4 video coding. Based on this framework, a unified R-D estimation and control algorithm is proposed for all typical transform coding systems. We have also provided a theoretical justification for the unique properties of the characteristic rate curves. A linear rate regulation scheme is designed to further improve the estimation accuracy and robustness, as well as to reduce the computational complexity of the R-D estimation algorithm. Our extensive experimental results show that with the proposed algorithm, we can accurately estimate the R-D functions and robustly control the output bit rate or picture quality of the image/video encoder.

Optimum bit allocation and accurate rate control for video coding via /spl rho/-domain source modeling

We present a new framework for rate-distortion (R-D) analysis, where the coding rate R and distortion D are considered as functions of /spl rho/ which is the percentage of zeros among the quantized transform coefficients. Previously (see He, Z. et al., Int. Conf. Acoustics, Speech and Sig. Proc., 2001), we observed that, in transform coding of images and videos, the rate function R(/spl rho/) is approximately linear. Based on this linear rate model, a simple and unified rate control algorithm was proposed for all standard video coding systems, such as MPEG-2, H.263, and MPEG-4. We further develop a distortion model and an optimum bit allocation scheme in the /spl rho/ domain. This bit allocation scheme is applied to MPEG-4 video coding to allocate the available bits among different video objects. The bits target of each object is then achieved by our /spl rho/-domain rate control algorithm. When coupled with a macroblock classification scheme, the above bit allocation and rate control scheme can also be applied to other video coding systems, such as H.263, at the macroblock level. Our extensive experimental results show that the proposed algorithm controls the encoder bit rate very accurately and improves the video quality significantly (by up to 1.5 dB).

Low-delay rate control for DCT video coding via /spl rho/-domain source modeling

By introducing the new concepts of characteristic rate curves and rate curve decomposition, a generic source-modeling framework is developed for transform coding of videos. Based on this framework, the rate-quantization (R-Q) and distortion-quantization (D-Q) functions (collectively called R-D functions in this work) of the video encoder can be accurately estimated with very low computational complexity before quantization and coding. With the accurate estimation of the R-Q function, a frame-level rate control algorithm is proposed for DCT video coding. The proposed algorithm outperforms the TMN8 rate control algorithm by providing more accurate and robust rate regulation and better picture quality. Based on the estimated R-D functions, an encoder-based rate-shape-smoothing algorithm is proposed. With this smoothing algorithm, the output bit stream of the encoder has both a smoothed rate shape and a consistent picture quality, which are highly desirable in practical video coding and transmission.