Moon Gi Kang

Super-resolution image reconstruction

The spatial resolution that represents the number of pixels per unit area in an image is the principal factor in determining the quality of an image. With the development of image processing applications, there is a big demand for high-resolution (HR) images since HR images not only give the viewer a pleasing picture but also offer additional detail that is important for the analysis in many applications. The current technology to obtain HR images mainly depends on sensor manufacturing technology that attempts to increase the number of pixels per unit area by reducing the pixel size. However, the cost for high-precision optics and sensors may be inappropriate for general purpose commercial applications, and there is a limitation to pixel size reduction due to shot noise encountered in the sensor itself. Therefore, a resolution enhancement approach using signal processing techniques has been a great concern in many areas, and it is called super-resolution (SR) (or HR) image reconstruction or simply resolution enhancement in the literature. In this issue, we use the term “SR image reconstruction” to refer to a signal processing approach toward resolution enhancement, because the term “super” very well represents the characteristics of the technique overcoming the inherent resolution limitation of low-resolution (LR) imaging systems. The term SR was originally used in optics, and it refers to the algorithms that mainly operate on a single image to extrapolate the spectrum of an object beyond the diffraction limit (SR restoration). These two SR concepts (SR reconstruction and SR restoration) have a common focus in the aspect of recovering high-frequency information that is lost or degraded during the image acquisition. However, the cause of the loss of high-frequency information differs between these two concepts. SR restoration in optics attempts to recover information beyond the diffraction cutoff frequency, while the SR reconstruction method in engineering tries to recover high-frequency components corrupted by aliasing. We hope that readers do not confuse the super resolution in this issue with the term super resolution used in optics. SR image reconstruction algorithms investigate the relative motion information between multiple LR images (or a video sequence) and increase the spatial resolution by fusing them into a single frame. In doing so, it also removes the effect of possible blurring and noise in the LR images. In summary, the SR image reconstruction method estimates an HR image with finer spectral details from multiple LR observations degraded by blur, noise, and aliasing. The major advantage of this approach is that it may cost less and the existing LR imaging systems can still be utilized. Considering the maturity of this field and its various prospective applications, it seems timely and appropriate to discuss and adjust the topic of SR in the special issue of the magazine, since we do not have enough materials for ready disposal. This special section contains five articles covering various aspects of SR techniques. The first article, “Super-Resolution Image Reconstruction: A Technical Overview” by Sungcheol Park, Minkyu Park, and Moon Gi Kang, provides an introduction to the concepts and definitions of the SR image reconstruction as well as an overview of various existing SR algorithms. Advanced issues that are currently under investigation in this area are also discussed. The second article, “High-Resolution Images from Low-Resolution Compressed Video,” by Andrew C. Segall, Rafael Molina, and Aggelos K. Katsaggelos, considers the SR techniques for compressed video. Since images are routinely compressed prior to transmission and storage in current acquisition systems, it is important to take into account the characteristics of compression systems in developing the SR techniques. In this article, they survey models for the compression system and develop SR techniques within the Bayesian framework. The third article, by Deepu Rajan, Subhasis Chaudhuri, and Manjunath V. Joshi, titled “Multi-Objective Super-Resolution Technique: Concept and Examples,”

General choice of the regularization functional in regularized image restoration

The determination of the regularization parameter is an important issue in regularized image restoration, since it controls the trade-off between fidelity to the data and smoothness of the solution. A number of approaches have been developed in determining this parameter. In this paper, a new paradigm is adopted, according to which the required prior information is extracted from the available data at the previous iteration step, i.e., the partially restored image at each step. We propose the use of a regularization functional instead of a constant regularization parameter. The properties such a regularization functional should satisfy are investigated, and two specific forms of it are proposed. An iterative algorithm is proposed for obtaining a restored image. The regularization functional is defined in terms of the restored image at each iteration step, therefore allowing for the simultaneous determination of its value and the restoration of the degraded image. Both proposed iteration adaptive regularization functionals are shown to result in a smoothing functional with a global minimum, so that its iterative optimization does not depend on the initial conditions. The convergence of the algorithm is established and experimental results are shown.

Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration

In this paper, we propose a high-resolution image reconstruction algorithm considering inaccurate subpixel registration. A regularized iterative reconstruction algorithm is adopted to overcome the ill-posedness problem resulting from inaccurate subpixel registration. In particular, we use multichannel image reconstruction algorithms suitable for applications with multiframe environments. Since the registration error in each low-resolution image has a different pattern, the regularization parameters are determined adaptively for each channel. We propose two methods for estimating the regularization parameter automatically. The proposed algorithms are robust against registration error noise, and they do not require any prior information about the original image or the registration error process. Information needed to determine the regularization parameter and to reconstruct the image is updated at each iteration step based on the available partially reconstructed image. Experimental results indicate that the proposed algorithms outperform conventional approaches in terms of both objective measurements and visual evaluation.

New edge dependent deinterlacing algorithm based on horizontal edge pattern

In this paper, we propose a new deinterlacing algorithm, which is an edge dependent interpolation (EDI) algorithm based on a horizontal edge pattern. Generally, a conventional EDI algorithm has a visually better performance than any other deinterlacing algorithms using one field. However, it produces unpleasant results due to the failure of estimating edge direction. In order to exactly detect edge direction, we use not only simple difference but also edge patterns. Experimental results indicate that the proposed algorithm outperforms conventional approaches with respect to both objective and subjective criteria.

Discrete cosine transform based regularized high-resolution image reconstruction algorithm

While high-resolution images are required for various applica- tions, aliased low-resolution images are only available due to the physi- cal limitations of sensors. We propose an algorithm to reconstruct a high- resolution image from multiple aliased low-resolution images, which is based on the generalized deconvolution technique. The conventional approaches are based on the discrete Fourier transform (DFT) since the aliasing effect is easily analyzed in the frequency domain. However, the useful solution may not be available in many cases, i.e., the underdeter- mined cases or the insufficient subpixel information cases. To compen- sate for such ill-posedness, the generalized regularization is adopted in the spatial domain. Furthermore, the usage of the discrete cosine trans- form (DCT) instead of the DFT leads to a computationally efficient recon- struction algorithm. The validity of the proposed algorithm is both theo- retically and experimentally demonstrated. It is also shown that the artifact caused by inaccurate motion information is reduced by regular- ization.

An iterative weighted regularized algorithm for improving the resolution of video sequences

This paper introduces an iterative regularized approach to increase the resolution of a video sequence. A multiple input smoothing convex functional is defined and used to obtain a globally optimal high resolution video sequence. A mathematical model of multiple inputs is described by using the point spread function between the original and bilinearly interpolated images in the spatial domain, and motion estimation between frames in the temporal domain. An iterative algorithm is utilized for obtaining the solution. The regularization parameter is updated at each iteration step from the partially restored video sequence. Experimental results demonstrate the capability of the proposed approach.

Simultaneous iterative image restoration and evaluation of the regularization parameter

A nonlinear regularized iterative image restoration algorithm is proposed, according to which only the noise variance is assumed to be known in advance. The algorithm results from a set theoretic regularization approach, where a bound of the stabilizing functional, and therefore the regularization parameter, are updated at each iteration step. Sufficient conditions for the convergence of the algorithm are derived and experimental results are shown. >

Design of real-time image enhancement preprocessor for CMOS image sensor

This paper presents a design of the real-time digital image enhancement preprocessor for a CMOS image sensor. The CMOS image sensor offers various advantages while it provides lower-quality images than the CCD does. In order to compensate for the physical limitation of the CMOS sensor, a spatially adaptive contrast enhancement algorithm was incorporated into the preprocessor with color interpolation, gamma correction, and automatic exposure control. The efficient hardware architecture for the preprocessor is proposed and was simulated in VHDL. It is composed of about 19 K logic gates, which is suitable for a low-cost one-chip PC camera. The test system was implemented on a FPGA chip in real-time mode, and performed successfully.

Simultaneous multichannel image restoration and estimation of the regularization parameters

In this correspondence, a constrained least-squares multichannel image restoration approach is proposed, in which no prior knowledge of the noise variance at each channel or the degree of smoothness of the original image is required. The regularization functional for each channel is determined by incorporating both within-channel and cross-channel information. It is shown that the proposed smoothing functional has a global minimizer.

Spatially adaptive multi-resolution multispectral image fusion

Due to the performance limit of remote sensing systems, multispectral images have limited spatial resolution. Their spatial resolution can be improved by merging them with higher resolution image data. A fundamental problem frequently occurring in existing fusion processes, however, is the distortion of spectral information. This paper presents a spatially adaptive image fusion algorithm which produces visually natural images and retains the quality of local spectral information as well. High frequency information of the high resolution image to be inserted to the resampled multispectral images is controlled by adaptive gains to incorporate the difference of local spectral characteristics between the high and the low resolution images into the fusion. Each gain is estimated to minimize the l 2-norm of the error between the original and the estimated pixel values defined in a spatially adaptive window of which the weights are proportional to the spectral correlation measurements of the corresponding regions. This method is applied to a set of co-registered Landsat 7 Enhanced Thematic Mapper (ETM)+ panchromatic and multispectral image data. The experimental results show that high resolution images can be synthesized by the proposed method, which successfully preserves spectral content of the multispectral images.