Nai-Xiang Lian

Adaptive Filtering for Color Filter Array Demosaicking

Most digital still cameras acquire imagery with a color filter array (CFA), sampling only one color value for each pixel and interpolating the other two color values afterwards. The interpolation process is commonly known as demosaicking. In general, a good demosaicking method should preserve the high-frequency information of imagery as much as possible, since such information is essential for image visual quality. We discuss in this paper two key observations for preserving high-frequency information in CFA demosaicking: (1) the high frequencies are similar across three color components, and 2) the high frequencies along the horizontal and vertical axes are essential for image quality. Our frequency analysis of CFA samples indicates that filtering a CFA image can better preserve high frequencies than filtering each color component separately. This motivates us to design an efficient filter for estimating the luminance at green pixels of the CFA image and devise an adaptive filtering approach to estimating the luminance at red and blue pixels. Experimental results on simulated CFA images, as well as raw CFA data, verify that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio, at a notably lower computational cost.

Reversing Demosaicking and Compression in Color Filter Array Image Processing: Performance Analysis and Modeling

In the conventional processing chain of single-sensor digital still cameras (DSCs), the images are captured with color filter arrays (CFAs) and the CFA samples are demosaicked into a full color image before compression. To avoid additional data redundancy created by the demosaicking process, an alternative processing chain has been proposed to move the compression process before the demosaicking. Recent empirical studies have shown that the alternative chain can outperform the conventional one in terms of image quality at low compression ratios. To provide a theoretically sound basis for such conclusion, we propose analytical models for the reconstruction errors of the two processing chains. The models developed confirm the results of existing empirical studies and provide better understanding of DSC processing chains. The modeling also allows performance predictions for more advanced compression and demosaicking methods, thus providing important cues for future development in this area

Edge-preserving image denoising via optimal color space projection

Denoising of color images can be done on each color component independently. Recent work has shown that exploiting strong correlation between high-frequency content of different color components can improve the denoising performance. We show that for typical color images high correlation also means similarity, and propose to exploit this strong intercolor dependency using an optimal luminance/color-difference space projection. Experimental results confirm that performing denoising on the projected color components yields superior denoising performance, both in peak signal-to-noise ratio and visual quality sense, compared to that of existing solutions. We also develop a novel approach to estimate directly from the noisy image data the image and noise statistics, which are required to determine the optimal projection

Improved color filter array demosaicking by accurate luminance estimation

Luminance information plays an important role in dictating the quality of a color image, and some existing color filter array (CFA) demosaicking methods achieve superiority by first reconstructing a satisfactory luminance plane. It is however difficult to accurately estimate the luminance from CFA samples since their color spectra are generally aliased. Extending a state-of-the-art luminance-based demosaicking scheme, in this paper we propose an improved demosaicking method using an efficient filter to estimate the luminance at green pixels and employing an effective edge-adaptive interpolation scheme to obtain the luminance at red and blue pixels. Experimental results demonstrate that not only is the proposed method less complex, it also performs noticeably better, both visually and in terms of peak signal-to-noise ratio, comparing with several recent methods.

Color image denoising using wavelets and minimum cut analysis

Wavelet thresholding has proven to be an efficient edge-preserving denoising method for grayscale images, especially when it exploits the interscale correlations of wavelet coefficients. Intrascale correlations can further improve the denoising performance, but the gain for grayscale images is generally small. In this letter, we demonstrate that the gain can become substantial in color image denoising, especially for smooth image color-difference components. We then propose a new denoising method, based on the minimum cut algorithm, to exploit both the interscale and intrascale correlations of wavelet coefficients. The proposed method achieves up to 5-dB gain in peak signal-to-noise ratio for color-difference images and leads to fewer visual color artifacts.

Video denoising using vector estimation of wavelet coefficients

Wavelet-based image denoising can be extended to a video by applying it to each video frame independently. The denoising performance can be improved by exploiting inter-frame correlations, for example, using appropriate temporal filtering. However, fixed temporal filters might not perform sufficiently well due to their inability to cope with the variability of inter-frame correlations across the video. While many adaptive temporal filtering approaches for denoising in spatial domain have been proposed, they do not straightforwardly extend to wavelet-based denoising. We propose a vector extension of popular hidden Markov tree modeling that flexibly exploits the color and frame dependency of wavelet coefficients. Experimental results confirm that the vector estimator of wavelet coefficients yields denoising performance superior to that of existing solutions, both in CPSNR and visual quality sense

Image Denoising Using Optimal Color Space Projection

Denoising of color images can be improved by exploiting strong correlation between high-frequency content of different color component. We show that for typical color images high correlation also means similarity, and propose to exploit this property using an optimal luminance/color-difference space projection. Experimental results confirm that denoising in the proposed color space yields superior performance, both in PSNR and visual quality sense, compared to that of existing solutions

An Efficient and Effective Color Filter Array Demosaicking Method

To reduce the cost and size, most digital still cameras (DSCs) capture only one color value at each pixel, and the results - color filter array samples - are then interpolated by a demosaicking method to construct a full-color image. Many advanced demosaicking methods have been proposed recently. However, the high complexity of these methods could prevent them from being used in DSCs. In this paper we propose an efficient and effective demosaicking method, which substitutes high-frequency component of color values in the spatial rather than frequency domain. We also propose a simple ternary, anisotropic interpolation scheme to obtain an initial full-color image required in the spatial-domain high-frequency substitution. Experimental results show that the proposed method can outperform recent state-of-the-art methods in terms of both PSNR performance and perceptual results, at the same time reducing the computational cost substantially.

Error Inhomogeneity in Wavelet-Based Compression

Despite the popularity of wavelet-based image compression, its shortcoming of having error inhomogeneity (EI), namely the error that is different for even and odd pixel location, has not been previously analyzed and formally addressed. The difference can be substantial, up to 3.4-dB peak signal-to-noise ratio (PSNR) for some images and compression ratios. In this paper, we show that the EI is caused by asymmetrical filtering of quantization errors after the upsampling step in wavelet synthesis process. Nonuniformity and correlation of quantization errors can also contribute to this EI, albeit to a smaller degree. In addition to explaining the source of EI, the model we developed in this paper also allows predicting its amount for a given wavelet. Furthermore, we show how to redesign wavelet filters to reduce this EI at a cost of a small reduction in the overall PSNR performance. For applications that are sensitive to PSNR degradation, we also show how to design wavelet filters that can gradually tradeoff PSNR performance for reduced EI.

Error Inhomogeneity of Wavelet Image Compression

Despite the popularity of wavelet-based image compression, its error inhomogeneity-the error that is different for even and odd pixel locations, has not been previously analyzed and formally addressed. The difference on PSNR performance can be substantial, up to 3.4 dB for some images and compression ratios. In this paper, we show that the error inhomogeneity is caused by asymmetrical filtering of quantization errors after the upsampling step in wavelet synthesis process. We also develop a model that also allows predicting the amount of inhomogeneity for a given wavelet. Furthermore, we show how to redesign wavelet filters to reduce the error inhomogeneity.