Keigo Hirakawa

Spatio-Spectral Color Filter Array Design for Optimal Image Recovery

In digital imaging applications, data are typically obtained via a spatial subsampling procedure implemented as a color filter array - a physical construction whereby only a single color value is measured at each pixel location. Owing to the growing ubiquity of color imaging and display devices, much recent work has focused on the implications of such arrays for subsequent digital processing, including in particular the canonical demosaicking task of reconstructing a full color image from spatially subsampled and incomplete color data acquired under a particular choice of array pattern. In contrast to the majority of the demosaicking literature, we consider here the problem of color filter array design and its implications for spatial reconstruction quality. We pose this problem formally as one of simultaneously maximizing the spectral radii of luminance and chrominance channels subject to perfect reconstruction, and - after proving sub-optimality of a wide class of existing array patterns - provide a constructive method for its solution that yields robust, new panchromatic designs implementable as subtractive colors. Empirical evaluations on multiple color image test sets support our theoretical results, and indicate the potential of these patterns to increase spatial resolution for fixed sensor size, and to contribute to improved reconstruction fidelity as well as significantly reduced hardware complexity.

Adaptive homogeneity-directed demosaicing algorithm

Most cost-effective digital camera uses a single image sensor, applying alternating patterns of red, green, and blue color filters to each pixel location. Demosaicing algorithm reconstructs a full three-color representation of color images from this sensor data. This paper identifies three inherent problems often associated with directional interpolation approach to demosaicing algorithms: misguidance color artifacts, interpolation color artifacts, and aliasing. The level of misguidance color artifacts present in two images can be compared using metric neighborhood modeling. The proposed demosaicing algorithm estimates missing pixels by interpolating in the direction with fewer color artifacts. The aliasing problem is addressed by applying filterbank techniques to directional interpolation. The interpolation artifacts are reduced using a nonlinear iterative procedure. Experimental results using digital images confirm the effectiveness of this approach.

Joint demosaicing and denoising

The output image of a digital camera is subject to a severe degradation due to noise in the image sensor. This paper proposes a novel technique to combine demosaicing and denoising procedures systematically into a single operation by exploiting their obvious similarities. We first design a filter as if we are optimally estimating a pixel value from a noisy single-color (sensor) image. With additional constraints, we show that the same filter coefficients are appropriate for color filter array interpolation (demosaicing) given noisy sensor data. The proposed technique can combine many existing denoising algorithms with the demosaicing operation. In this paper, a total least squares denoising method is used to demonstrate the concept. The algorithm is tested on color images with pseudorandom noise and on raw sensor data from a real CMOS digital camera that we calibrated. The experimental results confirm that the proposed method suppresses noise (CMOS/CCD image sensor noise model) while effectively interpolating the missing pixel components, demonstrating a significant improvement in image quality when compared to treating demosaicing and denoising problems independently

Color Constancy with Spatio-Spectral Statistics

We introduce an efficient maximum likelihood approach for one part of the color constancy problem: removing from an image the color cast caused by the spectral distribution of the dominating scene illuminant. We do this by developing a statistical model for the spatial distribution of colors in white balanced images (i.e., those that have no color cast), and then using this model to infer illumination parameters as those being most likely under our model. The key observation is that by applying spatial band-pass filters to color images one unveils color distributions that are unimodal, symmetric, and well represented by a simple parametric form. Once these distributions are fit to training data, they enable efficient maximum likelihood estimation of the dominant illuminant in a new image, and they can be combined with statistical prior information about the illuminant in a very natural manner. Experimental evaluation on standard data sets suggests that the approach performs well.

Image denoising using total least squares

In this paper, we present a method for removing noise from digital images corrupted with additive, multiplicative, and mixed noise. An image patch from an ideal image is modeled as a linear combination of image patches from the noisy image. We propose to fit this model to the real-world image data in the total least square (TLS) sense, because the TLS formulation allows us to take into account the uncertainties in the measured data. We develop a method to reduce the contribution from the irrelevant image patches, which will sharpen the edges and reduce edge artifacts at the same time. Although the proposed algorithm is computationally demanding, the image quality of the output image demonstrates the effectiveness of the TLS algorithms

A Framework for wavelet-Based Analysis and Processing of Color Filter Array Images with Applications to Denoising and Demosaicing

This paper presents a new approach to demosaicing of spatially sampled image data observed through a color filter array, in which properties of Smith-Barnwell filterbanks are employed to exploit the correlation of color components in order to reconstruct a subsampled image. The method is shown to be amenable to wavelet-domain denoising prior to demosaicing, and a general framework for applying existing image denoising algorithms to color filter array data is also described. Results indicate that the proposed method performs on a par with the state of the art for far lower computational cost, and provides a versatile, effective, and low-complexity solution to the problem of interpolating color filter array data observed in noise.

Color constancy beyond bags of pixels

Estimating the color of a scene illuminant often plays a central role in computational color constancy. While this problem has received significant attention, the methods that exist do not maximally leverage spatial dependencies between pixels. Indeed, most methods treat the observed color (or its spatial derivative) at each pixel independently of its neighbors. We propose an alternative approach to illuminant estimation-one that employs an explicit statistical model to capture the spatial dependencies between pixels induced by the surfaces they observe. The parameters of this model are estimated from a training set of natural images captured under canonical illumination, and for a new image, an appropriate transform is found such that the corrected image best fits our model.

Spatio-Spectral Color Filter Array Design for Enhanced Image Fidelity

In digital imaging applications, data are typically obtained via a spatial subsampling procedure implemented as a color filter array - a physical construction whereby only a single color representative is measured at each pixel location. Owing to the growing ubiquity of color imaging and display devices, much recent work has focused on the interplay between color filter array design and subsequent digital processing, including in particular the canonical spatio-chromatic reconstruction task known as demosaicking. Here we consider the problem of improved color filter array design, leading to enhanced image fidelity. We first analyze the limitations of the well-known Bayer pattern, currently most popular in industry. We then propose a framework for designing rectangular color filter arrays amenable to efficient and completely linear reconstruction, and provide examples of new patterns that enable improvements in reconstruction quality.

Skellam Shrinkage: Wavelet-Based Intensity Estimation for Inhomogeneous Poisson Data

The ubiquity of integrating detectors in imaging and other applications implies that a variety of real-world data are well modeled as Poisson random variables whose means are in turn proportional to an underlying vector-valued signal of interest. In this article, we first show how the so-called Skellam distribution arises from the fact that Haar wavelet and filterbank transform coefficients corresponding to measurements of this type are distributed as sums and differences of Poisson counts. We then provide two main theorems on Skellam shrinkage, one showing the near-optimality of shrinkage in the Bayesian setting and the other providing for unbiased risk estimation in a frequentist context. These results serve to yield new estimators in the Haar transform domain, including an unbiased risk estimate for shrinkage of Haar-Fisz variance-stabilized data, along with accompanying low-complexity algorithms for inference. We conclude with a simulation study demonstrating the efficacy of our Skellam shrinkage estimators both for the standard univariate wavelet test functions as well as a variety of test images taken from the image processing literature, confirming that they offer some performance improvements over existing alternatives.

Image denoising for signal-dependent noise

In this paper, we present a method for removing noise from digital images corrupted with additive, multiplicative, and mixed noise. An image patch from an ideal image is modeled as a linear combination of image patches from the noisy image. We propose to fit this image model to the real-world image data in the total least square (TLS) sense, because the TLS formulation allows us to take into account the uncertainties in the measured data. We develop a method to reduce the contribution from the irrelevant image patches, which will sharpen the edges and reduce edge artifacts at the same time. Although the proposed algorithm is computationally demanding, the image quality of the output image demonstrates the effectiveness of the TLS algorithms.