Ryo Matsuoka

Flash/no-flash image integration using convex optimization

When high ISO sensitivity is used to acquire images of dark scenes, their detail textures are often deteriorated by sensor noise. On the other hand, using flash photography with artificial light, one can shorten the exposure time, and obtain a sharp image under the low ISO sensitivity. However, the use of flash light changes the color tone and often generates unnatural images due to a specific color temperature of the additional light. This paper presents a new efficient method for flash/no-flash image integration. In contrast to conventional integration methods assuming that the flash image has a sharp texture without any noise, our method can successfully remove noise. Specifically, our method separately handles regions within the reach of the flash light and other regions out of range of the flash light, because the two regions have much different characteristics. As for the former well-exposed regions, we transferred the detail of the flash image to no-flash image by optimization and component separation. As for the latter under-exposed regions, an optimization based joint bilateral filtering that uses information of a flash image is performed to remove noise. Experimental results show the effectiveness of our method compared to the conventional methods.

High dynamic range image acquisition using flash image

We propose a denoising technique using multiple exposure image integration. When acquiring a dark scene, the detail of the dark area is often deteriorated by sensor noise. For a high dynamic range image acquisition, denoising in dark areas is a critical issue, since the dark area is, in general, enhanced by a tone-mapping and the noise is made more visible when displaying it on an output devise. In our method, a flash image is utilized as well as no-flash multiple exposure images to further reduce the noise. Multiple exposure integration is performed in a wavelet domain, where noise removal is achieved by the wavelet-shrinkage for multiple exposures. Our method works well especially for noise in shadows. We show the validity of the proposed algorithm by simulating the method with some actual noisy images.

Weight optimization for multiple image integration

We propose a denoising technique using multiple image integration. When acquiring a dark scene, the detail of the dark area is often deteriorated by sensor noise. A simple image integration inherently has the capability of reducing random noises. In this paper we develop the denoising performance of the multiple image integration by optimizing weight maps. We determine the optimal weight by solving a convex optimization problem. Through some experimental results, we show the weight optimization significantly improves the de-noising performance.

Constrained design of FIR filters with sparse coefficients

We present an algorithm for the constrained design of FIR filters with sparse coefficients. In general, the filter design approach aims to minimize a filter order and maximize the filter performance. Although the FIR filter coefficients designed by the least squares method is optimal in the least squares sense, it is not necessarily optimal among the set of filters with the same number of multipliers, that is, less mean squared error can be achieved by a filter that has the same number of multipliers, but has longer impulse response with some zero-valued entries. Our method minimizes the number of nonzero entries in the impulse response together with the least squares error of its frequency response. In addition, we incorporate some constraints to the design and realize better performance than conventional constrained least squares design.

Design of FIR filters with decimated impulse responses

In this paper, we present a numerical algorithm for the design of FIR filters with sparse impulse responses. Our method minimizes the number of nonzero entries in the impulse response together with the least squares error of its frequency response. We show that the FIR filters with sparse coefficients can outperform a conventional least squares approach and the Parks-McCllelan method under the condition of the same number of multipliers.

Misaligned Image Integration With Local Linear Model

We present a new image integration technique for a flash and long-exposure image pair to capture a dark scene without incurring blurring or noisy artifacts. Most existing methods require well-aligned images for the integration, which is often a burdensome restriction in practical use. We address this issue by locally transferring the colors of the flash images using a small fraction of the corresponding pixels in the long-exposure images. We formulate the image integration as a convex optimization problem with the local linear model. The proposed method makes it possible to integrate the color of the long-exposure image with the detail of the flash image without causing any harmful effects to its contrast, where we do not need perfect alignment between the images by virtue of our new integration principle. We show that our method successfully outperforms the state of the art in the image integration and reference-based color transfer for challenging misaligned data sets.

Reflectance estimation and white balancing using multiple images.

We propose a novel method for reflectance estimation by using a flash/no-flash image pair. In our method, by using multiple images of the same scene taken under different lighting conditions, we estimate a reflectance component which does not depend on scene illumination, and a shading component caused by illumination lights. Moreover, we apply it to white balance correction by appropriately correcting the estimated shading components. The proposed method achieve better performance than conventional methods especially under colored illumination and mixture lighting conditions.

Reflectance estimation using multiple exposure images

We introduce a method for intrinsic image decomposition from multiple exposure images. Our method jointly enhance the dynamic range of the images by incoorpolating the multiple exposures and estimate the reflectance and shading components of a scene. Since the reflectance component does not depend the exposure of the images, we estimate a single reflectance component and several shading componets that corresponds to the inputs.

LOSSLESS/NEAR-LOSSLESS COLOR IMAGE CODING BY INVERSE DEMOSAICING

In this paper, we introduce a novel framework for lossless/near-lossless (LS/NLS) color image coding assisted by an inverse demosaicing. Conventional frameworks are typically based on prediction (and quantization for NLS coding) followed by entropy coding, such as the JPEG-LS for bit rate saving. The approach of this work is totally different from the conventional ones. Basically, color images are created by demosaicing Bayer-pattern color filter array (CFA) whose operator can be expressed as square matrices. By using the (pseudo) inverse matrix of a joint demosaicing and color-to-gray conversion, the proposed decoder can recover the color image from its corresponding gray image data which is losslessly transmitted by the proposed encoder. Thus, LS/NLS color image reconstruction can be achieved while saving a bit rate significantly. In addition, using the same framework of color image coding, LS/NLS CFA coding can be realized by a comparable bit rate with JPEG-LS.