Hideto Motomura

Color image reproduction based on multispectral and multiprimary imaging: experimental evaluation

Multispectral imaging is significant technology for the acquisition and display of accurate color information. Natural color reproduction under arbitrary illumination becomes possible using spectral information of both image and illumination light. In addition, multiprimary color display, i.e., using more than three primary colors, has been also developed for the reproduction of expanded color gamut, and for discounting observer metamerism. In this paper, we present the concept for the multispectral data interchange for natural color reproduction, and the experimental results using 16-band multispectral camera and 6-primary color display. In the experiment, the accuracy of color reproduction is evaluated in CIE (Delta) Ea*b* for both image capture and display systems. The average and maximum (Delta) Ea*b* = 1.0 and 2.1 in 16-band mutispectral camera system, using Macbeth 24 color patches. In the six-primary color projection display, average and maximum (Delta) Ea*b* = 1.3 and 2.7 with 30 test colors inside the display gamut. Moreover, the color reproduction results with different spectral distributions but same CIE tristimulus value are visually compared, and it is confirmed that the 6-primary display gives improved agreement between the original and reproduced colors.

Fast color processor with programmable interpolation by small memory (PRISM)

We present a novel color processor with programmable interpolation by small memory (PRISM). The input/output signals to/from the devices are flexibly converted by a 3-0 look-up table (LUT) with a PRISM interpolator. The PRISM architecture provides a simple computation algorithm with sufficient accuracy. The performance of PRISM interpolation is compared with other conventional methods. In practice, PRISM is less complicated than CUBE and PYRAMID, and more accurate than PYRAMID and TETRAHEDRON. PRISM cuts the memory size of LUT drastically to an orderof iO compared with a full-size LUT method and brings with it a large-scale integration color processor operating at a higher than video rate. The PRISM structure is the most suitable for the perceptual color spaces such as YCrCb or CIELAB and very useful for device-independent color reproduction and transmission. Typical applications by a PRISM color processor are presented.

Color conversion for a multi-primary display using linear interpolation on equi-luminance plane method (LIQUID)

A color-conversion method, which converts an XYZ tristimulus value to a set of display signals for a multi-primary display, has been developed. It is necessary for a display-device driver in a color-management system to select an optimal set of display signals that presents a given colorimetric value on a display screen. A multi-primary display, which has more than four primaries, has more than two sets of display signals that generate the same colorimetric value theoretically. A display-device driver, therefore, has to be designed according to a certain rule in order to choose a unique set of display signals for one colorimetric value only. To meet the requirement, a linear interpolation method, which uses three points on an equi-luminance plane in XYZ tristimulus value space, has been developed. The developed method assigns three points on an equi-luminance plane in XYZ tristimulus value space to ensure smoothness of a continuous gradation of display signals. The colorimetric accuracy of the developed method was evaluated with a six-primary DLP projector. Colorimetric error in CIELAB space between the targets and reproductions is about 0.8 on average and 2.9 at maximum.

Analysis of gamut mapping algorithms from the viewpoint of color name matching

— The validity of color name matching in analyzing gamut mapping algorithms is described in this paper. The performances of three types of gamut mapping algorithms (cusp scaling, clipping, and categorical weighting) are evaluated in a psychophysical experiment from the viewpoint of color name matching. Based on this evaluation, we have found that cusp-scaling algorithm is capable of controlling lightness reasonably well, compensating for a decrease in chroma caused by the difference in the size of device gamut between a source and a destination device. Ranking of reproducible performance by the three gamut mapping algorithms in the psychophysical experiment is arrived at on the basis of color name matching.

Video generation for high spatio-temporal resolution imaging

We propose a color video generation method for spatio-temporal high-resolution video imaging in dark conditions. The proposed method consists of two steps. First, RGB-separated video sequences with different spatio-temporal resolution sets are captured to increase the amount of captured light. Second, a high spatio-temporal resolution color video is reconstructed from those input video sequences in the regularization framework. We show the advantages of our method using a prototype camera system and simulation results.

Anticipating Driver Actions via Deep Neural Networks and New Driver Personalization Technique through Transfer Learning.

Anticipating driving behaviours is a promising technology for novel advanced driver assistance systems. In recent years, predicting a driver’s future action became an important element to preventive safety technologies and has been advancing greatly contributing to a reduction in road accidents. In this paper, we propose a deep learning network that anticipates driving actions based on information of subject vehicle as well as surrounding vehicles and environment. By re-using a network trained on a great number of various drivers’ data with different driving behaviours and linking it to a particular driver with particular taste we propose a method that enables the anticipation of driving behaviours that can be tailored to each driver individually, leading to improved user experiences. We experimentally test our method for acceleration, deceleration and brake profile anticipation task using actual driving data. Our results demonstrate the effectiveness of our approach, achieving a great improvement when anticipating for individuals.

Lensfree on-chip high-resolution imaging using two-way lighting, and its limitations

A high-magnification image of a biological sample can generally be obtained by an optical microscope with an objective lens, moving the image sensor with a sub-pixel shift and the subsequent image processing for super-resolution. However, to obtain a high-resolution image, a large number of images will be required for the super-resolution, and thus it is difficult to achieve real-time operation, and the field-of-view (FOV) is not sufficiently wide. The currently proposed digital holography technique places a sample on the image sensor and captures the interference fringe (hologram) to reconstruct a 3D high-resolution image in a computer. This technique ensures the features of a wide FOV, whereas the high resolution obtained by image processing cannot ensure real-time operation, because it requires recursive calculations of light propagation and adequate computer resources. To realize wide FOV and the real-time operation at the same time, we have developed a new technique: Lensfree on-chip high-resolution imaging using two-way lighting. High-resolution image is immediately obtained by image processing of the low-resolution images of the samples. This makes it possible to ensure a wide FOV, a deep depth of focus without the need for focus adjustment, and a continuously expanding operation. We also discuss the limitations of the high resolution.

Real-time texton substitution for super resolution

In this paper, texton substitution is implemented by graphics hardware to achieve super resolution of a video stream in realtime processing. The texton substitution method was proposed previously and evaluated for the super resolution of still images, and the textons were defined in wavelet space. In this paper, the wavelet transformation was accelerated by the graphics hardware [Hopf et al. 2001]], followed by two-dimensional filtering of three layers to produce a 10-dimensional vector of the texture at each pixel. The obtained textons were classified into a cluster based on the tree-based clustering technique by the graphics hardware using the pixel shader. This system was applied to the video stream of a face, and the experimental results show the effectiveness of the proposed system.

Super resolution based on texton substitution

The resolution of a texture image in computer graphics is determined by the camera system used to take the image. Zooming in on the object beyond the optical resolution results in a blurring of edges and details. This paper presents a “super resolution” method that uses a database of textures and texton substitution to create plausible high-frequency detail in enlarged texture images[Malik et al. 2001]. Conventional example based super-resolution techniques require the high computational cost and must be given the optimal parameters that depend on the input image[Freeman et al. 2000; Hertzmann et al. 2001]. In this paper, texton in the wavelet space and a pixel based (not a patch based) substitution is proposed for the practical use by their low computation process[Hopf et al. 2001].