Motonori Doi

Spectral estimation of human skin color using the Kubelka-Munk theory

The present paper describes a method for modeling human skin coloring and estimating the surface-spectral reflectance by using the Kubelka-Munk theory. First, human skin is modeled as two layers of turbid materials. Second, we describe the reflectance estimation problem as the Kubelka-Munk equations with unknown six parameters. These parameters are the regular reflectance at skin surface and the five weights for spectral absorption of such different pigments as melanin, carotene, oxy-hemoglobin, deoxy-hemoglobin, and bilirubin. Moreover, the optical coefficients of spectral absorption and scattering for the two skin layers and the thickness values of these layers are used for the solution. Finally, experiments are done for estimating the skin surface-spectral reflectance on some body parts, such as the cheeks of human face, the palm, the backs of hand, the inside of arm, and the outside of arm. It is confirmed that the proposed method is more reliable in all cases.

Image Analysis and Synthesis of Skin Color Textures by Wavelet Transform

The present paper proposes a skin color texture analysis method based on wavelet transform for detecting different texture patterns including local non-periodic features. We modeled skin color texture as the composition of four texture components of base color, internal skin texture, regular surface texture and local texture. The skin color image is decomposed to the four texture components by multi-resolution analysis using wavelet transform. A variety of skin images with different conditions of skin color and texture are created in a linear combination of the texture components. Experimental results show good separation of skin textures by wavelet analysis and realistic synthesized images

Real-time video surveillance system using omni-directional image sensor and controllable camera

The present paper describes a new indoor surveillance system for detection of an accident, such as fall or fit, happened to an aged single person. This system uses two types of cameras. One of them is the omni-directional image sensor for the tracking of the person’s position and detection of fall, and another is the controllable camera for capturing the detail of the person’s condition. The system detects points of the person’s head in images captured by some omni-directional image sensors, firstly. Then, a position of the person’s head in a room is computed from the points of person’s head in the images. When the person stops, the system classifies the persons pose into standing, sitting or lying according to the persons height. Then, the system judges that the accident has happened or not from the person’s pose, position and action. We made a prototype system with three omni-directional image sensors and a controllable camera. Then, we set the system in our laboratory’s room and experimented with the system. The implemented system detected the person’s position with the frame rate of 6 fps. In experiments, the error of position detection was 18 cm on the average. The error didn’t give serious influence to the control of the controllable camera. The error of height estimation was 6.9 cm. The conditions played by subjects were distinguished correctly.

Spectral estimation of skin color with foundation makeup

The analysis of skin color with makeup is needed to examine the effect of makeup to human skin. Foundation is cosmetics to cover undesirable color on skin and gives basic color to skin. It is important evaluate a change of skin color by the foundation. The present paper modeled the optics of skin with foundation makeup by two layers. Then, the surface spectrum of skin with foundation makeup is estimated by the Kubelka-Munk theory for radiation transfer in the turbid medium. The proposed algorithm can predict the spectral shape of skin surface with different types of foundation by appropriately determining model parameters. As an application of the skin reflectance estimation, we have developed a color simulator for human skin with foundation makeup. Experimental results showed a good accuracy of our estimation results.

Lock control system based on face identification

Reliable and feasible security systems are needed at limited-access entrances. We have made an experimental lock-control system that unlocks the door when the system identifies a registered persons face. The face identification procedure was designed for practical use. There are three main processes in the face identification: 1) extraction of the face, 2) normalization of the size and inclination of the face image, and 3) matching of facial features (eye, nose, and mouth). In order to achieve high reliability, the normalization process uses inner corners of eyes as reference points for affine transformation. Matching is performed on the characteristic facial features (eye, nose, and mouth) for exclusion of features that are easy to imitate, such as eyeglass frames and hairstyle. Thresholds for the matching are determined by statistical estimation. In experiments, the system realized a correct acceptance rate of 92.2% and a correct rejection rate of 99.6%.

Three-phase quadrature spectral matching imager using correlation image sensor and wavelength-swept monochromatic illumination

We propose a three-phase spectral matching imager (3PSMI) to realize a novel spectral matching method called quadrature spectral matching (QSM) in real time. The 3PSMI is comprised of the correlation image sensor (CIS) and wavelength-swept monochromatic illumination (WSMI) to perform QSM at each pixel on the CIS at a video frame rate. QSM consists of spectral correlation between an ac component of an object spectrum and an orthonormal pair of reference spectra, being equivalent to projecting the ac object spectrum onto a two-dimensional subspace spanned by the reference spectra. Similarity of the ac object spectrum to the reference spectra is evaluated in terms of the azimuth angle of the projection, independently of the norm of the ac object spectrum as well as spatial intensity distribution of the WSMI. A programable spectral light source is employed to implement the WSMI so that the spectral characteristics of the WSMI and CIS cancel each other and thus do not affect QSM on the 3PSMI. Experimental results confirm that the developed 3PSMI system can distinguish objects with smaller difference in spectral reflectance in real time than RGB imaging with off-the-shelf cameras.

Spectral estimation of made-up skin color under various conditions

A method is proposed for estimating the spectral reflectance of made-up skin color under various conditions including the undesirable colored skin. The color of dark spot is caused by increasing the component of melanin. The reddish skin is caused by the increase of hemoglobin. Our method uses the Kubelka-Munk theory to calculate the surface spectral reflectance human skin. This theory calculates the reflectance and transmittance of the light passing through a turbid medium from the absorption and scattering of the medium. The spectral reflectance of made-up skin is estimated by adjusting parameters of the thickness of the makeup layer. The proposed estimation method is evaluated on an experiment in detail. First, we measure the spectral reflectance of facial skin under the three conditions of normal skin, undesirable skin, and made-up skin. The undesirable skin includes stain, suntan or ruddy skin. The made-up skin means the skin with foundation on the normal skin, the stain, the suntan and the ruddy skin. Second, we estimate the spectral reflectance of made-up skins from the reflectance of bare skins and optical characteristics of foundations. Good coincidence between the estimated reflectance and the direct measurement shows the feasibility of the proposed method.

Human interface based on finger gesture recognition using omni-directional image sensor

A camera is not a special device for a personal computer. An omni-directional image sensor is a camera that can capture a surrounding image in every direction at a time by using a convex mirror and a video camera. When we grab the omni-directional sensor, the camera captures image of fingers. We propose a human interface based on finger gesture recognition using an omni-directional image sensor. By grabbing the omni-directional image sensor, a user can input commands to the computer. The finger image provides the number of fingers, their area, and their position. The proposed interface recognizes the operations of pushing, pulling, rotating and translating the change of these finger parameters. We implemented the interface for operation of a virtual object using finger gestures to a prototype system.

Robust estimation of pigment distributions from multiband skin images and its application to realistic skin image synthesis

This paper describes a robust method for estimating pigment distributions on a skin surface from multiband images. The spatial distributions of the pigments such as melanin, oxy-hemoglobin and deoxy-hemoglobin give rise to a color texture. The distributions are estimated by using the Kubelka-Munk theory. The accuracy of estimating the pigment distributions is affected by a fine texture of sulcus cutis and a broad texture of shade caused by three-dimensional body shape. In order to separate these textures from the color texture, wavelet-based multi-resolution analysis (MRA) is applied to the multiband images before the pigment estimation, because the textures of sulcus cutis and shade predominantly have low and high spatial frequency components in the multiband skin images, respectively. Realistic skin image is synthesized from modified pigment distributions with additional features such as stain, inflammation and bruise by changing the concentrations of melanin, oxy-hemoglobin and deoxy-hemoglobin, respectively. The experimental results of skin image synthesis show good feasibility of the proposed method.

Spectral matching imager with three-phase quadrature detection

This paper proposes a spectral matching imager with a novel three-phase quadrature detection method. The proposed imager consists of the time-domain correlation image sensor (CIS) and a high-speed programmable spectral light source (PSLS) to produce the correlations between each of a quadrature pair of reference spectral functions and spectral functions of objects. This is realized by modulating the object spectra into temporal signals with the PSLS and then producing temporal correlations with reference signals at each pixel of the CIS. The object that has the same spectral function as the target spectral function is detected as a vector with an azimuthal angle of zero projected onto the two-dimensional (2-D) space spanned by the reference spectral functions. The proposed method can estimate the correlation coefficient between the object and reference spectra more reliably in a 2-D space than previous 1-D spectral matching methods, by factoring out the norm of the object spectral function. Experimental results for a color chart confirm the effectiveness of the proposed method.