Kiyotaka Fukumoto

Head-Free, Remote Eye-Gaze Detection System Based on Pupil-Corneal Reflection Method With Easy Calibration Using Two Stereo-Calibrated Video Cameras

We have developed a pupil-corneal reflection method-based gaze detection system, which allows large head movements and achieves easy gaze calibration. This system contains two optical systems consisting of components such as a camera and a near-infrared light source attached to the camera. The light source has two concentric LED rings with different wavelengths. The inner and outer rings generate bright and dark pupil images, respectively. The pupils are detected from a difference image created by subtracting the bright and dark pupil images. The light source also generates the corneal reflection. The 3-D coordinates of the pupils are determined by the stereo matching method using two optical systems. The vector from the corneal reflection center to the pupil center in the camera image is determined as r. The angle between the line of sight and the line passing through the pupil center and the camera (light source) is denoted as θ. The relationship θ = k | \bm r | is assumed, where k is a constant. The theory implies that head movement of the user is allowed and facilitates the gaze calibration procedure. In the automatic calibration method, k is automatically determined while the user looks around on the PC screen without fixating on any specific calibration target. In the one-point calibration method, the user is asked to fixate on one calibration target at the PC screen in order to correct the difference between the optical and visual axes. In the two-point calibration method, in order to correct the nonlinear relationship between θ and | r |, the user is asked to fixate on two targets. The experimental results show that the three proposed calibration methods improve the precision of gaze detection step by step. In addition, the average gaze error in the visual angle is less than 1° for the seven head positions of the user.

Prediction of the muscle strength by the muscle thickness and hardness using ultrasound muscle hardness meter

PurposesThe present study investigated whether a combination of the thickness and hardness of muscles without muscle tension can be used to estimate muscle strength during knee extension in adult males and females.MethodsSeventy-two males and thirty-three females, whose ages ranged from 18 to 35 years, participated in this study. We measured muscle thickness and hardness in the right anterior region of the thigh (rectus femoris muscle and vastus intermedius muscle) without muscle tension using an ultrasound muscle hardness meter, and the maximal voluntary isometric contraction of right knee extension (MVIC). The changing ratios (%) of the tissue thickness before compression to those during compression (compression ratio) are calculated as an index of the hardness. Higher ratio indicates a harder muscle compared with that of other individuals showing the same muscle thickness.ResultsIn male group, although the MVIC had significantly positive correlation to both muscle thickness (r=0.412, p<0.01) and compression ratio (r=0.233, p<0.05), their variables also had correlation mutually. In the female group, the MVIC has significantly positive correlation to only compression ratio (r=0.499, p<0.01), not muscle thickness (r=0.225, n.s.). On multiple linear regression analysis, the combination of two parameters (muscle thickness and compression ratio) allowed more accurate estimation of MVIC (r=0.573, p<0.01) in the female group.ConclusionThese findings suggested that the combination of muscle thickness and hardness is capable of effectively estimating muscle strength especially in females.

Head-free, remote gaze detection system based on pupil-corneal reflection method with using two video cameras: one-point and nonlinear calibrations

We developed a pupil-corneal reflection method-based gaze detection system, which allows head movements and achieves easy gaze calibration. The proposed gaze detection theory determines gaze points on a PC screen from the vector from the corneal reflection to pupil center, 3D pupil position, two cameras position, etc. In a gaze calibration procedure, after a user is asked to gaze one specific calibration target at a center of a PC screen, the nonlinear characteristic of the eyes has been automatically corrected while the user is using this gaze system. The experimental results show that the proposed calibration method improved the precision of gaze detection during browsing web pages. In addition, the average gaze error in the visual angle is less than 0.6 degree for the nine head positions.

Head-Free, Remote Eye-Gaze Detection System with Easy Calibration Using Stereo-Calibrated Two Video Cameras

The video-based, head-free, remote eye-gaze detection system based on detection of the pupil and the corneal reflection was developed using stereocalibrated two cameras. The gaze detection theory assumed the linear relationship; θ = k|r′|. Here, θ is the angle between the line of sight and the line connecting between the pupil and the camera, and |r’| indicates the size of the corneal reflection - pupil center vector. Three novel easy calibration methods were proposed; ‘automatic’, ‘one-point’, and ‘two-point’. In the ‘automatic’, the user does not have to fixate the specific location in the PC screen. In the ‘one-point’, the angular difference between the optical and visual axes of the eye was determined and used for compensation. The ‘two-point’ was proposed to compensate the nonlinear relationship between |r’| and θ, which occurs when θ is large. The precision of gaze detection was compared among the three methods using the developed gaze detection system.

Development of a Flexible System for Measuring Muscle Area Using Ultrasonography

Muscular strength can be estimated by quantification of muscle area. For this purpose, we developed a flexible measuring system for muscle area using ultrasonography. This method is completely safe and is particularly suitable for elderly people because the subjects are not required to perform any muscular contraction during measurement. The ultrasound probe is installed on a mechanical arm, and continuously scans fragmental images along the body surface. A wide-area cross-sectional image is then constructed using the measured images. The link mechanism is very flexible, enabling the operator to measure images for any body posture and body site. Use of the spatial compounding method reduces speckle and artifact noise in the resultant cross-sectional images. The operator can observe individual muscles (extensor, flexor muscle, etc.) in detail. We conducted experiments to evaluate the performance of the system. In the experiments, the position of the ultrasound probe was calculated with high accuracy according to the link posture. In addition, a high degree of correlation was verified between MR images and those of the developed system. We observed a reduction in noise due to use of the spatial compounding method, and propose a new calibration method for correcting the measured muscle area, which were slightly deformed by the contact pressure of the ultrasound probe. Finally, we examined the relation between muscular area and muscular strength in young and middle-aged subjects. The results of these experiments confirm that the developed system can estimate muscular strength based on muscular area.

A cross-sectional ultrasound imaging for measuring body composition

We have developed a measuring system to visualize a complete cross-sectional image of the human extremity using ultrasonography. This system uses several ultrasound probes, and these probes measure fragmentary graphical images of one cross-sectional plane which are then transformed into a complete cross-sectional image. The developed system is superior to other imaging devices in many respects because it is portable, safe, and inexpensive. These advantages can be available in the field survey. This paper discusses the system components and the experiments to verify the proposed method.

Analysis of thigh cross-sectional proportion using the portable ultrasound imaging system

To evaluate capacity of the elderly body and the extremities, we developed a measurement system for thigh cross-sectional image. The cross-sectional images were suitable for observing muscle volume. Some previous studies estimated muscular strength based on muscle volume measured the medical images. However there were no studies that used information regarding proportion in human extremities, e. g. muscles, subcutaneous fat and a bone. In this case, the measurements provide the same results even if the proportion is different among two people. We have investigated changes of thigh proportion by age and gender using the cross-sectional ultrasound images. In the survey, 150 ultrasound echo images and the body measurements (height, weight, and so on) were used. We conducted t-test between two groups that have different age, and discussed the causes of their differences. The statistical analysis revealed that the proportion of human thigh was dramatically changed with aging; the shape of thigh outline was deformed and the position of the thigh bone in the cross-sectional image was moved. We considered that the decreasing of the tissue elasticity with aging is responsible for these changes. The proportion analysis can be expected to be a new approach for developing a novel evaluation technique of the elderly body and extremity ability.

A Portable Measuring System for Cross-Sectional Ultrasound Images Using Spatial Compounding and Edge Sharpening

This paper proposes a compensation for a blurred image caused by misregistration in spatial compounding. The edge sharpening based on morphological operations is applied to the compounded image. A portable measuring system of a cross-sectional ultrasound image is designed for measuring body composition. The system is developed for non-medical applications, for example, measurement of the volume of muscle and subcutaneous fat in lower extremities, upper extremities and abdomen for health care and sports science. A number of partially overlapping images that have been obtained from multiple angles are combined into a single compounded image. In experiment on human abdomen, the image quality was improved in speckle suppression and edge enhancement by using ensemble averaging and morphological edge sharpening. Resulting wide field-of-view image was sufficiently deblurred without emphasizing noise.

Proposal of Remote Face-to-Face Communication System with Line of Sight Matching Based on Pupil Detection

In general, remote face-to-face communication systems via the Internet, it is impossible to match the eyes of a user and a communication partner with each other because their face positions displayed in PC screens, at which they look, are different from the positions of the cameras for shooting their faces. To solve this problem, we proposed a remote head-free system which was based on the combination of a wide-angle color camera located behind a 45-deg inclined magic mirror, our pupil detection technique, image deformation techniques, and so on. The same two systems were assigned to the user and the partner. The narrow region image including a whole of the user’s face image was cut out from the image obtained from the wide-angle color camera in the user’s system, was displayed in the partner’s system so that the midpoint of the detected right and left pupils of the user’s face image virtually located at the position of the color camera in the partner’s system, and vice versa. In addition, to deal with large back and forth or lateral head movements, the displayed face image of the user was adjusted in its size and distorted by using the projection transformation so that it always looked like the same size as the user’s face and looked like the front face as seen from the partner, and vice versa. The questionnaire results obtained from experiment showed the tendency that the subjects’ lines of sight matched each other, and the other questionnaire items about the system showed high evaluation.

Improvement of Accuracy in Remote Gaze Detection for User Wearing Eyeglasses Using Relative Position Between Centers of Pupil and Corneal Sphere

One of the general problems of the pupil-corneal reflection-based gaze detection systems is that the frames and lens of eyeglasses produce reflection images of the light sources in the camera image when a user wears eyeglasses. The glass reflections tend to be misdetected as the pupil and corneal reflections. In the present paper, we propose a novel geometrical methodology based on the optical structure of the eyeball to detect a true pair of the pupil and corneal reflection. The experimental results show that the proposed method improved the precision of gaze detection when the subjects wore glasses or when disturbance light sources existed.