Hongsheng He

DietCam: Multiview Food Recognition Using a Multikernel SVM

Food recognition is a key component in evaluation of everyday food intakes, and its challenge is due to intraclass variation. In this paper, we present an automatic food classification method, DietCam, which specifically addresses the variation of food appearances. DietCam consists of two major components, ingredient detection and food classification. Food ingredients are detected through a combination of a deformable part-based model and a texture verification model. From the detected ingredients, food categories are classified using a multiview multikernel SVM. In the experiment, DietCam presents reliability and outperformance in recognition of food with complex ingredients on a database including 15,262 food images of 55 food types.

Wearable Ego-Motion Tracking for Blind Navigation in Indoor Environments

This paper proposes an ego-motion tracking method that utilizes visual-inertial sensors for wearable blind navigation. The unique challenge of wearable motion tracking is to cope with arbitrary body motion and complex environmental dynamics. We introduce a visual sanity check to select accurate visual estimations by comparing visually estimated rotation with measured rotation by a gyroscope. The movement trajectory is recovered through adaptive fusion of visual estimations and inertial measurements, where the visual estimation outputs motion transformation between consecutive image captures, and inertial sensors measure translational acceleration and angular velocities. The frame rates of visual and inertial sensors are different, and vary with respect to time owning to visual sanity checks. We hence employ a multirate extended Kalman filter (EKF) to fuse visual and inertial estimations. The proposed method was tested in different indoor environments, and the results show its effectiveness and accuracy in ego-motion tracking.

Recognition of Car Makes and Models From a Single Traffic-Camera Image

This paper proposes the recognition framework of car makes and models from a single image captured by a traffic camera. Due to various configurations of traffic cameras, a traffic image may be captured in different viewpoints and lighting conditions, and the image quality varies in resolution and color depth. In the framework, cars are first detected using a part-based detector, and license plates and headlamps are detected as cardinal anchor points to rectify projective distortion. Car features are extracted, normalized, and classified using an ensemble of neural-network classifiers. In the experiment, the performance of the proposed method is evaluated on a data set of practical traffic images. The results prove the effectiveness of the proposed method in vehicle detection and model recognition.

Robust line detection using two-orthogonal direction image scanning

we propose a detector using two-orthogonal direction image scanning (TODIS) with the capability of multi-scale detection by utilizing the prior knowledge of lines. The two step scanning procedures and the prominence filter are designed to impose the geometric constrains on pixel edges for the purpose of providing prior assessment on the structure of line segments. In addition, through dividing the original problem into smaller subproblems, the computational load has been reduced, and there is no extra storage cost for the detectors adaptiveness on the slope resolution of lines during detection. The simulation results shows significant improved detection accuracy as compared to the popular methods.

A saliency-driven robotic head with bio-inspired saccadic behaviors for social robotics

This paper presents a robotic head for social robots to attend to scene saliency with bio-inspired saccadic behaviors. Scene saliency is determined by measuring low-level static scene information, motion, and object prior knowledge. Towards the extracted saliency spots, the designed robotic head is able to turn gazes in a saccadic manner while obeying eye–head coordination laws with the proposed control scheme. The results of the simulation study and actual applications show the effectiveness of the proposed method in discovering of scene saliency and human-like head motion. The proposed techniques could possibly be applied to social robots to improve social sense and user experience in human–robot interaction.

Neural-network-based human intention estimation for physical human-robot interaction

To realize physical human-robot interaction, it is essential for the robot to understand the motion intention of its human partner. In this paper, human motion intention is defined as the desired trajectory in human limb model, of which the estimation is obtained based on neural network. The proposed method employs measured interaction force, position and velocity at the interaction point. The estimated human motion intention is integrated to the control design of the robot arm. The validity of the proposed method is verified through simulation.

Design and development of Nancy, a social robot

In this paper, we present the design of a social robot, Nancy, which is developed as a platform for engaging social interaction. Targeting for a social, safe, interactive and user-friendly robot mate, the design philosophy of Nancy is presented with mechanical, electrical, artificial skin and software specifications. In particular, there are 32 degrees of freedom (DOFs) through the whole body of Nancy, and the social intelligence is implemented based on vision, audio and control subsystems.

DietCam: Multi-view regular shape food recognition with a camera phone

Abstract This paper presents an automatic multi-view food classification of a food intake assessment system on a smart phone. Food intake assessment plays important roles in obesity management, which has shown significant impacts on public healthcare. Conventional dietary record-based food intake assessment methods are not popularly applied due to their inconvenience and high reliance on human interactions. This paper presents a smart phone application, named DietCam, to recognize food intakes automatically. The major difficulties in food recognition from images come from uncertainties of food appearances and deformable nature of food especially when they are on a complex background environment. The proposed DietCam system utilizes a multi-view recognition method that separates every food by estimating the best perspective and recognizing them using a probabilistic method. The implemented DietCam system on an iPhone 4 platform showed improved performance compared with baseline methods for food recognition, with an average accuracy of 84% for the selective regular shape foods.

Bottom-up saliency detection for attention determination

In this paper, the technique of saliency detection is proposed to model people’s biological ability of attending to their interest. There are two phases in the scheme of intelligent saliency searching: saliency filtering and saliency refinement. In saliency filtering, non-salient regions of a scene image are filtered out by measuring information entropy and biological color sensitivity. The information entropy evaluates the level of knowledge and energy contained, and the color sensitivity measures biological stimulation of a presented scene. In saliency refinement, candidate salient regions obtained are cultivated for a good representation of saliency by extracting salient objects, similarly to people’s manner of perception. The performance of the proposed technique is studied on noiseless and noisy natural scenes and evaluated with eye fixation data. The evaluation proved the effectiveness of the approach in discovering salient regions or objects from scene images. The performance of addressing transformation and illumination variance is also investigated.

Visual Attention Prediction Using Saliency Determination of Scene Understanding for Social Robots

In this paper, the biological ability of visual attention is modeled for social robots to understand scenes and circumstance. Visual attention is determined by evaluating visual stimuli and prior knowledge in the intelligent saliency searching. Visual stimuli are measured using information entropy and biological color sensitivities, where the information entropy evaluates information qualities and the color sensitivity assesses biological attraction of a presented scene. We also learn and utilize the prior knowledge of people’s focus in the prediction of visual attention. The performance of the proposed technique is studied on different sorts of natural scenes and evaluated with fixation data of actual eye-tracking database. The experimental results proved the effectiveness of the proposed technique in discovering salient regions and predicting visual attention. The robustness of the proposed technique to transformation and illumination variance is also investigated. Social robots equipped with the proposed technique can autonomously determine their attention to a scene autonomously so as to behave naturally in the human robot interaction.