Hitoshi Konosu

Audiovisual recognition of spontaneous interest within conversations

In this work we present an audiovisual approach to the recognition of spontaneous interest in human conversations. For a most robust estimate, information from four sources is combined by a synergistic and individual failure tolerant fusion. Firstly, speech is analyzed with respect to acoustic properties based on a high-dimensional prosodic, articulatory, and voice quality feature space plus the linguistic analysis of spoken content by LVCSR and bag-of-words vector space modeling including non-verbals. Secondly, visual analysis provides patterns of the facial expression by AAMs, and of the movement activity by eye tracking. Experiments base on a database of 10.5h of spontaneous human-to-human conversation containing 20 subjects in gender and age-class balance. Recordings are fulfilled with a room microphone, camera, and headsets for close-talk to consider diverse comfort and noise conditions. Three levels of interest were annotated within a rich transcription. We describe each information stream and a fusion on an early level in detail. Our experiments aim at a person-independent system for real-life usage and show the high potential of such a multimodal approach. Benchmark results based on transcription versus automatic processing are also provided.

A bio-inspired predictive sensory-motor coordination scheme for robot reaching and preshaping

Abstract This paper presents a sensory-motor coordination scheme for a robot hand-arm-head system that provides the robot with the capability to reach an object while pre-shaping the fingers to the required grasp configuration and while predicting the tactile image that will be perceived after grasping. A model for sensory-motor coordination derived from studies in humans inspired the development of this scheme. A peculiar feature of this model is the prediction of the tactile image. The implementation of the proposed scheme is based on a neuro-fuzzy module that, after a learning phase, starting from visual data, calculates the position and orientation of the hand for reaching, selects the best-suited hand configuration, and predicts the tactile feedback. The implementation of the scheme on a humanoid robot allowed experimental validation of its effectiveness in robotics and provided perspectives on applications of sensory predictions in robot motor control.

Warning: to err is human [human-friendly robot dependability]

In this article we describe current and future technologies for ensuring the safety of the operators maneuvering a Skill-Assist working in an automobile final assembly process. It proposes a method that allows a Skill-Assist to detect operation slip from a sequence of an operators hand motion trajectory data represented using hidden Markov models (HMM). The method can also determine either safety- or productivity-oriented control policy throughout two-way interpretation of the belief structure. A workability improvement process that comprises a fault-warning (FW) safety-preservation (SP) space policy and teaching data renewal subprocesses allows for optimal reconstruction of the policy determinant observation space and HMMs. Some experimental results are shown to demonstrate the effectiveness of both the control policy determination and the workability improvement process.

Artificial balancer - Supporting device for postural reflex

The evolutionarily novel ability to keep ones body upright while standing or walking, the human balance, deteriorates in old age or can be compromised after accidents or brain surgeries. With the aged society, age related balance problems are on the rise. Persons with balance problems are more likely to fall during their everyday life routines. Especially in elderly, falls can lead to bone fractures making the patient bedridden, weakening the body and making it more prone to other diseases. Health care expenses for a fall patient are often very high. There is a great deal of research being done on exoskeletons and power assists. However, these technologies concentrate mainly on the amplifications of human muscle power while balance has to be provided by the human themself. Our research has been focused on supporting human balance in harmony with the humans own posture control mechanisms such as postural reflexes. This paper proposes an artificial balancer that supports human balance through acceleration of a flywheel attached to the body. Appropriate correcting torques are generated through our device based on the measurements of body deflections. We have carried out experiments with test persons standing on a platform subject to lateral perturbations and ambulatory experiments while walking on a balance beam. These experiments have demonstrated the effectiveness of our device in supporting balance and the possibility of enhancing balance-keeping capability in human beings through the application of external torque.

The features of Gait Exercise Assist Robot: Precise assist control and enriched feedback

BACKGROUND In a patient with severe hemiplegia, the risk of the knee giving way is high during the early stage of gait exercise with an ankle-foot orthosis. However, use of a knee-ankle-foot orthosis has many problems such as large amount of assistance and compensatory motions. To resolve these problems, we have engaged in the development of the Gait Exercise Assist Robot (GEAR). OBJECTIVE To evaluate the improvement efficiency of walk with GEAR in a stroke patient. METHODS The subject was a 70-year-old man presented with left thalamus hemorrhage and right hemiplegia. The patient underwent exercise with the GEAR 5 days a week, for 40 minutes per day. We evaluated the Functional Independence Measure score for walk (FIM-walk score) every week. The control group consisted of 15 patients aged 20-75 years with hemiplegia after primary stroke, who had equivalent walking ability with the subject at start. As the primary outcome, we defined improvement efficiency of FIM-walk, which was gain of FIM-walk divided the number of required weeks. RESULTS Improvement efficiency of FIM-walk of the subject was 1.5, while that of control group was 0.48±3.2 (mean±SD). CONCLUSIONS GEAR is potentially useful for gait exercise in hemiplegic patients.