Miwako Doi

A real-time vision-based interface using Motion Processor and applications to robotics

The authors developed a new vision-based interface that detects gestures in real time to enable interaction between the user and computer. This vision-based interface consists of two elements: the “Motion Processor™” and the “extraction of the region of interest (ROI).” The Motion Processor is a new image acquisition device. By illuminating the target object with near-infrared light and using image sensors to collect the reflected light, the Motion Processor can eliminate the background and collect shape, motion, and depth information of only the target object. In addition, the authors proposed a method of using the collected image depth information to quickly and stably extract the ROI and showed, according to evaluation experiments on a PC, that the region can be detected with high precision within 0.06 second. Real-time sensing of the shape and motion of a specific object from an image can be implemented by using the Motion Processor and this ROI extraction method, enabling this interface to function as a computer eye that can easily recognize a target object. The authors applied this real-time vision-based interface to the field of robotics and used it together with speech recognition on a single PC to create a prototype of a pet robot system that can respond to the users gestures.

LifeMinder: an evidence-based wearable healthcare assistant

Recent needs for evidence-based healthcare systems are increasing. We have been developing a wearable healthcare assistant system LifeMinder that synchronously records physiological information and contextual information. LifeMinder is a long-term monitoring and easy retrieval system for an evidence-based healthcare. This paper discusses a prototype of LifeMinder. This prototype can sense pulse waves and users actions / postures and capture contextual photos and continuous voices. These collected data are automatically synchronized, sent to a healthcare PC, stored in organized XML formats and can be easily retrieved on Web pages.

A kinetic and 3D image input device

Gesture recognition in real time can bridge a gap between humans and computers. Object segmentation from the background is a critical problem in the conventional gesture recognition technology. We have developed a new input device which can detect a kinetic and 3D image of a hand in real time. We call it “Motion Processor”. The Motion Processor with infrared light sources and an area sensor can detect the reflected light image of a hand at 30 frames per second. The image resolution is 64 pixels by 64 pixels. It is easy to recognize gestures and motions in real time based on the detected hand images. This gesture recognition bridges a gap between humans and computers.

Design targeting voice interface robot capable of active listening

The EU, South Korea and Japan have a pressing need to compensate for growing labor shortages in their aging societies. There is growing awareness that robotic technology has the potential to ameliorate this problem in terms of both physical and mental labor. To take an example of mental labor, a human therapist dealing with elderly people must be an active listener. In order to realize a robot capable of active listening, we adopt Iveys basic listening sequence skills in microcounseling. In this paper, we describe a voice interface robot that realizes simple feedback, repeat feedback and questions for Iveys basic listening sequence. We conducted an experiment, whose results show that 69% of incidences of feedback have adequate reflective words for spoken sentences and 56% of questions are adequate for these reflective words.

A real-time living activity recognition system using off-the-shelf sensors on a mobile phone

We propose an in-home living activity recognition method using only off-the-shelf sensors, namely, an accelerometer and a microphone, which are commonly applied in mobile phones. The proposed method firstly estimates a users movement condition roughly by acceleration sensing. Secondly, it classifies the working condition in detail by acoustic sensing when it estimates the condition to be working by acceleration sensing. We developed a prototype system to recognize the users living activity in real time and conducted two experiments to confirm the feasibility of the proposed method. As a result of the first experiment, three movement conditions; quiet, walking, and working, are classified with more than 95% accuracy by acceleration sensing. And it classified working into seven conditions with 85.9% accuracy by acoustic sensing. Moreover, the result of the second experiment shows that it is effective to adopt instance-based recognition according to the assumed application.

Background sensing control for planning agents working in the real world

Online planning agents can adapt to the dynamic environment by continuously modifying plans during the plan execution. This adaptability of the online planning agents is based on the assumption that they can recognise the events happening in the environment. However, robotic agents working in the real environment cannot obtain necessary information from the outer world by simply turning all sensors on while executing a plan. They have to actively sense the world by changing the direction of the sensor to the target, analysing the raw sensor data, extracting important information and recognising the situation. Online planning is useless if they cannot obtain necessary information from the world. Therefore, it is crucial to decide what events the agent needs to recognise in the background while executing a plan. To solve this problem, this paper introduces a new background sensing control technique by which planning agents can effectively observe the real environment and obtain important information when necessary during the plan execution.

Development of document architecture extraction

The purpose of this study is the reduction of the burden in the document structurization process. A technique is presented for extracting the document architecture. As the technical document, 12,000 articles are extracted from the proceedings of a national convention. A summary of sample sentences as well as approximately 500 office documents within the organization also are examined as business documents. The rules for extracting the architecture are derived. The technique developed for document architecture extraction can extract such hierarchical structures as chapters and sections, as well as the reference structure to figures and tables from the technical document. The technique can also extract the hierarchical structure such as communications and reports from the business document. The technical and business documents can be discriminated by analyzing the character strings. As a result of evaluation using proceedings and in-office documents other than those used for deriving the rules, the error rate is 10.0 percent for the technical document and 23.0 percent for the business document. The error in extracting the reference structure is 8 percent. A field test is executed after improving the method so that the equations, figures and tables embedded in the text can be handled. The error rate is 5.4 percent for the technical document and 15.4 percent for the business document. It is verified through examples that the structurization can be achieved in a considerably shorter time than by manual processing. The developed document architecture extraction technique is commercialized as an automatic system by combining the technique with the layout attribute. The developed extraction technique will be utilized effectively in the hypertext conversion of the existing document and other problems, in addition to the layout processing.

Personal and Home Electronics and Our Changing Lifestyles

In this paper, we review the dramatic impact that technology has had on society and past predictions about future technology, and predict the impact of future technology. The methods with which technology affects society can be categorized into what we will call lifestyle “waves” of change. We will describe three specific types of waves and how they affect specific types of personal and home electronics. The lifestyle waves we will consider are: 1) time and place independence; 2) interactivity; and 3) integration of the physical and information worlds. We will consider how these waves influence three categories of personal and home electronics: 1) home appliances (i.e., washing machines, dishwashers, etc.); 2) audio/video products (i.e., TVs, stereos, etc.); and 3) personal electronic gadgets (i.e., mobile phones, navigational devices, etc.). These waves typically are introduced into one category of electronics and rapidly cross pollinate to all categories. We will also discuss how specific trends in technology, such as hardware, software, user interfaces, and content distribution, can generate these lifestyle waves of change. Finally, we make some predictions on how these waves and trends in technology may impact personal and home electronics in the future.

A wearable authoring system using organized multimedia data

We developed a wearable authoring system which can record a variety of multimedia data and automatically generate daily reports and personal diary. A user can record multimedia data by context sensor without using keyboard or mouse. Recorded multimedia data are indexed and organized by recognizing spoken key words and voice annotations. Using these organized multimedia data, the system can automatically generate daily reports and statistical charts in real time without editing.

A motional interface approach based on user's tempo

People naturally use gesture to aid inter-personal communication. However, during computer interaction, users are obliged to use conventional devices such as keyboards, mice, and game-pads. In this paper a motional interface is described that allows processing of gestural inputs through the medium of a 3D image-input device, or Motion Processor, which provides a more natural framework for human-computer interaction. Several PC applications for this real-time motional interface will be presented in the live demonstration session. These applications are edutainment prototypes that make use of natural hand movement.