Masahiro Shiomi

Interactive humanoid robots for a science museum

One objective of the Intelligent Robotics and Communication Laboratories is to develop an intelligent communication robot that supports people in an open everyday environment by interacting with them. A humanoid robot can help achieve this objective because its physical structure lets it interact through human-like body movements such as shaking hands, greeting, and pointing. Both adults and children are more likely to understand such interactions than interactions with an electronic interface such as a touch panel or buttons. To behave intelligently during an interaction, a robot requires many types of information about its environment and the people with whom it interacts. However, in open everyday environments, simple recognition functions such as identifying an individual are difficult because the presence and movement of a large number of people as well as unfavorable illumination and background conditions affect the robots sensing ability. We integrated humanoid robots and ubiquitous sensors in an autonomous system to assist visitors at an Osaka Science Museum exhibit

An affective guide robot in a shopping mall

To explore possible robot tasks in daily life, we developed a guide robot for a shopping mall and conducted a field trial with it. The robot was designed to interact naturally with customers and to affectively provide shopping information. It was also designed to repeatedly interact with people to build a rapport; since a shopping mall is a place people repeatedly visit, it provides the chance to explicitly design a robot for multiple interactions. For this capability, we used RFID tags for person identification. The robot was semi-autonomous, partially controlled by a human operator, to cope with the difficulty of speech recognition in a real environment and to handle unexpected situations. A field trial was conducted at a shopping mall for 25 days to observe how the robot performed this task and how people interacted with it. The robot interacted with approximately 100 groups of customers each day. We invited customers to sign up for RFID tags and those who participated answered questionnaires. The results revealed that 63 out of 235 people in fact went shopping based on the information provided by the robot. The experimental results suggest promising potential for robots working in shopping malls.

Interactive Humanoid Robots for a Science Museum

One objective of the Intelligent Robotics and Communication Laboratories is to develop an intelligent communication robot that supports people in an open everyday environment by interacting with them. A humanoid robot can help achieve this objective because its physical structure lets it interact through human-like body movements such as shaking hands, greeting, and pointing. Both adults and children are more likely to understand such interactions than interactions with an electronic interface such as a touch panel or buttons. To behave intelligently during an interaction, a robot requires many types of information about its environment and the people with whom it interacts. However, in open everyday environments, simple recognition functions such as identifying an individual are difficult because the presence and movement of a large number of people as well as unfavorable illumination and background conditions affect the robots sensing ability. We integrated humanoid robots and ubiquitous sensors in an autonomous system to assist visitors at an Osaka Science Museum exhibit

Who will be the customer?: a social robot that anticipates people's behavior from their trajectories

For a robot providing services to people in a public space such as a train station or a shopping mall, it is important to distinguish potential customers, such as window-shoppers, from other people, such as busy commuters. In this paper, we present a series of techniques for anticipating peoples behavior in a public space, mainly based on the analysis of accumulated trajectories, and we demonstrate the use of these techniques in a social robot. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks peoples positions as well as their local behaviors such as fast walking, idle walking, or stopping. We accumulated peoples trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and peoples typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors, and also the probable local behaviors of individuals a few seconds in the future. In a field experiment we demonstrate that this system enables the robot to serve people efficiently.

Abstracting People's Trajectories for Social Robots to Proactively Approach Customers

For a robot providing services to people in a public space such as a shopping mall, it is important to distinguish potential customers, such as window shoppers, from other people, such as busy commuters. In this paper, we present a series of abstraction techniques for peoples trajectories and a service framework for using these techniques in a social robot, which enables a designer to make the robot proactively approach customers by only providing information about target local behavior. We placed a ubiquitous sensor network consisting of six laser range finders in a shopping arcade. The system tracks peoples positions as well as their local behaviors, such as fast walking, idle walking, wandering, or stopping. We accumulated peoples trajectories for a week, applying a clustering technique to the accumulated trajectories to extract information about the use of space and peoples typical global behaviors. This information enables the robot to target its services to people who are walking idly or stopping. The robot anticipates both the areas in which people are likely to perform these behaviors as well as the probable local behaviors of individuals a few seconds in the future. In a field experiment, we demonstrate that this service framework enables the robot to serve people efficiently.

A semi-autonomous communication robot: a field trial at a train station

This paper reports an initial field trial with a prototype of a semi-autonomous communication robot at a train station. We developed an operator-requesting mechanism to achieve semi-autonomous operation for a communication robot functioning in real environments. The operator-requesting mechanism autonomously detects situations that the robot cannot handle by itself; a human operator helps by assuming control of the robot. This approach gives semi-autonomous robots the ability to function naturally with minimum human effort. Our system consists of a humanoid robot and ubiquitous sensors. The robot has such basic communicative behaviors as greeting and route guidance. The experimental results revealed that the operator-requesting mechanism correctly requested operators help in 85% of the necessary situations; the operator only had to control 25% of the experiment time in the semi-autonomous mode with a robot system that successfully guided 68% of the passengers. At the same time, this trial provided the opportunity to gather user data for the further development of natural behaviors for such robots operating in real environments.

Analysis of People Trajectories with Ubiquitous Sensors in a Science Museum

This paper reports a study that estimated visitor positions, visiting patterns, and inter-human relationships at a science museum using information from RFID readers. In the science museum, we exhibited humanoid robots. Visitors were invited to wear RFID tags to interact with the robots. Visitor behavior was simultaneously observed using 20 RFID readers, distributed throughout the entire floor, that roughly measured the distances of nearby tags. We integrated the outputs from all RFID readers to estimate visitor trajectories that were used to analyze three perspectives: space, visiting patterns, and relationships. Regarding space, we identified crowded and uncrowded areas. We found several typical visiting patterns, such as visited at every exhibit and directly going to robot area. We also identified atypical visiting behavior. Regarding relationships, for example, we estimated 68% coverage of group-member relationships with 91% reliability.

Do elderly people prefer a conversational humanoid as a shopping assistant partner in supermarkets

Assistive robots can be perceived in two main ways: tools or partners. In past research, assistive robots that offer physical assistance for the elderly are often designed in the context of a tool metaphor. This paper investigates the effect of two design considerations for assistive robots in a partner metaphor: conversation and robot-type. The former factor is concerned with whether robots should converse with people even if the conversation is not germane for completing the task. The latter factor is concerned with whether people prefer a communication/function oriented design for assistive robots. To test these design considerations, we selected a shopping assistance situation where a robot carries a shopping basket for elderly people, which is one typical scenario used for assistive robots. A field experiment was conducted in a real supermarket in Japan where 24 elderly participants shopped with robots. The experimental results revealed that they prefer a conversational humanoid as a shopping assistant partner.

Field trial of networked social robots in a shopping mall

This paper reports the challenges of developing multiple social robots that operate in a shopping mall. We developed a networked robot system that coordinates multiple social robots and sensors to provide efficient service to customers. It directs the tasks of robots based on their positions and peoples walking behavior, manages the paths of robots, and coordinates the conversation-performance between two robots. Laser range finders were distributed in the environment to estimate peoples positions. The system estimates such human walking behaviors as “stopping” or “idle walking” to direct robots to provide appropriate tasks to appropriate people. Each robot interacts with people to provide recommendation information and route information about shops. The system sometimes simultaneously uses two robots to lead people from one place to another. The field trial, which was conducted in a shopping mall where four robots interacted with 414 people, revealed the effectiveness of the network robot system for guiding people around a shopping mall as well as increasing their interest.

Towards a Socially Acceptable Collision Avoidance for a Mobile Robot Navigating Among Pedestrians Using a Pedestrian Model

Safe navigation is a fundamental capability for robots that move among pedestrians. The traditional approach in robotics to attain such a capability has treated pedestrians as moving obstacles and provides algorithms that assure collision-free motion in the presence of such moving obstacles. In contrast, recent studies have focused on providing the robot not only collision-free motion but also a socially acceptable behavior by planning the robot’s path to maintain a “social distance” from pedestrians and respect their personal space. Such a social behavior is perceived as natural by the pedestrians and thus provides them a comfortable feeling, even if it may be considered a decorative element from a strictly safety oriented perspective. In this work we develop a system that realizes human-like collision avoidance in a mobile robot. In order to achieve this goal, we use a pedestrian model from human science literature, a version of the popular Social Force Model that was specifically designed to reproduce conditions similar to those found in shopping malls and other pedestrians facilities. Our findings show that the proposed system, which we tested in 2-h field trials in a real world environment, not only is perceived as comfortable by pedestrians but also yields safer navigation than traditional collision-free methods, since it better fits the behavior of the other pedestrians in the crowd.