Koji Kamei

Object-based activity recognition with heterogeneous sensors on wrist

This paper describes how we recognize activities of daily living (ADLs) with our designed sensor device, which is equipped with heterogeneous sensors such as a camera, a microphone, and an accelerometer and attached to a users wrist. Specifically, capturing a space around the users hand by employing the camera on the wrist mounted device enables us to recognize ADLs that involve the manual use of objects such as making tea or coffee and watering plant. Existing wearable sensor devices equipped only with a microphone and an accelerometer cannot recognize these ADLs without object embedded sensors. We also propose an ADL recognition method that takes privacy issues into account because the camera and microphone can capture aspects of a users private life. We confirmed experimentally that the incorporation of a camera could significantly improve the accuracy of ADL recognition.

Object-Blog System for Environment-Generated Content

The object-blog service application automatically converts raw sensor data to environment-generated content (EGC), including texts, graphs, and figures. This conversion facilitates data searching and browsing. Generated content can serve several purposes, including memory aids, security, and communication media. In object-blog, personified objects automatically post entries to a Weblog about sensor data obtained from sensors attached to the objects. Feedback thus far from participants working with object-blog in an experimental environment has been positive.

Context-aware web search in ubiquitous sensor environments

This article proposes a new concept for a context-aware Web search method that automatically retrieves a webpage related to the daily activity that a user currently is engaged in and displays the page on nearby Internet-connected home appliances such as televisions. For example, when a user is washing a coffeemaker, a webpage is retrieved that includes tips such as “cleaning a coffee maker with vinegar removes stains well,” and the page is displayed on a nearby appliance. In this article, we design and implement a Web search method that employs ubiquitous sensors to monitor a users daily life. Our proposed method automatically searches for a webpage related to a daily activity by using a query constructed from the use of daily objects employed in the activity that is detected with object-attached sensors. We evaluate the search method with real datasets collected from vast numbers of sensors and achieve very accurate webpage retrieval. We then investigate the usefulness and effectiveness of a daily life Web search with Wizard-of-Oz (WOz)-like experiments. We confirm that the presentation of webpages related to daily activities improves participants future daily lives and triggers communication among the participants in the experiment.

Web searching for daily living

The new concept proposed in this paper is a query free web search that automatically retrieves a web page including information related to the daily activity that we are currently engaged in for automatically displaying the page on Internet-connected domestic appliances around us such as televisions. When we are washing a coffee maker, for example, a web page is retrieved that includes tips such as `cleaning a coffee maker with vinegar removes stains well. A method designed on the basis of this concept automatically searches for a web page by using a query constructed from the use of ordinary household objects that is detected by sensors attached to the objects. An in-situ experiment tests a variety of IR techniques and the experiment confirmed that our daily activities can produce related web pages with high accuracy.

Formation conditions of mutual adaptation in human-agent collaborative interaction

When an adaptive agent works with a human user in a collaborative task, in order to enable flexible instructions to be issued by ordinary people, it is believed that a mutual adaptation phenomenon can enable the agent to handle flexible mapping relations between the human user’s instructions and the agent’s actions. To elucidate the conditions required to induce the mutual adaptation phenomenon, we designed an appropriate experimental environment called “WAITER” (Waiter Agent Interactive Training Experimental Restaurant) and conducted two experiments in this environment. The experimental results suggest that the proposed conditions can induce the mutual adaptation phenomenon.

A Platform System for Developing a Collaborative Mutually Adaptive Agent

The characteristic task of service robots that can interact with humans is to achieve human-robot collaboration. Mutual adaptation is considered to be an important characteristic of robots, required for carrying out such collaborative tasks. Here, we introduce the concept of mutual adaptation, propose a learning model, and describe an experimental task to explain the above concept. A waiter robot performs a collaborative task using a platform system, which is developed by a constructive approach. The interactive and manual modes of this system are compared by performing a preliminary experiment to evaluate the effectiveness of the robots autonomous function. The results indicate that the robots autonomous function works well when operated in the interactive mode under short time or slow speed conditions.

Active adaptation in human-agent collaborative interaction

When a human user interacts with an adaptive agent to achieve human-agent collaboration, active adaptation is considered to be one of the critical characteristics of the agent. In order to investigate the principal features of active adaptation, we developed a human-agent collaborative experimental environment called WAITER (waiter agent interactive training experimental restaurant) and conducted two types of experiments, a Wizard of OZ (WOZ) agent experiment and an autonomous agent experiment. The objective of these experiments is to observe how human users perceive the agents and change their instructions when interacting with adaptive agents. The results indicate that humans can recognize changes in the agent’s actions and change their instruction methods accordingly. This implies that active adaptation of the agents may encourage the adaptation of the human users and help to build an adaptation loop between them. The experimental results also suggest that active adaptation may play an important role in a human-agent collaborative task.

Incremental Knowledge Construction for Real-World Event Understanding

The construction of real-world knowledge is required if we are to understand real-world events that occur in a networked sensor environment. Since it is difficult to select suitable events for recognition in a sensor environment a priori, we propose an incremental model for constructing real-world knowledge. Labeling is the central plank of the proposed model because the model simultaneously improves both the ontology of real-world events and the implementation of a sensor system based on a manually labeled event corpus. A labeling tool is developed in accordance with the model and is evaluated in a practical labeling experiment.

The Event Search Engine

In a remote or local environment in which a sensor network always collects data produced by sensors attached to physical objects, the engine presented here saves the data sent through the Internet and searches for data segments that correspond to real-world events by using natural language NL words in a query that are input in an web browser. The engine translates each query into a physical quantity representation searches for a sensor data segment that satisfies the representation, and sends back the event occurrence time, place, or related objects as a reply to the query to the remote or local environment in which the web browser displays them. The engine, which we expect to be one of the upcoming Internet services, exemplifies the concept of symbiosis that bridges the gaps between the real space and the digital space.

Actively Adaptive Agent for Human-Agent Collaborative Task

Active interface is one of critical characteristics of agents who have to interact with human users to achieve human-agent collaboration. This characteristic is especially important in beginning phase of human-agent interaction when an ordinary human user begins to interact with an adaptive autonomous agent. In order to investigate principal characteristics of an active interface, we developed a human-agent collaborative experimental environment named WAITER. Two types of experiment: WOZ agent experiment and autonomous agent experiment were conducted. Objective of the experiment is to observe how human users change their instructions when interacting with adaptive agents with different degree of freedom. Experimental results indicate that participants can recognize changes of agents actions and change their instruction methods accordingly. It infers that changes of instruction method depend not only on waiter agents reactions, but also on human managers cognitive models of the agent.