Yasser F. O. Mohammad

Unsupervised simultaneous learning of gestures, actions and their associations for Human-Robot Interaction

Human-Robot Interaction using free hand gestures is gaining more importance as more untrained humans are operating robots in home and office environments. The robot needs to solve three problems to be operated by free hand gestures: gesture (command) detection, action generation (related to the domain of the task) and association between gestures and actions.

Constrained Motif Discovery in Time Series

The goal of motif discovery algorithms is to efficiently find unknown recurring patterns. In this paper, we focus on motif discovery in time series. Most available algorithms cannot utilize domain knowledge in any way which results in quadratic or at least super-linear time and space complexity. In this paper we define the Constrained Motif Discovery problem which enables utilization of domain knowledge into the motif discovery process. The paper then provides two algorithms called MCFull and MCInc for efficiently solving the constrained motif discovery problem. We also show that most unconstrained motif discovery problems be converted into constrained ones using a change-point detection algorithm. A novel change-point detection algorithm called the Robust Singular Spectrum Transform (RSST) is then introduced and compared to traditional Singular Spectrum Transform using synthetic and real-world data sets. The results show that RSST achieves higher specificity and is more adequate for finding constraints to convert unconstrained motif discovery problems to constrained ones that can be solved using MCFull and MCInc. We then compare the combination of RSST and MCFull or MCInc with two state-of-the-art motif discovery algorithms on a large set of synthetic time series. The results show that the proposed algorithms provided four to ten folds increase in speed compared the unconstrained motif discovery algorithms studied without any loss of accuracy. RSST+MCFull is then used in a real world human-robot interaction experiment to enable the robot to learn free hand gestures, actions, and their associations by watching humans and other robots interacting.

Learning interaction protocols using Augmented Baysian Networks applied to guided navigation

Research in robot navigation usually concentrates on implementing navigation algorithms that allow the robot to navigate without human aid. In many real world situations, it is desirable that the robot is able to understand natural gestures from its user or partner and use this understanding to guide its navigation. Some algorithms already exist for learning natural gestures and/or their associated actions but most of these systems does not allow the robot to automatically generate the associated controller that allows it to actually navigate in the real environment. Furthermore, a technique is needed to combine the gestures/actions learned from interacting with multiple users or partners. This paper resolves these two issues and provides a complete system that allows the robot to learn interaction protocols and act upon them using only unsupervised learning techniques and enables it to combine the protocols learned from multiple users/partners. The proposed approach is general and can be applied to other interactive tasks as well. This paper also provides a real world experiment involving 18 subjects and 72 sessions that supports the ability of the proposed system to learn the needed gestures and to improve its knowledge of different gestures and their associations to actions over time.

Robust Singular Spectrum Transform

Change Point Discovery is a basic algorithm needed in many time series mining applications including rule discovery, motif discovery, casual analysis, etc. Several techniques for change point discovery have been suggested including wavelet analysis, cosine transforms, CUMSUM, and Singular Spectrum Transform. Of these methods Singular Spectrum Transform (SST) have received much attention because of its generality and because it does not require ad-hoc adjustment for every time series. In this paper we show that traditional SST suffers from two major problems: the need to specify five parameters and the rapid reduction in the specificity with increased noise levels. In this paper we define the Robust Singular Spectrum Transform (RSST) that alleviates both of these problems and compare it to RSST using different synthetic and real-world data series.

Using physiological signals to detect natural interactive behavior

Many researchers in the Human Robot Interaction (HRI) and Embodied Conversational Agents (ECA) domains try to build robots and agents that exhibit human-like behavior in real-world close encounter situations. One major requirement for comparing such robots and agents is to have an objective quantitative metric for measuring naturalness in various kinds of interactions. Some researchers have already suggested techniques for measuring stress level, awareness etc using physiological signals like Galvanic Skin Response (GSR) and Blood Volume Pulse (BVP). One problem of available techniques is that they are only tested with extreme situations and cannot according to the analysis provided in this paper distinguish the response of human subjects in natural interaction situations. One other problem of the available techniques is that most of them require calibration and some times ad-hoc adjustment for every subject. This paper explores the usefulness of various kinds of physiological signals and statistics in distinguishing natural and unnatural partner behavior in a close encounter situation. The paper also explores the usefulness of these statistics in various time slots of the interaction. Based on this analysis a regressor was designed to measure naturalness in close encounter situations and was evaluated using human-human and human-robot interactions and shown to achieve statistically significant distinction between natural and unnatural situations.

Toward Robots as Embodied Knowledge Media

We describe attempts to have robots behave as embodied knowledge media that will permit knowledge to be communicated through embodied interactions in the real world. The key issue here is to give robots the ability to associate interactions with information content while interacting with a communication partner. Toward this end, we present two contributions in this paper. The first concerns the formation and maintenance of joint intention, which is needed to sustain the communication of knowledge between humans and robots. We describe an architecture consisting of multiple layers that enables interaction with people at different speeds. We propose the use of an affordance-based method for fast interactions. For medium-speed interactions, we propose basing control on an entrainment mechanism. For slow interactions, we propose employing defeasible interaction patterns based on probabilistic reasoning. The second contribution is concerned with the design and implementation of a robot that can listen to a human instructor to elicit knowledge, and present the content of this knowledge to a person who needs it in an appropriate situation. In addition, we discuss future research agenda toward achieving robots serving as embodied knowledge media, and fit the robots-as-embodied-knowledge-media view in a larger perspective of Conversational Informatics.

Exact Discovery of Length-Range Motifs

Motif discovery is the problem of finding unknown patterns that appear frequently in real valued timeseries. Several approaches have been proposed to solve this problem with no a-priori knowledge of the timeseries or motif characteristics. MK algorithm is the de facto standard exact motif discovery algorithm but it can discover a single motif of a known length. In this paper, we argue that it is not trivial to extend this algorithm to handle multiple motifs of variable lengths when constraints of maximum overlap are to be satisfied which is the case in many real world applications. The paper proposes an extension of the MK algorithm called MK++ to handle these conditions. We compare this extensions with a recently proposed approximate solution and show that it is not only guaranteed to find the exact top pair-motifs but that it is also faster. The proposed algorithm is then applied to several real-world time series.

Toward combining autonomy and interactivity for social robots

The success of social robots in achieving natural coexistence with humans depends on both their level of autonomy and their interactive abilities. Although a lot of robotic architectures have been suggested and many researchers have focused on human–robot interaction, a robotic architecture that can effectively combine interactivity and autonomy is still unavailable. This paper contributes to the research efforts toward this architecture in the following ways. First a theoretical analysis is provided that leads to the notion of co-evolution between the agent and its environment and with other agents as the condition needed to combine both autonomy and interactivity. The analysis also shows that the basic competencies needed to achieve the required level of autonomy and the envisioned level of interactivity are similar but not the same. Secondly nine specific requirements are then formalized that should be achieved by the architecture. Thirdly a robotic architecture that tries to achieve those requirements by utilizing two main theoretical hypothesis and several insights from social science, developmental psychology and neuroscience is detailed. Lastly two experiments with a humanoid robot and a simulated agent are reported to show the potential of the proposed architecture.

Human adaptation to a miniature robot: Precursors of mutual adaptation

Mutual adaptation is an important phenomenon in human-human communications. Traditionally HRI research was more interested in investigating adaptation of the robot to the human using machine learning techniques but the possibility of utilizing the natural ability of humans to adapt to other humans and artifacts including robots is recently becoming increasingly attractive. This paper presents some of the results from an experiment conducted to investigate the interaction patterns and effectiveness of motion cues as a feedback modality between a human operator and a miniature robot in a confined collaborative navigation task. The results presented in this paper show evidence of human adaptation to the robot and moreover suggest that the adaptation rate is not constant or continuous in time but is discontinuous and nonlinear. The results also show evidence of a starting exploration stage before the adaptation with duration dependent on the expectations of the human regarding the capabilities of the robot in the given task. The paper investigates how to utilize these and related findings for building robots not only capable of adapting to human operators but can also help those operators adapt to them.

The H 3 R Explanation Corpus human-human and base human-robot interaction dataset

Natural interaction between humanoid robots and humans is one of the major goals of the HRI field. Two major requirements for advancing this direction of research are the availability of human-human and base human-robot interaction datasets for training and evaluation purposes and the availability of general agreed upon objective metrics for judging the performance of proposed robots and algorithms. In this paper we report details of the H3R explanation corpus dataset of human-human and base human-robot interactions in assembly/disassembly explanation scenarios that combines five kinds of data: video, audio, motion tracking, subjective, and physiological data. 44 subjects and 66 sessions were conducted during this experiment. The corpus contains 22 natural human-human interactions, 22 un-natural human-human interactions, and 22 baseline human-robot interactions. To our best knowledge this is the first database that combines these five data types and three types of interactions. The paper also reports the first usage of this explanation corpus to compare subjective and physiological evaluations of various dimensions of listenerpsilas behavior.