Stanislao Lauria

Mobile robot programming using natural language

Abstract How will naive users program domestic robots? This paper describes the design of a practical system that uses natural language to teach a vision-based robot how to navigate in a miniature town. To enable unconstrained speech the robot is provided with a set of primitive procedures derived from a corpus of route instructions. When the user refers to a route that is not known to the robot, the system will learn it by combining primitives as instructed by the user. This paper describes the components of the Instruction-Based Learning architecture and discusses issues of knowledge representation, the selection of primitives and the conversion of natural language into robot-understandable procedures.

Training personal robots using natural language instruction

As domestic robots become pervasive, uninitiated users will need a way to instruct them to adapt to their particular needs. The authors are designing a practical system that uses natural language to instruct a vision-based robot.

Vision-based urban navigation procedures for verbally instructed robots

Abstract When humans explain a task to be executed by a robot they decompose it into chunks of actions. These form a chain of search-and-act sensory-motor loops that exit when a condition is met. In this paper we investigate the nature of these chunks in an urban visual navigation context, and propose a method for implementing the corresponding robot primitives such as “take the nth turn right/left”. These primitives make use of a “short-lived” internal map updated as the robot moves along. The recognition and localisation of intersections is done in the map using task-guided template matching. This approach takes advantage of the content of human instructions to save computation time and improve robustness.

Converting natural language route instructions into robot executable procedures

Humans explaining a task to a robot use chunks of actions that are often complex procedures for robots. An instructable robot needs to be able to map such chunks to existing pre-programmed primitives. We investigate an architecture used in spoken dialogue systems that is able to extract executable robot procedures from user instructions. A suitable representation of route instructions is introduced, then a Procedure Specification Language (PSL) is described that allows to extract from the semantic representation of the dialogue both the robot executable procedures and their parameters.

Teaching Introductory Programming: A Quantitative Evaluation of Different Approaches

Teaching programming to beginners is a complex task. In this article, the effects of three factors—choice of programming language, problem-solving training, and the use of formative assessment—on learning to program were investigated. The study adopted an iterative methodological approach carried out across 4 consecutive years. To evaluate the effects of each factor (implemented as a single change in each iteration) on students’ learning performance, the study used quantitative, objective metrics. The findings revealed that using a syntactically simple language (Python) instead of a more complex one (Java) facilitated students’ learning of programming concepts. Moreover, teaching problem solving before programming yielded significant improvements in student performance. These two factors were found to have variable effects on the acquisition of basic programming concepts. Finally, it was observed that effective formative feedback in the context of introductory programming depends on multiple parameters. The article discusses the implications of these findings, identifies avenues for further research, and argues for the importance of studies in computer science education anchored on sound research methodologies to produce generalizable results.

Exploring Miscommunication and Collaborative Behaviour in Human-Robot Interaction

This paper presents the first step in designing a speech-enabled robot that is capable of natural management of miscommunication. It describes the methods and results of two WOz studies, in which dyads of naive participants interacted in a collaborative task. The first WOz study explored human miscommunication management. The second study investigated how shared visual space and monitoring shape the processes of feedback and communication in task-oriented interactions. The results provide insights for the development of human-inspired and robust natural language interfaces in robots.

Mobile robot localization using robust extended H∞ filtering

Abstract In this paper, a novel methodology is provided for accurate localization of a mobile robot using autonomous navigation based on internal and external sensors. A new robust extended H∞ filter is developed to deal with the non-linear kinematic model of the robot and the non-linear distance measurements, together with process and measurement noises. The proposed filter relies on a two-step prediction-correction structure, which is similar to a Kalman filter. Simulations are provided to demonstrate the effectiveness of the proposed method.

Do and Say as I Say: Linguistic Adaptation in Human–Computer Dialogs

There is strong research evidence showing that people naturally align to each other’s vocabulary, sentence structure, and acoustic features in dialog, yet little is known about how the alignment mechanism operates in the interaction between users and computer systems let alone how it may be exploited to improve the efficiency of the interaction. This article provides an account of lexical alignment in human–computer dialogs, based on empirical data collected in a simulated human–computer interaction scenario. The results indicate that alignment is present, resulting in the gradual reduction and stabilization of the vocabulary-in-use, and that it is also reciprocal. Further, the results suggest that when system and user errors occur, the development of alignment is temporarily disrupted and users tend to introduce novel words to the dialog. The results also indicate that alignment in human–computer interaction may have a strong strategic component and is used as a resource to compensate for less optimal (visually impoverished) interaction conditions. Moreover, lower alignment is associated with less successful interaction, as measured by user perceptions. The article distills the results of the study into design recommendations for human–computer dialog systems and uses them to outline a model of dialog management that supports and exploits alignment through mechanisms for in-use adaptation of the system’s grammar and lexicon. Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/hhci.

Are we there yet?: The role of gender on the effectiveness and efficiency of user-robot communication in navigational tasks

Many studies have identified gender differences in communication related to spatial navigation in real and virtual worlds. Most of this research has focused on single-party communication (monologs), such as the way in which individuals either give or follow route instructions. However, very little work has been reported on spatial navigation dialogs and whether there are gender differences in the way that they are conducted. This article will address the lack of research evidence by exploring the dialogs between partners of the same and of different gender in a simulated Human-Robot Interaction study. In the experiments discussed in this article, pairs of participants communicated remotely; in each pair, one participant (the instructor) was under the impression that s/he was giving route instructions to a robot (the follower), avoiding any perception of gendered communication. To ensure the naturalness of the interaction, the followers were given no guidelines on what to say, however, each had to control a robot based on the users instructions. While many monologe-based studies suggest male superiority in a multitude of spatial activities and domains, this study of dialogs highlights a more complex pattern of results. As anticipated, gender influences task performance and communication. However, the findings suggest that it is the interaction—the combination of gender and role (i.e., instructor or follower)—that has the most significant impact. In particular, pairs of female users/instructors and male “robots”/followers are associated with the fastest and most accurate completion of the navigation tasks. Moreover, dialoge-based analysis illustrates how pairs of male users/instructors and female “robots”/followers achieved successful communication through “alignment” of spatial descriptions. In particular, males seem to adapt the content of their instructions when interacting with female “robots”/followers and employ more landmark references compared to female users/instructors or when addressing males (in male-male pairings). This study describes the differences in how males and females interact with the system, and proposes that any female “disadvantage” in spatial communication can disappear through interactive mechanisms. Such insights are important for the design of navigation systems that are equally effective for users of either gender.

Robot Navigation Control Based on Monocular Images: An Image Processing Algorithm for Obstacle Avoidance Decisions

This paper covers the use of monocular vision to control autonomous navigation for a robot in a dynamically changing environment. The solution focused on using colour segmentation against a selected floor plane to distinctly separate obstacles from traversable space: this is then supplemented with canny edge detection to separate similarly coloured boundaries to the floor plane. The resulting binary map (where white identifies an obstacle-free area and black identifies an obstacle) could then be processed by fuzzy logic or neural networks to control the robot’s next movements. Findings show that the algorithm performed strongly on solid coloured carpets, wooden, and concrete floors but had difficulty in separating colours in multicoloured floor types such as patterned carpets.