Daqing Yi

Toward Task-Based Mental Models of Human-Robot Teaming: A Bayesian Approach

We consider a set of team-based information tasks, meaning that the team’s goals are to choose behaviors that provide or enhance information available to the team. These information tasks occur across a region of space and must be performed for a period of time. We present a Bayesian model for (a) how information flows in the world and (b) how information is altered in the world by the location and perceptions of both humans and robots. Building from this model, we specify the requirements for a robot’s computational mental model of the task and the human teammate, including the need to understand where and how the human processes information in the world. The robot can use this mental model to select its behaviors to support the team objective, subject to a set of mission constraints.

Homotopy-Aware RRT*: Toward Human-Robot Topological Path-Planning

An important problem in human-robot interaction is for a human to be able to tell the robot go to a particular location with instructions on how to get there or what to avoid on the way. This paper provides a solution to problems where the human wants the robot not only to optimize some objective but also to honor “soft” or “hard” topological constraints, i.e. “go quickly from A to B while avoiding C”. The paper presents the HARRT* (homotopy-aware RRT*) algorithm, which is a computationally scalable algorithm that a robot can use to plan optimal paths subject to the information provided by the human. The paper provides a theoretic justification for the key property of the algorithm, proposes a heuristic for RRT*, and uses a set of simulation case studies of the resulting algorithm to make a case for why these properties are compatible with the requirements of human-robot interactive path-planning.

Supporting task-oriented collaboration in human-robot teams using semantic-based path planning

Improvements in robot autonomy are changing the human-robot interaction from low-level manipulation to high-level task-based collaboration. For a task-oriented collaboration, a human assigns sub-tasks to robot team members. In this paper, we consider task-oriented collaboration of humans and robots in a cordon and search problem. We focus on a path-planning framework with natural language input. By the semantic elements in a shared mental model, a natural language command can be converted into optimization objectives. We import multi-objective optimization to facilitate modeling the “adverb” elements in natural language commands. Finally, human interactions are involved in the optimization search process in order to guarantee that the found solution correctly reflects the human’s intent.

Expressing homotopic requirements for mobile robot navigation through natural language instructions

Allowing a human to express topological requirements to a robot in language enables untrained users to guide robot movement without requiring the human to understand sophisticated robot algorithms. By using a homotopy class or classes to represent one or more topological requirements, we build a framework that helps a robot understand a humans intent. This paper reviews a homotopic decomposition method that is used to convert any path into a string, which allows homotopic path equivalence to be performed by comparing strings. We then integrate the Homotopic Distributed Correspondence Graph (HoDCG) to infer the homotopic constraint in the format of strings from a language instruction. Finally, we use a homotopic path-planning algorithm that finds the optimal paths for a given objective and homotopic constraint. Experiment results show how a language instruction is converted into a path driven by an implicit topological requirement.

Interactive multi-objective path planning through a palette-based user interface

n a problem where a human uses supervisory control to manage robot path-planning, there are times when human does the path planning, and if satisfied commits those paths to be executed by the robot, and the robot executes that plan. In planning a path, the robot often uses an optimization algorithm that maximizes or minimizes an objective. When a human is assigned the task of path planning for robot, the human may care about multiple objectives. This work proposes a graphical user interface (GUI) designed for interactive robot path-planning when an operator may prefer one objective over others or care about how multiple objectives are traded off. The GUI represents multiple objectives using the metaphor of an artist’s palette. A distinct color is used to represent each objective, and tradeoffs among objectives are balanced in a manner that an artist mixes colors to get the desired shade of color. Thus, human intent is analogous to the artist’s shade of color. We call the GUI an “Adverb Palette” where the word “Adverb” represents a specific type of objective for the path, such as the adverbs “quickly” and “safely” in the commands: “travel the path quickly”, “make the journey safely”. The novel interactive interface provides the user an opportunity to evaluate various alternatives (that tradeoff between different objectives) by allowing her to visualize the instantaneous outcomes that result from her actions on the interface. In addition to assisting analysis of various solutions given by an optimization algorithm, the palette has additional feature of allowing the user to define and visualize her own paths, by means of waypoints (guiding locations) thereby spanning variety for planning. The goal of the Adverb Palette is thus to provide a way for the user and robot to find an acceptable solution even though they use very different representations of the problem. Subjective evaluations suggest that even non-experts in robotics can carry out the planning tasks with a great deal of flexibility using the adverb palette.

Enhancement of image luminance resolution by imposing random jitter

Inspired by biological eyes, silicon retinas with pixel-level processing have been developed to achieve very high-speed and high-quality image processing. Due to the limitation on the fill factor and the dimension of a silicon chip, both spatial and luminance resolutions have to be kept low. For recovering fine images from a silicon retina with a lower resolution, the authors propose a neural network model and its electronic counterpart by imposing random jitter to the sensor and collecting temporal statistics of the firing neurons. Statistical analysis shows that the scheme can enhance resolution of an image and emphasize contrast edges present in the image. It is further proved that the enhancement in luminance resolution and sharpness is a trade-off between recovering bias and variance. Therefore, jitter intensity needs to be optimized by considering the luminance distribution. The simulations illustrate its effect on the fine detail reconstruction using the proposed scheme.

Adverb Palette: GUI-based Support for Human Interaction in Multi-Objective Path-Planning

Many fields of study involve decision-making where trade-offs between different desirable outcomes are to be made. This paper introduces four designs for a graphical user interface (GUI) called the Adverb Palette (AP) to support interactive robot path-planning under multiple performance objectives. The GUI designs apply when the robot must go from an initial configuration to a goal configuration while honoring competing objectives. This novel human-robot interface helps the operator issue commands to the robot to take a specific path from the many available paths, thereby helping the user in decision making process. AP is a metaphor that symbolizes an artists palette where an artist mixes different colors to get a desired shade of color. For the principal GUI designs, each objective is represented as a color. Since we assume that the human specifies the objectives using language, each objective is metaphorically called an adverb. Two of the approaches, the sliders and the waypoints indicator, have been explored by the researchers in the past, and the other two, palette and prism are novel and may be more useful and intuitive.

Iterative Learning Control for Visual Servoing with Unknown Homography Matrix

An iterative learning control scheme for robot planar motion visual servo with an arbitrarily mounted camera is presented in the paper. In our previous work, it has been proved that, when the image plane and the motion plane are parallel with a constant but unknown image Jacobian matrix, an iterative learning control law with a Nussbaum learning gain can be applied to visual servoing for trajectory tracking without camera calibration. How to design a visual servoing controller for an arbitrarily mounted camera, in which the image Jacobian matrix is shown to be time varying, remains to be an open problem. By utilizing the features of planar homography, the time-varying image Jacobian can be factorized as a constant transformation matrix and a time varying depth. As the time varying depth is bounded, an iterative learning control with a Nussbaum gain is proved to be convergent for planar trajectory tracking under control of an arbitrarily mounted camera.

Input-to-State Stability Analysis on Particle Swarm Optimization

This paper examines the dynamics of particle swarm optimization (PSO) by modeling PSO as a feedback cascade system and then applying input-to-state stability analysis. Using a feedback cascade system model we can include the effects of the global-best and personal-best values more directly in the model of the dynamics. Thus in contrast to previous study of PSO dynamics, the input-to-state stability property used here allows for the analysis of PSO both before and at stagnation. In addition, the use of input-to-state stability allows this analysis to preserve random terms which were heretofore simplified to constants. This analysis is important because it can inform the setting of PSO parameters and better characterize the nature of PSO as a dynamic system. This work also illuminates the way in which the personal-best and the global-best updates influence the bound on the particles position and hence, how the algorithm exploits and explores the fitness landscape as a function of the personal best and global best.

A Simple Neural Network for Enhancement of Image Acuity by Fixational Instability

Inspired by biological findings, this paper proposes a neural network model for achieving higher image acuity by introducing random eye movement. Statistical analysis and comparison study of the image quality in the presence and absence of random eye movement are carried out using the model. It is revealed that, as a noise source to a stationary image, the random eye movement can contribute to overcome the inherent resolution limits of photoreceptors and enhance sharpness of images by temporal statistics of firing neurons. Super-resolution and prominent edges can thus be achieved, with superior visual acuity to the absence of eye-movement. The acuity enhancement is in fact a trade-off between bias and variance and is related to the distribution of visual stimuli and eye-movement patterns. The simulations illustrate its effect on enhancement of image acuity.