David Kortenkamp

User Interaction with Multi-Robot Systems

There has been very little research on multiple human users interacting with multiple autonomous robots. In this paper we present some of the requirements of such user interaction. We present a prototype architecture for collaborative interaction. This architecture is put into the context of multiple space robots monitoring a space structure to assist human crew members.

Developing and Executing Goal-Based, Adjustably Autonomous Procedures

This paper describes an approach to representing, authoring and executing procedures during human spaceflight missions. The approach allows for the explicit incorporation of goals into procedures. The approach also allows for adjustably autonomous execution of procedures. That is, the procedure can be executed by a computer, by a human or in any combination of human and computer. A new procedure representation is described as are tools for authoring procedures in the new representation. An execution engine interprets the representation and executes it appropriately. An end-user procedure display provides guidance to the human as to execution status. Experiments were performed using actual International Space Station procedures being executed against a high-fidelity space station simulation. The new approach increases the efficiency of crew and grou nd controllers while executing procedures and reduces errors in procedure execution.

Trusted Autonomy for Space Flight Systems

NASA has long supported research on intelligent control technologies that could allow space systems to operate autonomously or with reduced human supervision. Proposed uses range from automated control of entire space vehicles to mobile robots that assist or substitute for astronauts to vehicle systems such as life support that interact with other systems in complex ways and require constant vigilance. The potential for pervasive use of such technology to extend the kinds of missions that are possible in practice is well understood, as is its potential to radically improve the robustness, safety and productivity of diverse mission systems. Despite its acknowledged potential, intelligent control capabilities are rarely used in space flight systems. Perhaps the most famous example of intelligent control on a spacecraft is the Remote Agent system flown on the Deep Space One mission (1998 - 2001). However, even in this case, the role of the intelligent control element, originally intended to have full control of the spacecraft for the duration of the mission, was reduced to having partial control for a two-week non-critical period. Even this level of mission acceptance was exceptional. In most cases, mission managers consider intelligent control systems an unacceptable source of risk and elect not to fly them. Overall, the technology is not trusted. From the standpoint of those who need to decide whether to incorporate this technology, lack of trust is easy to understand. Intelligent high-level control means allowing software io make decisions that are too complex for conventional software. The decision-making behavior of these systems is often hard to understand and inspect, and thus hard to evaluate. Moreover, such software is typically designed and implemented either as a research product or custom-built for a particular mission. In the former case, software quality is unlikely to be adequate for flight qualification and the functionality provided by the system is likely driven largely by the need to publish innovative work. In the latter case, the mission represents the first use of the system, a risky proposition even for relatively simple software.

Embedding planning technology into satellite systems

Satellites will need increasing amounts of autonomy in order to maximize their mission capabilities even in the face of events that may disrupt their systems. This paper describes an on-board planning and execution system for satellites that schedules system tasks and responds to real-time events. The system consists of a mission planner that schedules nominal activities, a threat response planner that schedules actions to mediate external threats to the satellite and its mission, and an executive that takes the resulting plan and commands the satellite subsystems. A state-based simulation of a satellite system was developed and used to demonstrate the planning and execution system. I. Introduction There is an increasing need to develop on-board autonomy for satellite systems, both to increase their productivity and to protect them from hazards and threats such as component faults, approaching space debris, and dangerous space weather. We are developing an integrated system that demonstrates solutions to many of the challenges inherent in developing embedded planning systems for satellites. The Highly Autonomous Mission Manager for Event Response (HAMMER) system is designed to allow a satellite to operate and respond to threats even when it is not in communication with the ground or when time constraints require immediate response to threats. The HAMMER system attempts to meet mission objectives even in the face of threats. HAMMER prioritizes multiple, competing user goals and requests and determines an optimal ordering of satellite tasks to conserve resources and maximize capability. End user goals and requests are expected to come to the satellite asynchronously as the satellite is operating. Thus, new task schedules will need to be generated on-the-fly. Threats are also expected occur asynchronously and require on-the-fly replanning to counter the threats and still attempt to meet mission objectives. User requests will be at a high level (e.g., take an image of location X by time Y and download to location Z) and will need to be turned into a detailed plan of low-level satellite actions. The tight coupling between end user goals, mission planning, threat response, and task execution is a key challenge for these systems. HAMMER integrates an on-board execution system with two different on-board planning and scheduling systems: a Mission Planner (MP) that plans optimal sequences of actions to achieve mission goals, and a Threat Response Planner (TRP) that refines the mission plan with pre-planned responses to threats. The benefits of the HAMMER system are that it: 1) can receive high-level end user goals and produce optimal satellite plans; 2) can respond to threats without ground intervention; 3) is scalable and reusable because the core components are model-driven.

Electronic Procedures for Medical Operations in Space

Electronic medical procedures are being used aboard the International Space Station. These XML-based procedures provide the ability to link in multi-modal information that offers the astronaut more information than the original paper counter-parts. Writing XMLbased procedures is difficult using traditional word processing systems. Instead, an easy to use drag and drop editor, called PrIDE, was developed so that physicians themselves, rather than procedure writers, are able to develop procedures more quickly and consistently than has been done in the past. We are now realizing many of the other benefits of these XMLbased procedures. I. Introduction STRONAUTS currently use the International Procedure Viewer (IPV) to view operational and medical procedures while in space. Medical procedures were the most recent addition to the collection of documents viewable by IPV and we are just now taking advantage of the benefits of modeling medical procedures in IPV’s XML format. Prior to using XML, medical procedures were written in Microsoft Word. But due to a redesign of the medical kits that were to be used in space, many of the medical procedures needed major overhauls. This led to the desire to rewrite them so they could be viewable using IPV. However, writing procedures directly in IPV’s XML format is not feasible. For operational procedures, there were a limited number of experts who used a licensed commercial tool and their knowledge of XML to convert Word documents to IPV’s XML format. The procedure author would create the initial procedure in Word, then the XML expert would translate that to XML. After making the translation, they would return the IPV documents to the procedure author and they would go through one or more rounds of negotiations to make the procedure look as close to the Word document as possible. When writing the medical procedures, the authors are expected to follow the Operations Data File (ODF) standards. The ODF is a one-hundred-and-fifty-page document that specifies all of the formatting standards for NASA documents that are to be used aboard the International Space Station (ISS). It includes details such as how far to tab over specific items, when to italicize, bold or underline, and a large list of symbols that take the place of words in commanding sequences. The procedure author bears the burden of ensuring that the document conforms to this large volume of standards. This takes a significant amount of time. And since Word is a free style word processing system and the authors are human, some medical procedures conformed to these standards better than others. Another issue that came up was in the maintenance of procedures. It was common for a medical kit to be removed or updated for a specific flight. When this would happen, it would affect all of the procedures that called out a piece of hardware from the affected medical kit. It was a manual process to respond to this. The authors of the various procedures would be notified of the update, and they would have to remember which procedures that might be affected. It was a similar situation when a procedure was deleted or renamed. The process relied on the memories

A Testbed for Evaluating Lunar Habitat Autonomy Architectures

A lunar outpost will involve a habitat with an integrated set of hardware and software that will maintain a safe environment for human activities. There is a desire for a paradigm shift whereby crew will be the primary mission operators, not ground controllers. There will also be significant periods when the outpost is uncrewed. This will require that significant automation software be resident in the habitat to maintain all system functions and respond to faults. JSC is developing a testbed to allow for early testing and evaluation of different autonomy architectures. This will allow evaluation of different software configurations in order to: 1) understand different operational concepts; 2) assess the impact of failures and perturbations on the system; and 3) mitigate software and hardware integration risks. The testbed will provide an environment in which habitat hardware simulations can interact with autonomous control software. Faults can be injected into the simulations and different mission scenari...

Managing Concurrent Activities of Humans and Software Agents

*† ‡ § For future manned space exploration, it will be necessary for humans, robots, and autonomous software agents to work together in an often harsh environment with constrained resources. Such concurrent distributed multi-human multi-agent operations will require accommodation by agents to prevent interference with others. We are integrating two complementary approaches for managing the concurrent activities of humans and software agents. The first approach is to automatically generate a plan that coordinates the activities of humans and autonomous agents. The second approach is to provide command and authorization services that prevent activities not in this plan from interfering. Both of these approaches are enabled by providing agents that support adjusting their level of control autonomy as a means of preventing commands from interfering. We have demonstrated both approaches for controlling crew life support systems at NASA Johnson Space Center. In this paper we describe these approaches for managing concurrent activities of humans and agents, and illustrate them with examples from our work with crew life support systems at NASA Johnson Space Center.