Alvar Saenz-Otero

Relative Computer Vision-Based Navigation for Small Inspection Spacecraft

W HEN a large spacecraft fails on orbit, the diagnostic and debugging process is challenging, time consuming, costly, and possibly dangerous. The ability to visually inspect a large spacecraft can be helpful because it provides a reliable source of independent data on the large spacecraft’s current state. Recent proposals by Henshaw et al. [1] have discussed launching large spacecraft with a small inspector satellite that is attached to the host. This inspector satellite could be deployed if an on-orbit anomaly were to occur. Henshaw et al. argued that, for this approach to be successful, the inspector spacecraft must minimize the requirements that are levied on the larger host spacecraft. Henshaw et al.’s approach implies that neither communications nor power systems should be sharedwith the host, and that a passive vision-based navigation system should be used with a single, small fiducial marker. It was also noted that it is important for the inspector to minimize the risk of colliding with the host spacecraft and causing further damage. This leads to an important design question: How should the relative state of the inspector be estimated in an accurate and reliable fashion? This Note presents a design for a relative state estimator based on a small fiducial target that is observed by a singlemonochrome camera. This design solves the exterior orientation problem [2] for four point features using a globally convergent nonlinear iterative algorithm. Note that four coplanar points are theminimum number of points that are required to solve for the relative posewithout ambiguity assuming perspective projection [3]. The relative pose is then filtered with the dynamics to estimate the 12-degrees-of-freedom relative state (i.e., three-dimensional position, orientation, and linear and angular velocity). This algorithm was tested on the Synchronized Position, Hold, Engage, and Reorient Experimental Satellites (SPHERES) system [4] with a computer vision upgrade [5], and a summary of these results are presented in this Note. This Note is an expanded version of earlier publications by the first author [6,7]. II. Literature Review

The SPHERES Guest Scientist Program: collaborative science on the ISS

The SPHERES Guest Scientist Program (GSP) supports the efforts of geographically distributed researchers at MIT, the U.S. Department of Defense, NASA, and elsewhere, in the development of algorithms for the SPHERES formation-flying and docking testbed. The GSP consists of a test development framework, a robust and flexible interface to the SPHERES flight software, a portable high-fidelity simulation, two laboratory testbeds, and data analysis utilities. The SPHERES testbed will be operated in bi-weekly test sessions on-board the International Space Station. Updates to the flight software can be uploaded immediately prior to each test session, allowing guest scientists the opportunity to revise and improve their algorithms from one session to the next. The SPHERES flight software architecture and the GSP interface design contribute to the flexibility of the testbed, and minimize nonproductive labor by simplifying algorithm implementation.

SPHERES: a platform for formation-flight research

New space missions, such as the Terrestrial Planet Finder (TPF) and Darwin programs, call for the use of spacecraft which maintain precise formation to achieve the effective aperture of a much larger spacecraft. Achieving this requires the development of several new space technologies. The SPHERES program was specifically designed to develop a wide range of algorithms in support of formation flight systems. Specifically, SPHERES allows the incremental development of metrology, control, autonomy, artificial intelligence, and communications algorithms. To achieve this, SPHERES exhibits a wide array of features to 1) facilitate the iterative research process, 2) support experiments, 3) support multiple scientists, and 4) enable reconfiguration and modularity. The effectiveness of these aspects of the facility have been demonstrated by several programs including development of system identification routines, coarse formation flight control algorithms, and demonstration of tethered systems.

SPHERES Zero Robotics software development: Lessons on crowdsourcing and collaborative competition

Crowdsourcing is the art of constructively organizing crowds of people to work toward a common objective. Collaborative competition is a specific kind of crowdsourcing that can be used for problems that require a collaborative or cooperative effort to be successful, but also use competition as a motivator for participation or performance. The DARPA InSPIRE program is using crowdsourcing to develop spaceflight software for small satellites under a sub-program called SPHERES Zero Robotics - a space robotics programming competition. The robots are miniature satellites, called SPHERES, that operate inside the International Space Station (ISS). The idea is to allow thousands of amateur participants to program using the SPHERES simulator and eventually test their algorithms in microgravity. The entire software framework for the program, to provide the ability for thousands to collaboratively use the SPHERES simulator and create algorithms, is also built by crowdsourcing. This paper describes the process of building the software framework for crowdsourcing SPHERES development in collaboration with a commercial crowdsourcing company called TopCoder. It discusses the applicability of crowdsourcing and collaborative competition in the design of the Zero Robotics software infrastructure, metrics of success and achievement of objectives.

Factor Graph Modeling of Rigid-body Dynamics for Localization, Mapping, and Parameter Estimation of a Spinning Object in Space

This paper presents a new approach for solving the simultaneous localization and mapping problem for inspecting an unknown and uncooperative object that is spinning about an arbitrary axis in space. This approach probabilistically models the six degree-of-freedom rigid-body dynamics in a factor graph formulation. Using the incremental smoothing and mapping system, this method estimates a feature-based map of the target object, as well as this objects position, orientation, linear velocity, angular velocity, center of mass, principal axes, and ratios of inertia. This solves an important problem for spacecraft proximity operations. Additionally, it provides a generic framework for incorporating rigid-body dynamics that may be applied to a number of other terrestrial-based applications. To evaluate this approach, the Synchronized Position Hold Engage Reorient Experimental Satellites SPHERES were used as a testbed within the microgravity environment of the International Space Station. The SPHERES satellites, using body-mounted stereo cameras, captured a dataset of a target object that was spinning at ten rotations per minute about its unstable, intermediate axis. This dataset was used to experimentally evaluate the approach described in this paper, and it showed that it was able to estimate a geometric map and the position, orientation, linear and angular velocities, center of mass, and ratios of inertia of the target object.

Flight Results of Vision-Based Navigation for Autonomous Spacecraft Inspection of Unknown Objects

This paper describes a vision-based relative navigation and control strategy for inspecting an unknown, noncooperative, and possibly spinning object in space using a visual–inertial system that is designed to minimize the computational requirements while maintaining a safe relative distance. The proposed spacecraft inspection system relies solely on a calibrated stereo camera and a three-axis gyroscope to maintain a safe inspection distance while following a circular trajectory around the object. The navigation system is based on image processing algorithms, which extract the relative position and velocity between the inspector and the object, and a simple control approach is used to ensure that the desired range and bearing are maintained throughout the inspection maneuver. The hardware implementation details of the system are provided. Computer simulation results and experiments conducted aboard the International Space Station during Expedition 34 are reported to demonstrate the performance and applicab...

Simple Adaptive Control for Spacecraft Proximity Operations

This paper addresses the problem of adaptive output feedback control for spacecraft proximity operations under parametric uncertainties and unknown disturbances. Control laws using the simple adaptive control theory, which is based on the so-called model reference adaptive control approach, are derived. In the first control scheme, a position feedback adaptive control law employing a parallel feedforward configuration to satisfy sufficient conditions guaranteeing closed-loop stability is developed. Then, it is shown how the performance of this adaptive controller can be significantly improved by using a position-plus-velocity feedback adaptive control strategy. Simulations compare the performance of the adaptive controllers with a fixed gain proportional-derivative controller. Obtained results demonstrate that both simple adaptive control methodologies yield improved performance, regardless of an uncertainty in the spacecraft mass and an unknown external perturbation, when compared to the linear-time invariant benchmark controller. In addition, the position-plus-velocity adaptive feedback methodology is shown to greatly reduce the required control input force, making its implementation onboard nanosatellites feasible. Finally, experiments conducted at the Massachusetts Institute of Technology’s Synchronized Position Hold Engage Reorient Experimental Satellites research facility are reported and discussed.

SPHERES interact—Human–machine interaction aboard the International Space Station

The deployment of space robots for servicing and maintenance operations that are teleoperated from the ground is a valuable addition to existing autonomous systems, because it will provide flexibility and robustness in mission operations. In this connection, not only robotic manipulators are of great use, but also free-flying inspector satellites supporting the operations through additional feedback to the ground operator. The manual control of such an inspector satellite at a remote location is challenging, because navigation in three-dimensional space is unfamiliar and large time delays can occur in the communication channel. This paper shows a series of robotic experiments, in which free flyers are controlled by astronauts aboard the International Space Station (ISS). The Synchronized Position Hold Engage Reorient Experimental Satellites (SPHERES) were utilized to study several aspects of a remotely controlled inspector satellite. The focus in this case study is investigating different approaches to human–spacecraft interaction with varying levels of autonomy under zero-gravity conditions.

Development and demonstration of an autonomous collision avoidance algorithm aboard the ISS

Since 2006 the SPHERES facility aboard the International Space Station has enabled research of high-risk autonomy algorithms which would not otherwise be conducted in a regular space mission. A thread of research with several demonstrations aboard the ISS is Collision Avoidance. Through two SPHERES test sessions over the course of a year, researchers have developed an efficient autonomous collision avoidance controller, deployed it on a representative microprocessor, and demonstrated its effectiveness as a reliable low-level safetey routine.

Passivity-Based Adaptive Control of Robotic Spacecraft for Proximity Operations Under Uncertainties

T HE Defense Advanced Research Projects Agency (DARPA) initiated the Phoenix Program, a venture to build on the success of the Japan Aerospace Exploration Agency’s Engineering Test SatelliteVII [1] andBoeing’sOrbital Express [2] proximity operation demonstration missions. One of the objectives of this program is to develop and demonstrate a new class of small modular satlets, similar to nanosatellites, capable of assisting in the retrieval of valuable components from depleted satellites in geosynchronous orbits for use in the assembly of new space systems. As part of the first demonstration mission under this program, the free-flyer robots will be required to manipulate a variety of components of different mass, such as space apertures and antennas. In this context, the relative position control systems must be robust to uncertainties in the freeflyer robot mass to ensure consistent and safe relative motion control with minimal trajectory overshoots. In close-proximity operations, the spacecraft would typically be close enough to employ the Clohessy–Wiltshire equations [3] to model the relative dynamics. Furthermore, when the time scale is significantly less than the orbital period of the target spacecraft, and the distance between both vehicles is limited to a fewmeters, a simple double-integrator model can be applied [4]. Being linear, these dynamics representations are particularly attractive for control purposes, which is why the literature proposes numerous linear continuous control laws to regulate or track a prescribed relative trajectory. However, most existing continuous control techniques for proximity operations are model-based and can only achieve good tracking performance when substantial information of the plant mathematical model and its dynamics parameters is available (e.g., mass and inertia). If the dynamics parameters are uncertain or there are unmodeled dynamics effects acting on the plant, model-based control approaches could perform inadequately. Dynamics uncertainties may arise when neglecting nonlinear dynamics effects or external perturbations in the plant model, whereas parameter uncertainties may arise from mass-inertia properties that vary due to fuel consumption, solar array deployment, payload variation, or, in the case of DARPA’s Phoenix, when the free-flyer servicer robot harvests a component of unknown mass from the target spacecraft. One way to manage both dynamics and parameter uncertainties is to employ indirect adaptive control techniques, which can, in realtime, identify the unknown plant parameters and external perturbations from which the control gains are obtained using an automatic design procedure. However, this class of adaptive control methodologies requires accurate information about the plant dynamics model [5]. For example, de Queiroz et al. [6] proposed an indirect adaptive control strategy, in which an adaptation law based on precise knowledge of the dynamics model identifies the unknown mass, which is then used explicitly in their control law. An adverse consequence of such identification procedures is the increased computational burden associated with real-time estimation of unknown parameters and dynamics effects. This could rule out using such indirect adaptive controllers for small space robot applications with limited computational resources. Alternatively, direct adaptive control techniques, with controller gains updated directly without requiring estimates of unknown plant parameters or mathematical models of the system to be controlled, can be used to address this problem. Based on our knowledge, no previous work investigated the use of direct adaptive control techniques for this application. In view of the preceding, this Note addresses the problem of continuous relative motion control under large uncertainties in the plant parameter, dynamics uncertainties, and unmodeled external perturbations, through the development of an output feedback direct adaptive control law. More specifically, this Note proposes a passivity-based adaptive control strategy derived upon the simple adaptive control (SAC) theory [7]. The stability of the passivity-based adaptive system is guaranteed through the Lyapunov direct method [8] and LaSalle’s invariance principle for nonautonomous systems [7,9–11] arguments, by applying the almost strictly passive conditions. These conditions are herein demonstrated to be satisfied by modeling both the Clohessy–Wiltshire and the double-integrator relative dynamics models as square linear time-invariant systems with a scaledposition-to-velocity output matrix. In addition, based on recent development in the area of nonlinear stability [12], this Note clarifies the use of LaSalle’s invariance principle (as opposed to the widely used Barbalat’s lemma [8]) for this particular problem, where the Lyapunov derivative function is negative-semidefinite. These two aspects correspond to the original theoretical contributions of this work. Finally, most previous work in the area of close-proximity operations only assessed the performance of the controllers in numerical simulations. In this context, another original contribution of this Note is to validate the passivity-based adaptive controller against a nonadaptive proportional-derivative (PD) controller, through experiments at the Massachusetts Institute of Technology’s Synchronized Position Hold Engage Reorient Experimental Satellites (SPHERES) research facility. The experimental results demonstrate that the adaptive control law is applicable for real-time implementation onboard a small free-flyer robot with practical Presented as Paper 2014-1288 at the AIAA Guidance, Navigation and Control Conference, National Harbor, MD, 13–17 January 2014; received 22 May 2015; revision received 24 December 2015; accepted for publication 4 January 2016; published online 29 March 2016. Copyright