Christopher Assad

Gesture-based robot control with variable autonomy from the JPL BioSleeve

This paper presents a new gesture-based human interface for natural robot control. Detailed activity of the users hand and arm is acquired via a novel device, called the BioSleeve, which packages dry-contact surface electromyography (EMG) and an inertial measurement unit (IMU) into a sleeve worn on the forearm. The BioSleeves accompanying algorithms can reliably decode as many as sixteen discrete hand gestures and estimate the continuous orientation of the forearm. These gestures and positions are mapped to robot commands that, to varying degrees, integrate with the robots perception of its environment and its ability to complete tasks autonomously. This flexible approach enables, for example, supervisory point-to-goal commands, virtual joystick for guarded teleoperation, and high degree of freedom mimicked manipulation, all from a single device. The BioSleeve is meant for portable field use; unlike other gesture recognition systems, use of the BioSleeve for robot control is invariant to lighting conditions, occlusions, and the human-robot spatial relationship and does not encumber the users hands. The BioSleeve control approach has been implemented on three robot types, and we present proof-of-principle demonstrations with mobile ground robots, manipulation robots, and prosthetic hands.

Phase and amplitude maps of the electric organ discharge of the weakly electric fish, Apteronotus leptorhynchus

SummaryThe electric organ discharge (EOD) potential was mapped on the skin and midplane of several Apteronotus leptorhynchus. The frequency components of the EOD on the surface of the fish have extremely stable amplitude and phase. However, the waveform varies considerably with different positions on the body surface. Peaks and zero crossings of the potential propagate along the fishs body, and there is no point where the potential is always zero. The EOD differs significantly from a sinusoid over at least one third of the body and tail. A qualitative comparison between fish showed that each individual had a unique spatiotemporal pattern of the EOD potential on its body.The potential waveforms have been assembled into high temporal and spatial resolution maps which show the dynamics of the EOD. Animation sequences and Macintosh software are available by anonymous ftp (mordor.cns.caltech.edu; cd/pub/ElectricFish).We interpret the EOD maps in terms of ramifications on electric organ control and electroreception. The electrocytes comprising the electric organ do not all fire in unison, indicating that the command pathway is not synchronized overall. The maps suggest that electroreceptors in different regions fulfill different computational roles in electroreception. Receptor mechanisms may exist to make use of the phase information or harmonic content of the EOD, so that both spatial and temporal patterns could contribute information useful for electrolocation and communication.

Electric organ discharges of the gymnotiform fishes: III. Brachyhypopomus.

Abstract We measured and mapped the electric fields produced by three species of neotropical electric fish of the genus Brachyhypopomus (Gymnotiformes, Rham phichthyoidea, Hypopomidae), formerly Hypopomus. These species produce biphasic pulsed discharges from myogenic electric organs. Spatio-temporal false-color maps of the electric organ discharges measured on the skin show that the electric field is not a simple dipole in Brachyhypopomus. Instead, the dipole center moves rostro-caudally during the 1st phase (P1) of the electric organ discharge, and is stationary during the 2nd phase (P2). Except at the head and tip of tail, electric field lines rotate in the lateral and dorso-ventral planes. Rostro-caudal differences in field amplitude, field lines, and spatial stability suggest that different parts of the electric organ have undergone selection for different functions; the rostral portions seem specialized for electrosensory processing, whereas the caudal portions show adaptations for d.c. signal balancing and mate attraction as well. Computer animations of the electric field images described in this paper are available on web sites http://www.bbb.caltech.edu/ElectricFish or http://www.fiu.edu/∼stoddard/electricfish.html.

Dynamics and the single spike

Responses of vestibular primary afferent neurons to head rotation exhibit fractional-order dynamics. As a consequence, the head tends to be in a localized region of its state-space at spike times of a particular neuron during arbitrary head movements, and single spikes can be interpreted as state measurements. We are developing a model of neural computations underlying trajectory prediction and control tasks, based on this experimental observation. This is a step toward a formal neural calculus in which single spikes are modeled realistically as the operands of neural computation.

BioSleeve: a natural EMG-based interface for HRI

This paper presents the BioSleeve, a new gesture-based human interface for natural robot control. Detailed activity of the users hand and arm is acquired via surface electromyography sensors and an inertial measurement unit that are embedded in a forearm sleeve. The BioSleeves accompanying software decodes the sensor signals, classifies gesture type, and maps the result to output commands to an external robot. The current BioSleeve system can reliably decode as many as sixteen discrete hand gestures and estimate the continuous orientation of the forearm. The gestures are used in several modes: for supervisory point-to-goal commands, virtual joystick for teleoperation, and high degree-of-freedom (DOF) mimicked manipulation. We report results from three control applications: a manipulation robot, a small ground vehicle, and a 5-DOF hand.

Unmanned Ground Vehicle Perception Using Thermal Infrared Cameras

The ability to perform off-road autonomous navigation at any time of day or night is a requirement for some unmanned ground vehicle (UGV) programs. Because there are times when it is desirable for military UGVs to operate without emitting strong, detectable electromagnetic signals, a passive only terrain perception mode of operation is also often a requirement. Thermal infrared (TIR) cameras can be used to provide day and night passive terrain perception. TIR cameras have a detector sensitive to either mid-wave infrared (MWIR) radiation (3-5μm) or long-wave infrared (LWIR) radiation (7-14μm). With the recent emergence of high-quality uncooled LWIR cameras, TIR cameras have become viable passive perception options for some UGV programs. The Jet Propulsion Laboratory (JPL) has used a stereo pair of TIR cameras under several UGV programs to perform stereo ranging, terrain mapping, tree-trunk detection, pedestrian detection, negative obstacle detection, and water detection based on object reflections. In addition, we have evaluated stereo range data at a variety of UGV speeds, evaluated dual-band TIR classification of soil, vegetation, and rock terrain types, analyzed 24 hour water and 12 hour mud TIR imagery, and analyzed TIR imagery for hazard detection through smoke. Since TIR cameras do not currently provide the resolution available from megapixel color cameras, a UGVs daytime safe speed is often reduced when using TIR instead of color cameras. In this paper, we summarize the UGV terrain perception work JPL has performed with TIR cameras over the last decade and describe a calibration target developed by General Dynamics Robotic Systems (GDRS) for TIR cameras and other sensors.

The electric organ discharges of the gymnotiform fishes: II. Eigenmannia

Abstract We present detailed measurements of the electric organ discharge of the weakly electric fish, Eigenmannia sp. These maps illuminate, with high resolution in both space and time, the electric organ discharge potential and electric field patterns in the water about the fish and on the skin surface itself. The results demonstrate that the electric organ discharge of Eigenmannia approximates a simple oscillating dipole, which confirms previous descriptions and assumptions, but is in contrast to the electric organ discharges of several other gymnotiform species. Over each cycle of Eigenmannias electric organ discharge, the electric field amplitude measured at any point near the fish oscillates from positive to negative, but the field vector remains nearly constant in direction. This electric organ discharge pattern is correlated with known anatomical and physiological features of the fishs electric organ, and confirms that the activation of electrocytes comprising the organ is well synchronized. As a result, the relatively simple electric organ discharge leads to a fairly uniform pattern of electrosensory stimuli along the body surface, which may facilitate central processing of electrosensory images. Electric organ discharge maps and animations resulting from this series of studies are available via the Internet (http://www.bbb.caltech.edu/ElectricFish, or www.fiu.edu/∼stoddard/electricfish.html).

Applications of multimedia computers and video mixing to neuroethology

Inexpensive multimedia computers offer new possibilities for mixing video and computer images, videotaping these mixed images, and extracting quantitative data from videotape. In this paper we describe methods for mixing images from a video camera and a Macintosh computer display using chroma keying, and we describe a simple circuit for analog video mixing and frame-counting. We present three applications of these video mixing methods to our neurophysiological and behavioral research with awake, behaving animals. These technologies enhance accuracy, speed, and flexibility during experiments, by allowing us to record an information-rich videotape of the subject and state of the experimental apparatus. After the experiment, these technologies facilitate selecting and extracting quantitative data from the videotape for further analysis. The videotape is especially useful in resolving minor inconsistencies or incomplete information in the notes and data files that often arise during analysis of complex experiments.

A Model of Cerebellar Computations for Dynamical State Estimation

The cerebellum is a neural structure that is essential for agility in vertebrate movements. Its contribution to motor control appears to be due to a fundamental role in dynamical state estimation, which also underlies its role in various non-motor tasks. Single spikes in vestibular sensory neurons carry information about head state. We show how computations for optimal dynamical state estimation may be accomplished when signals are encoded in spikes. This provides a novel way to design dynamical state estimators, and a novel way to interpret the structure and function of the cerebellum.

Hypercube Simulation of Electric Fish Potentials

We present a simulation of the electrosensory input of the weakly electric fish Apteronotus leptorhynchus. This fish senses its environment by producing a sinusoidal voltage difference between its body and tail sections, causing an electric field and a current distribution in the surrounding water. If an object is nearby which has different electrical conductivity from the surrounding water, the current distribution is disturbed on the skin of the fish. The fish senses this difference from the usual current distribution, and infers the presence and location of the object. Mathematically, the problem is to solve a potential equation in the domain exterior to the fish with Cauchy boundary conditions, in the presence of an induced dipole arising from the object, and extract the potential difference across the fish skin. We have created an unstructured triangular mesh covering the two-dimensional manifold of the fish skin, using the distributed Irregular Mesh Environment (DIME), then used the Boundary Element Method to solve for the potential derivative at the fish skin. The computational problem is the solution of a full set of simultaneous linear equations, where there is an equation for each node of the boundary mesh, typically about 100 - 200. We have used an NCUBE hypercube to calculate the matrix elements and solve these equations, once for each relative position of the fish and the test object. We present some early results from the simulation.