Michael J. Massimino

Apparatus for providing vibrotactile sensory substitution of force feedback

A feedback apparatus for an operator to control an effector that is remote from the operator to interact with a remote environment has a local input device to be manipulated by the operator. Sensors in the effectors environment are capable of sensing the amplitude of forces arising between the effector and its environment, the direction of application of such forces, or both amplitude and direction. A feedback signal corresponding to such a component of the force, is generated and transmitted to the environment of the operator. The signal is transduced into a vibrotactile sensory substitution signal to which the operator is sensitive. Vibration producing apparatus present the vibrotactile signal to the operator. The full range of the force amplitude may be represented by a single, mechanical vibrator. Vibrotactile display elements can be located on the operators limbs, such as on the hand, fingers, arms, legs, feet, etc. The location of the application of the force may also be specified by the location of a vibrotactile display on the operators body. Alternatively, the location may be specified by the frequency of a vibrotactile signal.

Improved force perception through sensory substitution

Abstract The objective of the research presented in this paper was to study the capabilities of sensory substitution for force feedback, presented to the operator of a teleoperation system, through the tactile and auditory senses. Traditional bilateral force feedback or force reflection, which applies forces to a human operators hand or arm muscles, while generally beneficial can be limited in the operators ability to perceive small force values. Sensory substitution for force feedback was shown in this study to increase the operators ability to perceive small forces by allowing an increase in the effective feedback gain without risking instability or impeding the operators inputs to the system.

One handed tracking in six degrees of freedom

The effect of movement in each of the six degrees of freedom (d.o.f.) on operator tracking performance is studied. Subjects used a single six-d.o.f. hand controller to track a target ball with a control ball on a computer monitor. Depth cues in the displayed tracking environment made tracking in three dimensions possible. Three sets of experiments were conducted in which movement occurred in one, three, and six d.o.f. at a time. Root mean square error (r.m.s.e.) was calculated for each d.o.f. as the measure of performance. The experiments were performed with velocity and acceleration control plant inputs.<<ETX>>

Design and testing of a nonreactive, fingertip, tactile display for interaction with remote environments

During interaction with remote environments, the operator may benefit from the addition of force feedback to the ubiquitous visual feedback. However, the apparatus required for reactive force feedback (feedback which imposes the remote environments motion-constraints on the user by applying joint torques) is cumbersome and expensive, especially when implemented in conjunction with high degree-offreedom precision joint motion sensing. Non-reactive, tactile feedback can provide similar information, and can be implemented at much lower cost. The purposes of this research were (1)todesign and demonstrate an inexpensive tactile feedback system, and (2) to determine the extent to which such a system could aid in the performance of a simple teleoperation task. After some experimentation with some different display technologies, and preliminary design, a vibrotactile display was chosen because of its low weight, size, and low cost. The final design consisted of two voice-coils, one each for the thumb and the index finger, which were driven by a 250 Hz variableamplitude signal produced by an analog electronics unit which was controlled by a PC. Experimental results are provided to show that the addition of the tactile display provides a small but significant improvement in manual tracking performance over the use of the visual display alone, and that the tracking task may be performed with only the tactile display. In further experiments the tactile display is compared with reactive force-feedback and is shown to confer most of the reactive displays performance improvement over tracking with only a visual display.

A multi-mode manipulator display system for controlling remote robotic systems

The objective and contribution of the research presented in this paper is to provide a Multi-Mode Manipulator Display System (MMDS) to assist a human operator with the control of remote manipulator systems. Such systems include space based manipulators such as the space shuttle remote manipulator system (SRMS) and future ground controlled teleoperated and telescience space systems. The MMDS contains a number of display modes and submodes which display position control cues position data in graphical formats, based primarily on manipulator position and joint angle data. Therefore the MMDS is not dependent on visual information for input and can assist the operator especially when visual feedback is inadequate. This paper provides descriptions of the new modes and experiment results to date.

Using auditory and tactile displays for force feedback

Traditional force feedback or force reflection, which applies forces to a human operators hand or arm muscles, has been shown in several studies to be beneficial to a person performing remote manipulation tasks with a teleoperation system. However, force reflection can have its disadvantages including operator induced instabilities in the presence of time delays. The use of tactile and auditory displays to present force feedback will be discussed. These displays can provide the human operator with force information without some of the disadvantages of force reflection. The design of the displays are explained, as well as an experimental study on the effectiveness of the displays for remote manipulation tasks. These displays compared favorably to traditional force reflection for basic force perception tests, and improve the human operators sensitivity for detecting small forces. With a time delay, the displays improved operator performance for peg-in-hole tasks without instabilities. They also improved performance during degraded visual conditions. The benefits of using such displays for telemanipulation tasks is discussed, as well as potential applications and future research.

Using bisensory feedback displays for space teleoperation.

This paper investigates the use of tactile and auditory displays to present feedback to the pilot of a spacecraft or the operator of a space teleoperated system. Force feedback is given particular attention for teleoperator scenarios in the presence of a time delay. The motivation for and potential benefits of developing bisensory feedback displays are identified. Pioneering research in developing auditory and tactile displays is discussed. Several models of the human operator concerning the processing of bisensory information are outlined, along with a discussion of presenting redundant information across sensory modalities. Preliminary experimental results concerning sensory substitution of force feedback with a vibrotactile display are presented along with future research plans.

Sensory Substitution for Force Feedback in Teleoperation

Abstract The objective of this research was to study the capabilities of sensory substitution for force feedback through the tactile and auditory senses for teleoperation tasks, with and without time delay. The motivation and potential benefits of sensory substitution for force feedback with vibrotactile and auditory displays are discussed. Teleoperator experiments that examined the presentation of basic force information through object contact tasks, indicated that operator performance was improved by using the vibrotactile and auditory displays to present force information. Further, the vibrotactile and auditory displays compared favorably to traditional bilateral force feedback. Common manipulation experiments with peg-in-hole tasks of varying complexity were also conducted and showed that when the subjects’ view was fully obstructed, the subjects were able to successfully complete the task by using either of the sensory substitution displays. Sensory substitution was also tested in the presence of a three second time delay and significantly improved performance without instabilities.

Flexible operator aids for telemanipulation

To control current and future space telerobotics systems, the human operator can encounter challenging human-machine interface problems. These include operating the arm with limited positioning cues, avoiding joint limits or singularities, and operating the arm a single joint at a time rather than with the hand controllers which can be required due to system failures. We are developing a multi-mode manipulator display system (MMDS) that addresses these problems. The first mode, manipulator position display (MPD) mode provides the operator with positioning cues that are particularly helpful during operations with constrained viewing conditions. The second mode, joint angle display (JAD) mode assists the operator with avoiding joint limits and singularities, and can provide cues to alleviate these conditions once they occur. Single joint operations display (SJOD) mode is the third mode and provides cues to assist the operator when operating the manipulator a single joint at a time. The fourth mode of the MMDS is sensory substitution (SS) mode which can provide force feedback information through vibrotactile or auditory displays. The MMDS has been designed for space-based applications, but can be extended to a variety of human-machine telerobotic applications including toxic waste cleanup, undersea robotic operations, manufacturing systems, and control of prosthetic devices.