R. Brent Gillespie

Sharing control between humans and automation using haptic interface : Primary and secondary task performance benefits

This paper describes a paradigm for human/automation control sharing in which the automation acts through a motor coupled to a machines manual control interface. The manual interface becomes a haptic display, continually informing the human about automation actions. While monitoring by feel, users may choose either to conform to the automation or override it and express their own control intentions. This papers objective is to demonstrate that adding automation through haptic display can be used not only to improve performance on a primary task but also to reduce perceptual demands or free attention for a secondary task. Results are presented from three experiments in which 11 participants completed a lane-following task using a motorized steering wheel on a fixed-base driving simulator. The automation behaved like a copilot, assisting with lane following by applying torques to the steering wheel. Results indicate that haptic assist improves lane following by least 30%, p < .0001, while reducing visual demand by 29%, p < .0001, or improving reaction time in a secondary tone localization task by 18 ms, p = .0009. Potential applications of this research include the design of automation interfaces based on haptics that support human/automation control sharing better than traditional push-button automation interfaces.

Shared Control between Human and Machine: Using a Haptic Steering Wheel to Aid in Land Vehicle Guidance

When humans interface with machines, the control interface is usually passive and its response contains little information pertinent to the state of the environment. Usually, information flows through the interface from human to machine but not so often in the reverse direction. This work proposes a control architecture in which bi-directional information transfer occurs across the control interface, allowing the human to use the interface to simultaneously exert control and extract information. In this alternative control architecture, which we call shared control, the human utilizes the haptic sensory modality to share control of the machine interface with an automatic controller. We present a fixed-base driving simulator experiment in which subjects take advantage of a haptic steering wheel, which aids them in a path following task. Results indicate that the haptic steering wheel allows a significant reduction in visual demand while improving path following performance.

An Energy Management Controller to Optimally Trade Off Fuel Economy and Drivability for Hybrid Vehicles

Hybrid vehicle fuel economy performance is highly sensitive to the energy management strategy used to regulate power flow among the various energy sources and sinks. Optimal non-causal solutions are easy to determine if the drive cycle is known a priori. It is very challenging to design causal controllers that yield good fuel economy for a range of possible driver behavior. Additional challenges come in the form of constraints on powertrain activity, such as shifting and starting the engine, which are commonly called “drivability” metrics and can adversely affect fuel economy. In this paper, drivability restrictions are included in a shortest path stochastic dynamic programming (SP-SDP) formulation of the real-time energy management problem for a prototype vehicle, where the drive cycle is modeled as a stationary, finite-state Markov chain. When the SP-SDP controllers are evaluated with a high-fidelity vehicle simulator over standard government drive cycles, and compared to a baseline industrial controller, they are shown to improve fuel economy more than 11% for equivalent levels of drivability. In addition, the explicit tradeoff between fuel economy and drivability is quantified for the SP-SDP controllers.

Visual and haptic feedback contribute to tuning and online control during object manipulation.

The authors employed a virtual environment to investigate how humans use haptic and visual feedback in a simple, rhythmic object-manipulation task. The authors hypothesized that feedback would help participants identify the appropriate resonant frequency and perform online control adjustments. The 1st test was whether sensory feedback is needed at all; the 2nd was whether the motor system combines visual and haptic feedback to improve performance. Task performance was quantified in terms of work performed on the virtual inertia, ability to identify the correct rhythm, and variability of movement. Strict feedforward control was found to be ineffective for this task, even when participants had previous knowledge of the rhythm. Participants (N = 11) performed far better when feedback was available (11 times more work, 2.2 times more precise frequency, 30% less variability; p < .05 for all 3 performance measures). Using sensory feedback, participants were able to rapidly identify 4 different spring-inertia systems without foreknowledge of the corresponding resonant frequencies. They performed over 20% more work with 24% less variability when provided with both visual and haptic feedback than they did with either feedback channel alone (p < .05), providing evidence that they integrated online sensory channels. Whereas feedforward control alone led to poor performance, feedback control led to fast tuning or calibration of control according to the resonant frequency of the object, and to better control of the rhythmic movement itself.

Providing a sense of touch to prosthetic hands.

Summary: Each year, approximately 185,000 Americans suffer the devastating loss of a limb. The effects of upper limb amputations are profound because a person’s hands are tools for everyday functioning, expressive communication, and other uniquely human attributes. Despite the advancements in prosthetic technology, current upper limb prostheses are still limited in terms of complex motor control and sensory feedback. Sensory feedback is critical to restoring full functionality to amputated patients because it would relieve the cognitive burden of relying solely on visual input to monitor motor commands and provide tremendous psychological benefits. This article reviews the latest innovations in sensory feedback and argues in favor of peripheral nerve interfaces. First, the authors examine the structure of the peripheral nerve and its importance in the development of a sensory interface. Second, the authors discuss advancements in targeted muscle reinnervation and direct neural stimulation by means of intraneural electrodes. The authors then explore the future of prosthetic sensory feedback using innovative technologies for neural signaling, specifically, the sensory regenerative peripheral nerve interface and optogenetics. These breakthroughs pave the way for the development of a prosthetic limb with the ability to feel.

Refreshing Refreshable Braille Displays

The increased access to books afforded to blind people via e-publishing has given them long-sought independence for both recreational and educational reading. In most cases, blind readers access materials using speech output. For some content such as highly technical texts, music, and graphics, speech is not an appropriate access modality as it does not promote deep understanding. Therefore blind braille readers often prefer electronic braille displays. But, these are prohibitively expensive. The search is on, therefore, for a low-cost refreshable display that would go beyond current technologies and deliver graphical content as well as text. And many solutions have been proposed, some of which reduce costs by restricting the number of characters that can be displayed, even down to a single braille cell. In this paper, we demonstrate that restricting tactile cues during braille reading leads to poorer performance in a letter recognition task. In particular, we show that lack of sliding contact between the fingertip and the braille reading surface results in more errors and that the number of errors increases as a function of presentation speed. These findings suggest that single cell displays which do not incorporate sliding contact are likely to be less effective for braille reading.

Toward improved sensorimotor integration and learning using upper-limb prosthetic devices

To harness the increased dexterity and sensing capabilities in advanced prosthetic device designs, amputees will require interfaces supported by novel forms of sensory feedback and novel control paradigms. We are using a motorized elbow brace to feed back grasp forces to the user in the form of extension torques about the elbow. This force display complements myoelectric control of grip closure in which EMG signals are drawn from the biceps muscle. We expect that the action/reaction coupling experienced by the biceps muscle will produce an intuitive paradigm for object manipulation, and we hope to uncover neural correlates to support this hypothesis. In this paper we present results from an experiment in which 7 able-bodied persons attempted to distinguish three objects by stiffness while grasping them under myoelectric control and feeling reaction forces displayed to their elbow. In four conditions (with and without force display, and using biceps myoelectric signals ipsilateral and contralateral to the force display,) ability to correctly identify objects was significantly increased with sensory feedback.

Kinematic Creep in a Continuously Variable Transmission: Traction Drive Mechanics for Cobots

Two continuously variable transmissions are examined, one that relates a pair of linear speeds and another that relates a pair of angular speeds. These devices are elemental in the design of cobots, a new class of robot that creates virtual guiding surfaces to aid a human operator in assembly tasks. Both of these transmissions are traction drive mechanisms that rely on the support of either lateral or longitudinal forces across rolling contacts with spin. When a rolling contact between elastic bodies or even between rigid bodies in spin is called upon to transmit a tractive force, kinematic creep develops, expressing a departure from the intended rolling constraint. Creep in turn gives rise to nonideal properties in a cobot’s virtual guiding surfaces. This paper develops simple models of the two transmissions by expressing the relative velocity field in the contact patch between rolling bodies in terms of creep and spin. Coulomb friction laws are applied in a quasi-static analysis to produce complete force-motion models. These models may be used to evaluate a cobot’s ability to support forces against its virtual guiding surfaces. @DOI: 10.1115/1.1517560#

A Fundamental Linear Systems Conflict Between Performance and Passivity in Haptic Rendering

This paper analyzes inherent conflicts between model-matching goals of haptic rendering and passivity requirements for coupled stability. We apply results from complex analysis to prove that certain linear passive virtual environments cannot be rendered passively with a desired level of accuracy and over a given finite bandwidth. One practical consequence is that, under appropriate hypotheses, passivity will be violated when accurately rendering inertia in a virtual environment that is less than the inertia of the uncompensated hardware dynamics. In a related result, we show that there exists a waterbed-type tradeoff between performance and phase lag in the rendered dynamics. Both design constraints arise from feedback-bandwidth limitations and not sampled-data effects, quantization, or nonlinearities. The key to our analysis is an interpretation of a Bode gain-phase integral relationship that relates magnitude at low frequencies to phase at high frequencies. The performance limitation and the waterbed tradeoff are illustrated through an experimental study.

Effects of haptic device attributes on vibration detection thresholds

Human vibrotactile detection experiments were used to compare temporal sinusoids displayed on three commercial haptic devices to a high-fidelity linear voice-coil actuator. The three commercial haptic devices we used span the cost spectrum, supposing that cost of a device is correlated with the fidelity of its virtual textures. This turned out not to be the case. The results indicated that none of the three haptic devices we tested were able to render perceptually distortion-free, periodically regular vibrations at detection threshold levels. Further investigation into the electrical and mechanical device properties that limited the performance of these devices revealed that D/A resolution, amplifier non-linearity and stiction were the primary sources of signal corruption.