Raz Leib

The effect of force feedback delay on stiffness perception and grip force modulation during tool-mediated interaction with elastic force fields

During interaction with objects, we form an internal representation of their mechanical properties. This representation is used for perception and for guiding actions, such as in precision grip, where grip force is modulated with the predicted load forces. In this study, we explored the relationship between grip force adjustment and perception of stiffness during interaction with linear elastic force fields. In a forced-choice paradigm, participants probed pairs of virtual force fields while grasping a force sensor that was attached to a haptic device. For each pair, they were asked which field had higher level of stiffness. In half of the pairs, the force feedback of one of the fields was delayed. Participants underestimated the stiffness of the delayed field relatively to the nondelayed, but their grip force characteristics were similar in both conditions. We analyzed the magnitude of the grip force and the lag between the grip force and the load force in the exploratory probing movements within each trial. Right before answering which force field had higher level of stiffness, both magnitude and lag were similar between delayed and nondelayed force fields. These results suggest that an accurate internal representation of environment stiffness and time delay was used for adjusting the grip force. However, this representation did not help in eliminating the bias in stiffness perception. We argue that during performance of a perceptual task that is based on proprioceptive feedback, separate neural mechanisms are responsible for perception and action-related computations in the brain.

Stimulation of PPC Affects the Mapping between Motion and Force Signals for Stiffness Perception But Not Motion Control

How motion and sensory inputs are combined to assess an objects stiffness is still unknown. Here, we provide evidence for the existence of a stiffness estimator in the human posterior parietal cortex (PPC). We showed previously that delaying force feedback with respect to motion when interacting with an object caused participants to underestimate its stiffness. We found that applying theta-burst transcranial magnetic stimulation (TMS) over the PPC, but not the dorsal premotor cortex, enhances this effect without affecting movement control. We explain this enhancement as an additional lag in force signals. This is the first causal evidence that the PPC is not only involved in motion control, but also has an important role in perception that is disassociated from action. We provide a computational model suggesting that the PPC integrates position and force signals for perception of stiffness and that TMS alters the synchronization between the two signals causing lasting consequences on perceptual behavior. SIGNIFICANCE STATEMENT When selecting an object such as a ripe fruit or sofa, we need to assess the objects stiffness. Because we lack dedicated stiffness sensors, we rely on an as yet unknown mechanism that generates stiffness percepts by combining position and force signals. Here, we found that the posterior parietal cortex (PPC) contributes to combining position and force signals for stiffness estimation. This finding challenges the classical view about the role of the PPC in regulating position signals only for motion control because we highlight a key role of the PPC in perception that is disassociated from action. Altogether this sheds light on brain mechanisms underlying the interaction between action and perception and may help in the development of better teleoperation systems and rehabilitation of patients with sensory impairments.

Perception of Stiffness during Interaction with Delay-Like Nonlinear Force Field

The perception of linear stiffness, as well as delayed linear stiffness, was studied extensively during the last decades. In this study we set to explore the effects of non linear relation between force and position on perception of stiffness. We designed a state dependent non-linear force field, similar to the previously explored delayed force field, which is essentially a piecewise linear force field depending only on the position and the direction of movement and not on time. We show that the stiffness of this force field is overestimated. We suggest a model based on the inverse of the slope of a regression of position over force in order to explain these behavioral results, which indirectly implies that force control is used during this exploratory probing.

The Mechanical Representation of Temporal Delays

When we knock on a door, we perceive the impact as a collection of simultaneous events, combining sound, sight, and tactile sensation. In reality, information from different modalities but from a single source is flowing inside the brain along different pathways, reaching processing centers at different times. Therefore, interpreting different sensory modalities which seem to occur simultaneously requires information processing that accounts for these different delays. As in a computer-based robotic system, does the brain use some explicit estimation of the time delay, to realign the sensory flows? Or does it compensate for temporal delays by representing them as changes in the body/environment mechanics? Using delayed-state or an approximation for delayed-state manipulations between visual and proprioceptive feedback during a tracking task, we show that tracking errors, grip forces, and learning curves are consistent with predictions of a representation that is based on approximation for delay, refuting an explicit delayed-state representation. Delayed-state representations are based on estimating the time elapsed between the movement commands and their observed consequences. In contrast, an approximation for delay representations result from estimating the instantaneous relation between the expected and observed motion variables, without explicit reference to time.

Perception of Stiffness with Force Feedback Delay

This book is focused on understanding how the human sensorimotor system integrates various sources of information to form a representation of stiffness—the linear relation between position and force. In this chapter, we will examine attempts to answer this question when users interact with artificially changed environment in which the force resulting from an interaction with the object is delayed, such as in the case of remote bilateral teleoperation.

Stretching the skin of the fingertip creates a perceptual and motor illusion of touching a harder spring

We investigated how artificial tactile feedback in the form of a skin-stretch affects perceptions of stiffness and grip force adjustment. During interactions with objects, information from kinesthetic and tactile sensors is used to estimate the forces acting on the limbs. These enable perceptions of the mechanical properties of objects to form, and the creation of internal models to predict the consequences of interactions with these objects such as feedforward grip-force adjustments to prevent slippage. Artificial tactor displacement-induced skin stretch can produce a linear additive effect on stiffness perception, but it remains unclear how artificial stretch affects the control of grip force. Here, we used a robotic device and a custom-built skin-stretch to manipulate kinesthetic and tactile information. Using a stiffness discrimination task, we found that adding artificial tactile feedback to a kinesthetic force can create the illusion of touching a harder spring which affects both perception and action. The magnitude of the illusion is linearly related to the amplitude of the applied stretch. We also isolated the contribution of tactile stimulation to the predictive and reactive components of grip force adjustment, and found that unlike other cases of perceptual illusions, the predictive grip force is modulated consistently with the perceptual tactile-induced illusion. These results have major implications for the design of tactile interfaces across a variety of touch applications such as wearable haptic devices, teleoperations, robot-assisted surgery and prosthetics.

State-Based Delay Representation and Its Transfer from a Game of Pong to Reaching and Tracking

Abstract To accurately estimate the state of the body, the nervous system needs to account for delays between signals from different sensory modalities. To investigate how such delays may be represented in the sensorimotor system, we asked human participants to play a virtual pong game in which the movement of the virtual paddle was delayed with respect to their hand movement. We tested the representation of this new mapping between the hand and the delayed paddle by examining transfer of adaptation to blind reaching and blind tracking tasks. These blind tasks enabled to capture the representation in feedforward mechanisms of movement control. A Time Representation of the delay is an estimation of the actual time lag between hand and paddle movements. A State Representation is a representation of delay using current state variables: the distance between the paddle and the ball originating from the delay may be considered as a spatial shift; the low sensitivity in the response of the paddle may be interpreted as a minifying gain; and the lag may be attributed to a mechanical resistance that influences paddle’s movement. We found that the effects of prolonged exposure to the delayed feedback transferred to blind reaching and tracking tasks and caused participants to exhibit hypermetric movements. These results, together with simulations of our representation models, suggest that delay is not represented based on time, but rather as a spatial gain change in visuomotor mapping.

Smart switching in feedforward control of grip force during manipulation of elastic objects

Switching systems are common in artificial control systems. Here, we suggest that the brain adopts a switched feedforward control of grip forces during manipulation of objects. We measured how participants modulated grip force when interacting with soft and rigid virtual springs when stiffness varied nearly continuously between trials. We identified a sudden phase transition between two forms of feedforward control that differed in the timing of the synchronization between the anticipated load force and the applied grip force. The switch occurred several trials after a threshold stiffness level. These results suggest that in the control of grip force, the brain acts as a switching control system. This opens new research questions as to the nature of the discrete state variables that drive the switching.

An intermittent control model predicts the triphasic muscles activity during hand reaching

There are numerous ways to reach for an apple hanging from a tree. Yet, our motor system uses a specific muscle activity pattern to generate reaching movements that have similar characteristics. For many decades, we know that this pattern features activity bursts and silent periods. We suggest that these bursts are a strong evidence against the common view that the brain continuously controls the commands to the muscles. Instead, we suggest a model that changes these commands in a discrete way. We use unsupervised machine learning to identify transitions in the state of the muscles, and show that fitting a discrete model to the kinematics of movement using only one parameter predicts the transitions in the state of the muscles. Such discrete controller suggests that the brain reduces the complexity of the motor control problem as well as the wear-and-tear of the muscles by sending commands to the muscles at sparse times. Identifying this discrete controller can be applied in the control of prostheses and physical human-robot interaction systems such as exoskeletons and assistive devices.