Samuel B. Schorr

[D67] Sensory substitution using 3-Degree-of-Freedom tangential and normal skin deformation feedback

During manual interactions, we experience both kinesthetic forces and tactile sensations. Friction and normal force between the fingerpads and the tool/interaction surfaces cause shear and normal deformation of the skin. Capitalizing on this observation, we designed a 3-degree-of-freedom (DoF) tactile device that is grasped by a user and can render both tangential skin stretch and normal deformation on the skin of the users fingerpads. Tactile feedback from the device is delivered in a manner consistent with natural tactile cues from manual interaction. An experiment assessed the accuracy with which users can locate the center of a contoured hole on a virtual surface. The task was completed under four conditions: the cases of skin deformation and force feedback, with both 3- and 1-DoF feedback in each case. With 3-DoF feedback, users located the hole faster and more accurately than with 1-DoF feedback, for both force and skin deformation feedback. These results indicated that users were able to interpret the additional DoF cues provided by our 3-DoF tactile device to improve task performance.

Fingertip Tactile Devices for Virtual Object Manipulation and Exploration

One of the main barriers to immersivity during object manipulation in virtual reality is the lack of realistic haptic feedback. Our goal is to convey compelling interactions with virtual objects, such as grasping, squeezing, pressing, lifting, and stroking, without requiring a bulky, world-grounded kinesthetic feedback device (traditional haptics) or the use of predetermined passive objects (haptic retargeting). To achieve this, we use a pair of finger-mounted haptic feedback devices that deform the skin on the fingertips to convey cutaneous force information from object manipulation. We show that users can perceive differences in virtual object weight and that they apply increasing grasp forces when lifting virtual objects as rendered mass is increased. Moreover, we show how naive users perceive changes of a virtual objects physical properties when we use skin deformation to render objects with varying mass, friction, and stiffness. These studies demonstrate that fingertip skin deformation devices can provide a compelling haptic experience appropriate for virtual reality scenarios involving object manipulation.

Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback

During tool-mediated interaction with everyday objects, we experience kinesthetic forces and tactile sensations in the form of vibration and skin deformation at the fingerpad. Fingerpad skin deformation is caused by forces applied tangentially and normally to the fingerpad skin, resulting in tangential and normal skin displacement. We designed a device to convey 3-degree-of-freedom (DoF) force information to the user via skin deformation, and conducted two experiments to determine the devices effectiveness for force-feedback substitution and augmentation. For sensory substitution, participants used 1-DoF and 3-DoF skin deformation feedback to locate a feature in a 3-DoF virtual environment. Participants showed improved precision and shorter completion time when using 3-DoF compared to 1-DoF skin deformation feedback. For sensory augmentation, participants traced a path in space from an initial to a target location, while under guidance from force and/or skin deformation feedback. When force feedback was augmented with skin deformation, participants reduced their path-following error over the cases when force or skin deformation feedback are used separately. We conclude that 3-DoF skin deformation feedback is effective in substituting or augmenting force feedback. Such substitution or augmentation could be used when force feedback is unattainable or attenuated due to device limitations or system instability.

Three-Dimensional Skin Deformation as Force Substitution: Wearable Device Design and Performance During Haptic Exploration of Virtual Environments

Virtual reality systems would benefit from a compelling force sensory substitute when workspace or stability limitations prevent the use of kinesthetic force feedback systems. We present a wearable fingertip haptic device with the ability to make and break contact in addition to rendering both shear and normal skin deformation to the fingerpad. A delta mechanism with novel bias spring and tether actuator relocation method enables the use of high-end motors and encoders, allowing precise device control: 10 Hz bandwidth and 0.255 mm RMS tracking error were achieved during testing. In the first of two experiments, participants determined the orientation of a stiff region in a surrounding compliant virtual surface with an average angular error of 7.6 degree, similar to that found in previous studies using traditional force feedback. In the second experiment, we evaluated participants’ ability to interpret differences in friction. The Just Noticeable Difference (JND) of surface friction coefficient discrimination using our skin deformation device was 0.20, corresponding with a reference friction coefficient of 0.5. While higher than that found using kinesthetic feedback, this demonstrates that users can perceive differences in surface friction without world-grounded kinesthetic forces. These experiments show that three DoF skin deformation enables both stiffness and friction discrimination capability in the absence of kinesthetic force feedback.

Tactor-Induced Skin Stretch as a Sensory Substitution Method in Teleoperated Palpation

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

Environment Perception in the Presence of Kinesthetic or Tactile Guidance Virtual Fixtures

During multi-lateral collaborative teleoperation, where multiple human or autonomous agents share control of a teleoperation system, it is important to be able to convey individual user intent. One option for conveying the actions and intent of users or autonomous agents is to provide force guidance from one user to another. Under this paradigm, forces would be transmitted from one user to another in order to guide motions and actions. However, the use of force guidance to convey intent can mask environmental force feedback. In this paper we explore the possibility of using tactile feedback, in particular skin deformation feedback, skin deformation feedback to convey collaborative intent while preserving environmental force perception. An experiment was performed to test the ability of participants to use force guidance and skin deformation guidance to follow a path while interacting with a virtual environment. In addition, we tested the ability of participants to discriminate virtual environment stiffness when receiving either force guidance or skin deformation guidance. We found that skin deformation guidance resulted in a reduction of path-following accuracy, but increased the ability to discriminate environment stiffness when compared with force feedback guidance. Categories and Subject Descriptors H.1.2 [Models and Principles]: User/Machine Systems-Human information processing; H.5.2 [Information Interfaces and Presentation]: User Interfaces-Haptic I/O

Tactile Skin Deformation Feedback for Conveying Environment Forces in Teleoperation

Teleoperated robots are used in a variety of applications. The immersive nature of the teleoperated experience is often limited by a lack of haptic information. However, in many applications there are difficulties conveying force information due to limitations in hardware fidelity and the inherent tradeoffs between stability and transparency. In situations where force feedback is limited, it is possible to use sensory substitution methods to convey this force information via other sensory modalities. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. We created and tested a tactile device that emulates the natural skin deformation present during tool mediated manual interaction. With this device, experiment participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using skin stretch, force, reduced gain force, graphic, or vibration feedback. Participants using skin stretch feedback were able to determine the orientation of the region as accurately as when using force feedback and significantly better than when using vibration feedback, but also exhibited higher interaction forces. Thus, skin stretch feedback may be useful in scenarios where force feedback is reduced or infeasible.

The effect of manipulator gripper stiffness on teleoperated task performance

During robot-assisted minimally invasive surgery, surgeons perform challenging dexterous tasks, including the manipulation of soft tissue and suture tying. In the absence of environment force sensing of tool-tissue interaction forces to provide force feedback, surgeons must rely on visual feedback to modulate the grip force they apply on the environment. Clinical systems, like the da Vinci Surgical System (Intuitive Surgical, Inc.), use physical springs to provide closing resistance on the gripper degree-of-freedom (DOF) of the master manipulator. This feedback provides increasing force resistance as the gripper is closed. To determine the effect of master manipulator gripper stiffness on performance in a teleoperated manipulation task, we designed a new and open source gripper, the OmniGrip. The OmniGrip attaches to a SensAble Phantom Omni (now available as Geomagic Touch), replacing the stySensAble Phantom Omnilus end effector, and providing the ability for user programmable force characteristics. We conducted a study in which participants used an OmniGrip to teleoperate a Raven II surgical robRaven II surgical robotic systemotic system in a pick-and-place task. Increasing the stiffness of the OmniGrip resulted in reduced interaction forces at the slave-side environment. Additionally, these interaction forces were significantly lower when the OmniGrip as compared to when using the Phantom Omni stylus.