Jonathan Fiene

Improving contact realism through event-based haptic feedback

Tapping on surfaces in a typical virtual environment feels like contact with soft foam rather than a hard object. The realism of such interactions can be dramatically improved by superimposing event-based, high-frequency transient forces over traditional position-based feedback. When scaled by impact velocity, hand-tuned pulses and decaying sinusoids produce haptic cues that resemble those experienced during real impacts. Our new method for generating appropriate transients inverts a dynamic model of the haptic device to determine the motor forces required to create prerecorded acceleration profiles at the users fingertips. After development, the event-based haptic paradigm and the method of acceleration matching were evaluated in a carefully controlled user study. Sixteen individuals blindly tapped on nine virtual and three real samples, rating the degree to which each felt like real wood. Event-based feedback achieved significantly higher realism ratings than the traditional rendering method. The display of transient signals made virtual objects feel similar to a real sample of wood on a foam substrate, while position feedback alone received ratings similar to those of foam. This work provides an important new avenue for increasing the realism of contact in haptic interactions.

Robotic assistance for ultrasound-guided prostate brachytherapy

We present a robotically assisted prostate brachytherapy system and test results in training phantoms and Phase-I clinical trials. The system consists of a transrectal ultrasound (TRUS) and a spatially co-registered robot, fully integrated with an FDA-approved commercial treatment planning system. The salient feature of the system is a small parallel robot affixed to the mounting posts of the template. The robot replaces the template interchangeably, using the same coordinate system. Established clinical hardware, workflow and calibration remain intact. In all phantom experiments, we recorded the first insertion attempt without adjustment. All clinically relevant locations in the prostate were reached. Non-parallel needle trajectories were achieved. The pre-insertion transverse and rotational errors (measured with a Polaris optical tracker relative to the templates coordinate frame) were 0.25 mm (STD=0.17 mm) and 0.75 degrees (STD=0.37 degrees). In phantoms, needle tip placement errors measured in TRUS were 1.04 mm (STD=0.50mm). A Phase-I clinical feasibility and safety trial has been successfully completed with the system. We encountered needle tip positioning errors of a magnitude greater than 4mm in only 2 of 179 robotically guided needles, in contrast to manual template guidance where errors of this magnitude are much more common. Further clinical trials are necessary to determine whether the apparent benefits of the robotic assistant will lead to improvements in clinical efficacy and outcomes.

Event-based haptics and acceleration matching: portraying and assessing the realism of contact

Contact in a typical haptic environment resembles the experience of tapping on soft foam, rather than on a hard object. Event-based, high-frequency transient forces must be superimposed with traditional proportional feedback to provide realistic haptic cues at impact. We have developed a new method for matching the accelerations experienced during real contact, inverting a dynamic model of the device to compute appropriate force feedback transients. We evaluated this haptic rendering paradigm by conducting a study in which users blindly rated the realism of tapping on a variety of virtually rendered surfaces as well as on three real objects. Event-based feedback significantly increased the realism of the virtual surfaces, and the acceleration matching strategy was rated similarly to a sample of real wood on a foam substrate. This work provides a new avenue for achieving realism of contact in haptic interactions.

A high fidelity ungrounded torque feedback device: The iTorqU 2.0

This paper discusses the design and operation of the iTorqU 2.0, an ungrounded, handheld torque feedback device for haptic applications. Based upon the gyroscopic effect, the iTorqU 2.0 uses a metal flywheel inside of a two-axis actuated gimbal to create directional torques that are applied to the users hand. The coupling of angular velocity and angular momentum creates a torque that is orthogonal to the two input angular velocities, giving the user the impression that their hand is being twisted in free air. Following a review of prior work in the field of ungrounded torque feedback devices, we first present our preliminary prototype, the iTorqU 1.0. Building on empirical observations and user feedback from a public demonstration, we revised and augmented this design to create the iTorqU 2.0. This paper covers the major mechanical, electrical, and controls design considerations that went into creating the iTorqU 2.0, along with an analysis of its torque output capabilities.

Robotic needle guide for prostate brachytherapy: Clinical testing of feasibility and performance

PURPOSE Optimization of prostate brachytherapy is constrained by tissue deflection of needles and fixed spacing of template holes. We developed and clinically tested a robotic guide toward the goal of allowing greater freedom of needle placement. METHODS AND MATERIALS The robot consists of a small tubular needle guide attached to a robotically controlled arm. The apparatus is mounted and calibrated to operate in the same coordinate frame as a standard template. Translation in x and y directions over the perineum ±40 mm are possible. Needle insertion is performed manually. RESULTS Five patients were treated in an institutional review board-approved study. Confirmatory measurements of robotic movements for initial 3 patients using infrared tracking showed mean error of 0.489 mm (standard deviation, 0.328 mm). Fine adjustments in needle positioning were possible when tissue deflection was encountered; adjustments were performed in 54 (30.2%) of 179 needles placed, with 36 (20.1%) of 179 adjustments of >2mm. Twenty-seven insertions were intentionally altered to positions between the standard template grid to improve the dosimetric plan or avoid structures such as pubic bone and blood vessels. CONCLUSIONS Robotic needle positioning provided a means of compensating for needle deflections and the ability to intentionally place needles into areas between the standard template holes. To our knowledge, these results represent the first clinical testing of such a system. Future work will be incorporation of direct control of the robot by the physician, adding software algorithms to help avoid robot collisions with the ultrasound, and testing the angulation capability in the clinical setting.

Shaping Event-Based Haptic Transients Via an Improved Understanding of Real Contact Dynamics

Haptic interactions with stiff virtual surfaces feel more realistic when a short-duration transient is added to the spring force at contact. But how should this event-based transient be shaped? To answer this question, we present a targeted user study on virtual surface realism that demonstrates the importance of scaling transients correctly and hints at the complexity of this dynamic relationship. We then present a detailed examination of the dynamics of tapping on a rigid surface with a hand-held probe; theoretical modeling is combined with empirical data to determine the influence of impact velocity, impact acceleration, and user grip force on the resulting transient surface force. The derived mathematical relationships provide a formula for generating open-loop, event-based force transients upon impact with a virtual surface. By incorporating an understanding of the dynamics of real interactions into the re-creation of virtual contact, these findings promise to improve the performance and realism of a wide range of haptic simulations

Toward switching motor control

Traditionally focused on applications where humans are safely out of reach, the field of robotics was developed to perform repetitive tasks with high precision and accuracy. In contrast, recent developments have opened up a new subfield of robotics in which users are in direct contact with robotic devices. This physical interaction requires the device to be sensitive to human contact while maintaining performance. While hardware designs have progressed to fill this new area, the underlying controller remains nearly the same as that developed for industrial robots decades ago, imposing limits on the performance of the overall system. To address the unique requirements of these human-interactive robotic devices, we are working to replace the traditional amplifier with a high-speed switching controller. Combining the simplicity and efficiency of pulsewidth modulation with the high bandwidth of linear amplifiers, this approach demonstrates performance gains when implemented as a closed-loop current controller. We also extend this design to the proposal of a third-order architecture to combine an amplifier and servo controller into an integrated system with the potential to overcome the fundamental limits of todays systems and enable the future of human-interactive robotics.

Event-Based Haptic Tapping with Grip Force Compensation

Previous work in event-based haptics has demonstrated that augmenting position-based force feedback with high-frequency impact transients significantly improves the realism of virtual tapping. Transients can be portrayed more accurately by accounting for the dynamic relationship between actuator force and hand acceleration, a technique we call acceleration matching. This work extends the method of acceleration matching by analyzing how changes in user grip force affect the transients produced when tapping on real and virtual objects. We use this understanding to update the event-based paradigm, measuring grip force in real time and adjusting transient output accordingly. When implemented on a system that can provide high-bandwidth, high-magnitude current, this new approach successfully compensates for changes in user dynamics, significantly improving the robustness of event-based haptic tapping.

Switching motor control: an integrated amplifier design for improved velocity estimation and feedback

In this paper we propose an integrated motion control architecture based on a full third order motor model. We investigate the advantages and feasibility of controlling a motor using very fast (MHz) switching using digital components in place of traditional amplifiers. A switching controller combines the current and encoder (position) feedback paths into a single loop, and a model based observer estimates the unsensed motor velocity. When compared to second order controllers built with traditional amplifiers, the proposed design promises increased performance, better efficiency, and unproved velocity estimation. Advances in semiconductor technology facilitate the implementation of these fast switches and the required logic using relatively standard components. Preliminary experiments verify good behavior of a DC motor driven up to 5 MHz.

The Robockey Cup

Each fall, a group of intrepid students converge in a classroom on the third floor of the University of Pennsylvanias Towne Engineering Building for what they know will be one of the most challenging courses in their academic career. That first class begins with a discussion of where it will end. In particular, we talk about what they will accomplish in the final project, where they will design, fabricate, assemble, program, and debug small teams of autonomous hockey-playing robots, complete with wireless communications, infrared puck sensing, and enough onboard computational power to handle just about any task that can dream up. Most of the students look incredulous; after all, many of them have never built a circuit or written C code before.