Christoph W. Borst

Realistic virtual grasping

We present a physically-based approach to grasping and manipulation of virtual objects that produces visually realistic results, addresses the problem of visual interpenetration of hand and object models, and performs force rendering for force-feedback gloves in a single framework. Our approach couples tracked hand configuration to a simulation-controlled articulated hand model using a system of linear and torsional spring-dampers. We discuss an implementation of our approach that uses a widely-available simulation tool for collision detection and response. We illustrate the resulting behavior of the virtual hand model and of grasped objects, and we show that the simulation rate is sufficient for control of current force-feedback glove designs. We also present a prototype of a system we are developing to support natural whole-hand interactions in a desktop-sized workspace.

Evaluation of a haptic mixed reality system for interactions with a virtual control panel

We present a haptic feedback technique that combines feedback from a portable force-feedback glove with feedback from direct contact with rigid passive objects. This approach is a haptic analogue of visual mixed reality, since it can be used to haptically combine real and virtual elements in a single display. We discuss device limitations that motivated this combined approach and summarize technological challenges encountered. We present three experiments to evaluate the approach for interactions with buttons and sliders on a virtual control panel. In our first experiment, this approach resulted in better task performance and better subjective ratings than the use of only a force-feedback glove. In our second experiment, visual feedback was degraded and the combined approach resulted in better performance than the glove-only approach and in better ratings of slider interactions than both glove-only and passive-only approaches. A third experiment allowed subjective comparison of approaches and provided additional evidence that the combined approach provides the best experience.

A spring model for whole-hand virtual grasping

We present a physically-based approach to grasping and manipulation of virtual objects that produces visually realistic results, addresses the problem of visual inter-penetration of hand and object models, and performs force rendering for force-feedback gloves, in a single framework. Our approach couples a simulation-controlled articulated hand model to tracked hand configuration using a system of linear and torsional virtual spring-dampers. We discuss an implementation of our approach that uses a widely available simulation tool for collision detection and response. We pictorially illustrate the resulting behavior of the virtual hand model and of grasped objects, discuss user behavior and difficulties encountered, and show that the simulation rate is sufficient for control of current force-feedback glove designs. We also present a prototype system for natural whole-hand interactions in a desktop-sized workspace.

Predictive coding for efficient host-device communication in a pneumatic force-feedback display

We investigate predictive coding for reducing the amount of data communicated between a haptic controller and a host. This allows increased update rate, which potentially improves quality even if coding is lossy. A low-order predictive coding is investigated for a pneumatic force display. Due to human and device characteristics, some compression is possible without loss, although the technique is lossy in general. Lossy uniform and nonuniform quantizers are also investigated. An experiment was conducted to determine how much data reduction is possible before compression artifacts become detectable to users.

Low-cost simulated MIG welding for advancement in technical training

The simulated MIG lab (sMIG) is a training simulator for Metal Inert Gas (MIG) welding. It is based on commercial off the shelf (COTS) components and targeted at familiarizing beginning students with the MIG equipment and best practices to follow to become competent and effective MIG welders. To do this, it simulates the welding process as realistically as possible using standard welding hardware components (helmet, gun) for input and by using head-tracking and a 3D-capable low-cost monitor and standard speakers for output. We developed a simulation to generate realistic audio and visuals based on numerical heat transfer methods and verified the accuracy against real welds. sMIG runs in real time producing a realistic, interactive, and immersive welding experience while maintaining a low installation cost. In addition to being realistic, the system provides instant feedback beyond what is possible in a traditional lab. This help students avoid learning (and unlearning) incorrect movement patterns.

Visual feedback for virtual grasping

We investigate visual feedback for virtual grasps, especially cues to improve behavior after real fingers enter a virtual object. To date, such visual cues have usually been developed in an ad-hoc manner, with minimal or no studies that can guide selection. Existing guidelines are based largely on other interaction types and provide inconsistent and potentially-misleading information when applied to grasping. We compare several different visual feedback types including those most commonly seen for virtual hand interaction and with some novel visual aspects. The visuals were tuned in a pilot study, and our main study evaluated results in terms of objective performance (finger penetration, release time, and precision) and subjective rankings. Performancewise, the most promising techniques all directly reveal penetrating hand configuration in some way. Subjectively, most techniques are better than simple interpenetrating visuals, with color changes being most promising. The results enable selection of the best cues based on the relevant tradeoffs. Results also provide a needed basis for more focused studies of specific visual cues and for better informing studies of multimodal feedback.

An evaluation of menu properties and pointing techniques in a projection-based VR environment

We studied menu performance for a rear-projected VR system. Our experiment considered layout (pie vs. linear list), placement (fixed vs. contextual), and pointing method (ray vs. alternatives that we call PAM: pointer-attached-to-menu). We also discuss results with respect to breadth (number of menu items) and depth (top-level and child menus). Standard ray pointing was usually faster than PAM, especially in second-level (child) menus, but error rates were lower for PAM in some cases. Subjective ratings were higher for ray-casting. Pie menus performed better than list layouts. Contextual pop-up menus were faster than fixed location.

Virtual Welder Trainer

The goal of this project is to develop a training system that can simulate the welding process in real-time and give feedback that avoids learning wrong motion patterns for beginning welders and can be used to analyze the process by the teacher afterwards. The system is based mainly on COTS components. A standard PC with a Dual-core CPU and a medium-end nVidia graphics card is sufficient. Input is done with a regular welding gun to allow realistic training. The gun is tracked by an OptiTrack system with 3 FLEX:V100 cameras. The same is also used to track a regular welding helmet to get accurate eye positions for display, which was chosen over glasses for robustness. The display itself is a Zalman Trimon stereo monitor that is laid out horizontally. The software is designed around a main simulation component for solving heat conduction on a grid of simulation points based on local GaussSeidel elimination.

Visual interpenetration tradeoffs in whole-hand virtual grasping

We present the first experiment on tradeoffs involving visual interpenetration in whole-hand virtual grasping, with new findings that contrast prior interpenetration research and provide a stronger understanding of user behavior and beliefs. Most notably, preventing interpenetration reduced performance by increasing real hand closure and reducing release precision. Nonetheless, most users subjectively prefer interpenetration to be prevented and even expect this to perform better. Although reduced closure and true representation of real hand pose might help users anticipate a grasp release moment more precisely, most subjects believed release moment was better anticipated with virtual fingers constrained to object surfaces. Finally, we suggest how grasping techniques can resolve the tradeoffs.

Bi-level and anti-aliased rendering methods for a low-resolution 2D vibrotactile array

We investigate rendering methods for 2D tactile arrays. We present four rendering methods, two of which are anti-aliased methods to improve display quality and smoothness. We describe three experiments to evaluate the methods. The first experiment compares them based on subjective ratings and finds that anti-aliased methods improve perceived quality and smoothness. The second experiment investigates the potential for one approach to communicate information at a sub-tactor resolution, based on the shortest line segment allowing identification of its direction. The third experiment is a shape discrimination experiment and finds that only one of the approaches allows reasonable discrimination for the selected shapes.