Steven D. Pieper

Hands-on interaction with virtual environments

In this paper we describe the evolution of a whole-hand interface to our virtual-environment graphical system. We present a set of abstractions that can be used to implement device-independent interfaces for hand measurement devices. Some of these abstractions correspond to known logical device abstractions, while others take further advantage of the richness of expression in the human hand. We describe these abstractions in the context of their use in our development of virtual environments.

Interactive graphics for plastic surgery: a task-level analysis and implementation

We have implemented a system for Computer-Aided Plastic Surgery. Planning plastic surgery procedures is complex because the surgeon needs to stretch and reshape the patient’s skin to replace missing tissue while minimizing distortion of the surrounding tissue. Traditional planning techniques rely on the surgeon’s experience to select among a myriad of possible procedure designs. While mathematica1 techniques for predicting the outcome of surgery have been proposed in the past, these are not in widespread use by surgeons because they require the surgeon to perform manual constructions and geometric calculations. Our system makes the analysis process easier by allowing the surgeon to draw the surgical plan directly on a 3D model of the patient. An automatic mesh generator is used to convert that drawing into a well-formulated problem for finite element analysis.

A finite-element facial model for simulating plastic surgery

CAPS (Computer-Aided Plastic Surgery) is a prototype computer program that uses a three-dimensional graphic model of a human face and incorporates a finite-element mathematical model of the physical properties of the soft tissue. This program can estimate the biomechanic consequences of ablation and rearrangement of tissue. The results of two hypothetical surgeries on the face are presented: A surgeon could use this program as a sketch pad to predict and compare the outcome of facial plastic procedures on a patient-specific model. The relation of this program to previous work is discussed, and directions for research and possible applications are addressed.

Implementation and integration of a counterbalanced CRT-based stereoscopic display for interactive viewpoint control in virtual-environment applications

This paper describes the implementation and integration of the Ames counterbalanced CRT-based stereoscopic viewer (CCSV). The CCSV was developed as a supplementary viewing device for the Virtual Interface Environment Workstation project at NASA Ames in order to provide higher resolution than is currently possible with LCD based head-mounted viewers. The CCSV is currently used as the viewing device for a biomechanical CAD environment which we feel is typical of the applications for which the CCSV is appropriate. The CCSV also interfaces to a remote stereo camera platform. The CCSV hardware consists of a counterbalanced kinematic linkage, dual-CRT based stereoscopic viewer with wide angle optics, video electronics box, dedicated microprocessor system monitoring joint angles in the linkage, host computer interpreting the sensor values and running the application which renders right and left views for the viewers CRTs. CCSV software includes code resident on the microprocessor system, host computer device drivers to communicate with the microprocessor, a kinematic module to compute viewer position and orientation from sensor values, graphics routines to change the viewing geometry to match viewer optics and movements, and an interface to the application. As a viewing device, the CCSV approach is particularly well suited to applications in which 1) the user moves back and forth between virtual environment viewing and desk work, 2) high resolution views of the virtual environment are required or 3) the viewing device is to be shared among collaborators in a group setting. To capitalize on these strengths, planned improvements for future CCSVs include: defining an appropriate motion envelope for desk top applications, improving the feel of the kinematics within that envelope, improving realism of the display by adding color and increasing the spatial resolution, reducing lag, and developing interaction metaphors within the 3D environment.

Virtual environment system for simulation of leg surgery

A virtual environment system has been developed for viewing and manipulating a model of the human leg. The model can be used to simulate the biomechanical consequences of various reconstructive surgical procedures. Previously, the model was implemented on a standard engineering workstation, and interaction was limited to a mouse and screen cursor. By incorporating the leg model into a virtual environment, the authors were able to assess the value of a head-coupled stereo display and direct 3-D manipulation for a surgery simulation application. This application is an interesting test case for a virtual environment because it requires visualization and manipulation of complex 3-D geometries. Since the model can be used as the basis for a number of biomechanical analyses, the virtual environment provides an opportunity to visualize the resulting datasets in the context of the 3-D model. The components used in assembling the system are described the design and implementation of this system is discussed, and a set of interface techniques that allow direct 3-D interaction with the model is presented.

Control of a virtual actor: the roach

We have developed a virtual environment system which supports multiple simulations, including virtual actors. These actors exhibit motor behavior in response to activity in the environment. We present an example actor, whose low-level behavior is modeled after physiological analyses of cockroach motor behavior. The sensori-motor activity of our roach is generated by a hierarchical control structure. Coupled oscillators generate basic gait patterns, which are modified by reflexes feeding in from the environment. Stepping and stance are executed by kinematic motor programs, which move the legs and body. The reactive level associates motor behavior with events in the virtual environment, to simulate perception and implement higher level behaviors. The activity of the virtual actor is determined only when it is situated in the environment, and interacts with the user and other simulations.

Computer animation for minimally invasive surgery: computer system requirements and preferred implementations

We are interested in the application of computer animation to surgery. Our current project, a navigation and visualization tool for knee arthroscopy, relies on real-time computer graphics and the human interface technologies associated with virtual reality. We believe that this new combination of techniques will lead to improved surgical outcomes and decreased health care costs. To meet these expectations in the medical field, the system must be safe, usable, and cost-effective. In this paper, we outline some of the most important hardware and software specifications in the areas of video input and output, spatial tracking, stereoscopic displays, computer graphics models and libraries, mass storage and network interfaces, and operating systems. Since this is a fairly new combination of technologies and a new application, our justification for our specifications are drawn from the current generation of surgical technology and by analogy to other fields where virtual reality technology has been more extensively applied and studied.