Fredrik Nysjö

Haptics-assisted Virtual Planning of Bone, Soft Tissue, and Vessels in Fibula Osteocutaneous Free Flaps.

Background: Virtual surgery planning has proven useful for reconstructing head and neck defects by fibula osteocutaneous free flaps (FOFF). Benefits include improved healing, function, and aesthetics, as well as cost savings. But available virtual surgery planning systems incorporating fibula in craniomaxillofacial reconstruction simulate only bone reconstruction without considering vessels and soft tissue. Methods: The Haptics-Assisted Surgery Planning (HASP) system incorporates bone, vessels, and soft tissue of the FOFF in craniomaxillofacial defect reconstruction. Two surgeons tested HASP on 4 cases they had previously operated on: 3 with composite mandibular defects and 1 with a composite cervical spine defect. With the HASP stereographics and haptic feedback, using patient-specific computed tomography angiogram data, the surgeons planned the 4 cases, including bone resection, fibula design, recipient vessels selection, pedicle and perforator location selection, and skin paddle configuration. Results: Some problems encountered during the actual surgery could have been avoided as they became evident with HASP. In one case, the fibula reconstruction was incomplete because the fibula had to be reversed and thus did not reach the temporal fossa. In another case, the fibula had to be rotated 180 degrees to correct the plate and screw placement in relation to the perforator. In the spinal case, difficulty in finding the optimal fibula shape and position required extra ischemia time. Conclusions: The surgeons found HASP to be an efficient planning tool for FOFF reconstructions. The testing of alternative reconstructions to arrive at an optimal FOFF solution preoperatively potentially improves patient function and aesthetics and reduces operating room time.

Snap-to-fit, a haptic 6 DOF alignment tool for virtual assembly

Virtual assembly of complex objects has application in domains ranging from surgery planning to archaeology. In these domains the objective is to plan the restoration of skeletal anatomy or archaeological artifacts to achieve an optimal reconstruction without causing further damage. While graphical modeling plays a central role in virtual assembly, visual feedback alone is often insufficient since object contact and penetration is difficult to discern due to occlusion. Haptics can improve an assembly task by giving feedback when objects collide, but precise fitting of fractured objects guided by delicate haptic cues similar to those present in the physical world requires haptic display transparency beyond the performance of todays systems. We propose a haptic alignment tool that combines a 6 Degrees of Freedom (DOF) attraction force with traditional 6 DOF contact forces to pull a virtual object towards a local stable fit with a fixed object. The object forces are integrated into a virtual coupling framework yielding a stable haptic tool. We demonstrate the use of our system on applications from both cranio-maxillofacial surgery and archaeology, and show that we can achieve haptic rates for fractured surfaces with over 5000 points.

Physically co-located haptic interaction with 3D displays

Studies indicate that haptic interaction with a computer generated virtual scene may become more intuitive by aligning (co-locating) the visual and haptic workspaces so that the visual and haptic feedback coincide as they do in the real world. Co-located haptics may gain importance when more advanced haptic interfaces, such as high-fidelity whole hand devices, become available. We describe a user study that investigates the pros and cons with physically co-located versus non-collocated haptics on two different display types: a commercial half-transparent mirror 3D display with shutter glasses and a prototype autostereoscopic display based on a Holographic Optical Element (HOE). We use two accuracy tasks with spatial accuracy as the dependent variable and one manipulation task with time as the dependent variable. The study shows that on both displays co-location significantly improves completion time in the manipulation task. However, the study shows that co-location does not improve the accuracy in the spatial accuracy tasks.

Rendering stiffness with a prototype haptic glove actuated by an integrated piezoelectric motor

Bi-directional haptic devices incorporate both sensors and actuators. While small and compact sensors are readily available, actuators in haptic interfaces require a significant volume to produce needed forces. With many actuated degrees of freedom, the mass and size of the actuators become a problem in devices such as haptic gloves. Piezo-technology offers the possibility of compact actuators which can be controlled with high accuracy. We describe a prototype admittance-type haptic device for the hand with a compact integrated piezoelectric motor. The current implementation provides one degree of freedom, but it could be extended with more motors for additional degrees of freedom. We demonstrate both the accuracy with which the device can reproduce force-displacement responses of non-linear elastic material stiffness and the devices fast and stable response to an applied load.

SplineGrip: an eight degrees-of-freedom flexible haptic sculpting tool

Haptic sculpting allows a designer to create virtual models with the aid of haptic feedback. Just as in real life sculpting, different tools are used to work the model into a desired shape. Several haptic sculpting tools appear in the literature. For example, [Gao 2006] proposes an ellipsoidal tool whose pose (position and orientation) is controlled by a six degrees-of-freedom (DOF) haptic device, and [McDonnell 2001] describes a set of modeling tools for virtual clay controlled by three DOF haptics.

ProViz: a tool for explorative 3-D visualization and template matching in electron tomograms

Visual understanding is a key aspect when studying electron tomography data-sets, aside quantitative assessments such as registration of high-resolution structures. We here present the free software tool ProViz (Protein Visualization) for visualisation and template matching in electron tomograms of biological samples. The ProViz software contains methods and tools which we have developed, adapted and computationally optimised for easy and intuitive visualisation and analysis of electron tomograms with low signal-to-noise ratio. ProViz complements existing software in the application field and serves as an easy and convenient tool for a first assessment and screening of the tomograms. It provides enhancements in three areas: (1) improved visualisation that makes connections as well as intensity differences between and within objects or structures easier to see and interpret, (2) interactive transfer function editing with direct visual result feedback using both piecewise linear functions and Gaussian function elements, (3) computationally optimised template matching and tools to visually assess and interactively explore the correlation results. The visualisation capabilities and features of ProViz are demonstrated on various biological volume data-sets: bacterial filament structures in vitro, a desmosome and the transmembrane cadherin connections therein in situ, and liposomes filled with doxorubicin in solution. The explorative template matching is demonstrated on a synthetic IgG data-set.

Visuohaptic bone saw simulator: combining vibrotactile and kinesthetic feedback

The combination of stereo visualization and haptics provides a natural interface for surgical training simulators, an application which is inherently both highly visual and highly tactile. However, most off-the-shelf kinesthetic haptic devices, such as the popular Phantom devices, are not well-suited to display high-fidelity vibrotactile feedback for the high frequency force components in surgical tools such as a reciprocating bone saw. In these haptic devices, forces are mediated from the actuators to the user through a mechanical linkage, in which inertia, friction, and backlash may distort the feedback. In addition, sustained display of vibrations may cause undue wear of the device. We propose a hybrid solution combining kinesthetic feedback from an off-the-shelf haptic device with high-fidelity vibration feedback from a vibrotactile actuator, and show that the hybrid is able to reproduce vibrations of an actual surgical reciprocating saw within the full perceptible frequency range.

Custom Mandibular Implant Design with Deformable Models and Haptics

Purpose The use of a surgical template for computer-aided oral implantology is an effective method that can help achieve the preoperative plan exactly. Currently, several commercial software packages in this field have been developed, including SimPlant (Materialise, Belgium), NobelGuide (Nobel Biocare, USA), etc. However, most of those software packages just focus on preoperative planning. In this study, a surgical template design software for oral implantology is developed so that surgical guides can be designed and fabricated by dental clinics instead of the commercial companies through 3D printing technology. In addition, template design can be optimized conveniently, and reduction in time for delivery and cost consuming is realized. Methods The general framework of this software is as follows (Fig. 1a). Firstly, with the CT(or conebeam CT) data of a patient, the preoperative planning is achieved using the software of CAPPOIS (Computer-Assisted Preoperative Planning for Oral Implant Surgery) [1]. Then, a stereolithography (STL) model of dentition is generated by registration, which means superimposing the three-dimensional laser-scanned model of plaster casts of dentition onto the threedimensional skull model reconstructed from CT images. Subsequently, original control points are indicated to generate a curve, so that the target region can be clipped out as the inner surface of template. The positions of those points can also be adjusted for the preview of the approximate clipping result. After that, an initial template without drilling tubes will be generated. Then, several parameters including endpoint coordinates of axes of the virtual implants will be imported from the preoperative planning with CAPPOIS. The final STL model of the surgical template is achieved using algorithm of Boolean operation so that it can be fabricated through 3D printing technology. The general flowchart of the software is shown in Fig. 1b, concerning three critical procedures, which are respectively (1) clipping surface with points in a loop, (2) surface offset, and (3)collision detection. For procedure (1), clipping surface with points is achieved in the following procedure. Because surface geometry is presented in mesh in our algorithm, vertices and edges are feature elements for surface. To begin with, vertices closest to initial control points in a loop are ordered in a list, namely list0. Then another copy of list0, named list1, will be updated as follows. For each vertex in list1 from the first to the last one, all neighbor vertices are traversed and one will be added to list1 only if it has not been yet in list1, and the two expressions below are satisfied i [ 2 0 ; 90 di dj; j 2 1; n 1⁄2

SoftCut: A Virtual Planning Tool for Soft Tissue Resection on CT Images

With the increasing use of three-dimensional (3D) models and Computer Aided Design (CAD) in the medical domain, virtual surgical planning is now frequently used. Most of the current solutions focus on bone surgical operations. However, for head and neck oncologic resection, soft tissue ablation and reconstruction are common operations. In this paper, we propose a method to provide a fast and efficient estimation of shape and dimensions of soft tissue resections. Our approach takes advantage of a simple sketch-based interface which allows the user to paint the contour of the resection on a patient specific 3D model reconstructed from a computed tomography (CT) scan. The volume is then virtually cut and carved following this pattern. From the outline of the resection defined on the skin surface as a closed curve, we can identify which areas of the skin are inside or outside this shape. We then use distance transforms to identify the soft tissue voxels which are closer from the inside of this shape. Thus, we can propagate the shape of the resection inside the soft tissue layers of the volume. We demonstrate the usefulness of the method on patient specific CT data.

Comparison of Walking and Traveling-Wave Piezoelectric Motors as Actuators in Kinesthetic Haptic Devices

Piezoelectric motors offer an attractive alternative to electromagnetic actuators in portable haptic interfaces: they are compact, have a high force-to-volume ratio, and can operate with limited or no gearing. However, the choice of a piezoelectric motor type is not obvious due to differences in performance characteristics. We present our evaluation of two commercial, operationally different, piezoelectric motors acting as actuators in two kinesthetic haptic grippers, a walking quasi-static motor and a traveling wave ultrasonic motor. We evaluate each grippers ability to display common virtual objects including springs, dampers, and rigid walls, and conclude that the walking quasi-static motor is superior at low velocities. However, for applications where high velocity is required, traveling wave ultrasonic motors are a better option.