Michael Figl

A head-mounted operating binocular for augmented reality visualization in medicine - design and initial evaluation

Computer-aided surgery (CAS), the intraoperative application of biomedical visualization techniques, appears to be one of the most promising fields of application for augmented reality (AR), the display of additional computer-generated graphics over a real-world scene. Typically a device such as a head-mounted display (HMD) is used for AR. However, considerable technical problems connected with AR have limited the intraoperative application of HMDs up to now. One of the difficulties in using HMDs is the requirement for a common optical focal plane for both the realworld scene and the computer-generated image, and acceptance of the HMD by the user in a surgical environment. In order to increase the clinical acceptance of AR, we have adapted the Varioscope (Life Optics, Vienna), a miniature, cost-effective head-mounted operating binocular, for AR. In this paper, we present the basic design of the modified HMD, and the method and results of an extensive laboratory study for photogrammetric calibration of the Varioscopes computer displays to a real-world scene. In a series of 16 calibrations with varying zoom factors and object distances, mean calibration error was found to be 1.24 /spl plusmn/ 0.38 pixels or 0.12 /spl plusmn/ 0.05 mm for a 640 /spl times/ 480 display. Maximum error accounted for 3.33 /spl plusmn/ 1.04 pixels or 0.33 /spl plusmn/ 0.12 mm. The location of a position measurement probe of an optical tracking system was transformed to the display with an error of less than 1 mm in the real world in 56% of all cases. For the remaining cases, error was below 2 mm. We conclude that the accuracy achieved in our experiments is sufficient for a wide range of CAS applications.

Design and application of an assessment protocol for electromagnetic tracking systems.

This paper defines a simple protocol for competitive and quantified evaluation of electromagnetic tracking systems such as the NDI Aurora (A) and Ascension microBIRD with dipole transmitter (B). It establishes new methods and a new phantom design which assesses the reproducibility and allows comparability with different tracking systems in a consistent environment. A machined base plate was designed and manufactured in which a 50 mm grid of holes was precisely drilled for position measurements. In the center a circle of 32 equispaced holes enables the accurate measurement of rotation. The sensors can be clamped in a small mount which fits into pairs of grid holes on the base plate. Relative positional/orientational errors are found by subtracting the known distances/rotations between the machined locations from the differences of the mean observed positions/rotation. To measure the influence of metallic objects we inserted rods made of steel (SST 303, SST 416), aluminum, and bronze into the sensitive volume between sensor and emitter. We calculated the fiducial registration error and fiducial location error with a standard stylus calibration for both tracking systems and assessed two different methods of stylus calibration. The positional jitter amounted to 0.14 mm(A) and 0.08 mm(B). A relative positional error of 0.96mm±0.68mm, range -0.06 mm; 2.23 mm(A) and 1.14mm±0.78mm, range -3.72 mm; 1.57 mm(B) for a given distance of 50 mm was found. The relative rotation error was found to be 0.51° (A)/0.04° (B). The most relevant distortion caused by metallic objects results from SST 416. The maximum error 4.2mm(A)∕⩾100mm(B) occurs when the rod is close to the sensor(20 mm). While (B) is more sensitive with respect to metallic objects, (A) is less accurate concerning orientation measurements. (B) showed a systematic error when distances are calculated.

Evaluation of a miniature electromagnetic position tracker

The advent of miniaturized electromagnetic digitizers opens a variety of potential clinical applications for computer aided interventions using flexible instruments; endoscopes or catheters can easily be tracked within the body. With respect to the new applications, the systematic distortions induced by various materials such as closed metallic loops, wire guides, catheters, and ultrasound scan heads were systematically evaluated in this paper for a new commercial tracking system. We employed the electromagnetic tracking system Aurora (Mednetix/CH, NDI/Can); data were acquired using the serial port of a PC running SuSE Linux 7.1 (SuSE, Gmbh, Nürnberg). Objects introduced into the digitizer volume included wire loops of different diameters, wire guides, optical tracking tools, an ultrasonic (US) scan head, an endoscope with radial ultrasound scan head and various other objects used in operating rooms and interventional suites. Beyond this, we determined the influence of a C-arm fluoroscopy unit. To quantify the reliability of the system, the miniaturized sensor was mounted on a nonmetallic measurement rack while the transmitter was fixed at three different distances within the digitizer range. The tracker was shown to be more sensitive to distortions caused by materials close to the emitter (average distortion error 13.6 mm +/- 16.6 mm for wire loops positioned at a distance between 100 mm and 200 mm from the emitter). Distortions caused by materials near the sensor (distances smaller than 100 mm) are small (typical error 2.2 mm +/- 1.9 mm). The C-arm fluoroscopy unit caused considerable distortions and limits the reliability of the tracker (distortion error 18.6 mm +/- 24.9 mm). Distortions resulting from the US scan head are high at distances smaller than about 100 mm from the emitter. The distortions also increase when the scan head is positioned horizontally and close to the sensor (average error 4.1 mm +/- 1.5 mm when the scan head is positioned within a distance of 100 mm from the sensor). The distortions are slightly higher when the ultrasound machine is switched on. We also evaluated the influence of common medical instruments on distance measurements. For these measurements the average deviation from the known distance of 200 mm amounted to 3.0 mm +/- 1.5 mm (undistorted distance measurement 1.5 mm +/- 0.3 mm). The deviations also depend on the relative orientation between emitter and sensor. The results demonstrate that the miniature tracking system opens up new perspectives with regard to surgery applications where a flexible instrument is to be tracked within the body. Significant distortions caused by metallic objects only occur in the worst cases, for example, in the presence of a closed, unisiolated wire loop or a C-arm fluorescence unit close to the emitter and which can be avoided by suitable usage.

Stability of miniature electromagnetic tracking systems

This study aims at a comparative evaluation of two recently introduced electromagnetic tracking systems under reproducible simulated operating-room (OR) conditions: the recently launched Medtronic StealthStation, Treon-EM and the NDI Aurora. We investigate if and to what extent these systems provide improved performance and stability in the presence of surgical instruments as possible sources of distortions compared with earlier reports on electromagnetic tracking technology. To investigate possible distortions under pseudo-realistic OR conditions, a large Langenbeck hook, a dental drill with its handle and an ultrasonic (US) scanhead are fixed on a special measurement rack at variable distances from the navigation sensor. The position measurements made by the Treon-EM were least affected by the presence of the instruments. The lengths of the mean deviation vectors were 0.21 mm for the Langenbeck hook, 0.23 mm for the drill with handle and 0.56 mm for the US scanhead. The Aurora was influenced by the three sources of distortion to a higher degree. A mean deviation vector of 1.44 mm length was observed in the vicinity of the Langenbeck hook, 0.53 mm length with the drill and 2.37 mm due to the US scanhead. The maximum of the root mean squared error (RMSE) for all coordinates in the presence of the Langenbeck hook was 0.3 mm for the Treon and 2.1 mm for the Aurora; the drill caused a maximum RMSE of 0.2 mm with the Treon and 1.2 mm with the Aurora. In the presence of the US scanhead, the maximum RMSE was 1.4 mm for the Treon and 5.1 mm for the Aurora. The new generation of electromagnetic tracking systems has significantly improved compared to common systems that were available in the middle of the 1990s and has reached a high level of technical development. We conclude that, in general, both systems are suitable for routine clinical application.

Wobbled splatting—a fast perspective volume rendering method for simulation of x-ray images from CT

3D/2D registration, the automatic assignment of a global rigid-body transformation matching the coordinate systems of patient and preoperative volume scan using projection images, is an important topic in image-guided therapy and radiation oncology. A crucial part of most 3D/2D registration algorithms is the fast computation of digitally rendered radiographs (DRRs) to be compared iteratively to radiographs or portal images. Since registration is an iterative process, fast generation of DRRs-which are perspective summed voxel renderings-is desired. In this note, we present a simple and rapid method for generation of DRRs based on splat rendering. As opposed to conventional splatting, antialiasing of the resulting images is not achieved by means of computing a discrete point spread function (a so-called footprint), but by stochastic distortion of either the voxel positions in the volume scan or by the simulation of a focal spot of the x-ray tube with non-zero diameter. Our method generates slightly blurred DRRs suitable for registration purposes at framerates of approximately 10 Hz when rendering volume images with a size of 30 MB.

Evaluation of a new electromagnetic tracking system using a standardized assessment protocol

This note uses a published protocol to evaluate a newly released 6 degrees of freedom electromagnetic tracking system (Aurora, Northern Digital Inc.). A practice for performance monitoring over time is also proposed. The protocol uses a machined base plate to measure relative error in position and orientation as well as the influence of metallic objects in the operating volume. Positional jitter (E(RMS)) was found to be 0.17 mm +/- 0.19 mm. A relative positional error of 0.25 mm +/- 0.22 mm at 50 mm offsets and 0.97 mm +/- 1.01 mm at 300 mm offsets was found. The mean of the relative rotation error was found to be 0.20 degrees +/- 0.14 degrees with respect to the axial and 0.91 degrees +/- 0.68 degrees for the longitudinal rotation. The most significant distortion caused by metallic objects is caused by 400-series stainless steel. A 9.4 mm maximum error occurred when the rod was closest to the emitter, 10 mm away. The improvement compared to older generations of the Aurora with respect to accuracy is substantial.

Computer-enhanced stereoscopic vision in a head-mounted operating binocular

Based on the Varioscope, a commercially available head-mounted operating binocular, we have developed the Varioscope AR, a see through head-mounted display (HMD) for augmented reality visualization that seamlessly fits into the infrastructure of a surgical navigation system. We have assessed the extent to which stereoscopic visualization improves target localization in computer-aided surgery in a phantom study. In order to quantify the depth perception of a user aiming at a given target, we have designed a phantom simulating typical clinical situations in skull base surgery. Sixteen steel spheres were fixed at the base of a bony skull, and several typical craniotomies were applied. After having taken CT scans, the skull was filled with opaque jelly in order to simulate brain tissue. The positions of the spheres were registered using VISIT, a system for computer-aided surgical navigation. Then attempts were made to locate the steel spheres with a bayonet probe through the craniotomies using VISIT and the Varioscope AR as a stereoscopic display device. Localization of targets 4 mm in diameter using stereoscopic vision and additional visual cues indicating target proximity had a success rate (defined as a first-trial hit rate) of 87.5%. Using monoscopic vision and target proximity indication, the success rate was found to be 66.6%. Omission of visual hints on reaching a target yielded a success rate of 79.2% in the stereo case and 56.25% with monoscopic vision. Time requirements for localizing all 16 targets ranged from 7.5 min (stereo, with proximity cues) to 10 min (mono, without proximity cues). Navigation error is primarily governed by the accuracy of registration in the navigation system, whereas the HMD does not appear to influence localization significantly. We conclude that stereo vision is a valuable tool in augmented reality guided interventions.

Monitoring tumor motion by real time 2D/3D registration during radiotherapy

Background and purpose In this paper, we investigate the possibility to use X-ray based real time 2D/3D registration for non-invasive tumor motion monitoring during radiotherapy. Materials and methods The 2D/3D registration scheme is implemented using general purpose computation on graphics hardware (GPGPU) programming techniques and several algorithmic refinements in the registration process. Validation is conducted off-line using a phantom and five clinical patient data sets. The registration is performed on a region of interest (ROI) centered around the planned target volume (PTV). Results The phantom motion is measured with an rms error of 2.56 mm. For the patient data sets, a sinusoidal movement that clearly correlates to the breathing cycle is shown. Videos show a good match between X-ray and digitally reconstructed radiographs (DRR) displacement. Mean registration time is 0.5 s. Conclusions We have demonstrated that real-time organ motion monitoring using image based markerless registration is feasible.

Rigid 2D/3D slice‐to‐volume registration and its application on fluoroscopic CT images

Registration of single slices from FluoroCT, CineMR, or interventional magnetic resonance imaging to three dimensional (3D) volumes is a special aspect of the two-dimensional (2D)/3D registration problem. Rather than digitally rendered radiographs (DRR), single 2D slice images obtained during interventional procedures are compared to oblique reformatted slices from a high resolution 3D scan. Due to the lack of perspective information and the different imaging geometry, convergence behavior differs significantly from 2D/3D registration applications comparing DRR images with conventional x-ray images. We have implemented a number of merit functions and local and global optimization algorithms for slice-to-volume registration of computed tomography (CT) and FluoroCT images. These methods were tested on phantom images derived from clinical scans for liver biopsies. Our results indicate that good registration accuracy in the range of 0.50 and 1.0 mm is achievable using simple cross correlation and repeated application of local optimization algorithms. Typically, a registration took approximately 1 min on a standard personal computer. Other merit functions such as pattern intensity or normalized mutual information did not perform as well as cross correlation in this initial evaluation. Furthermore, it appears as if the use of global optimization algorithms such as simulated annealing does not improve reliability or accuracy of the registration process. These findings were also confirmed in a preliminary registration study on five clinical scans. These experiments have, however, shown that a strict breath-hold protocol is inevitable when using rigid registration techniques for lesion localization in image-guided biopsy retrieval. Finally, further possible applications of slice-to-volume registration are discussed.

Validation for 2D/3D registration. I: A new gold standard data set.

PURPOSE In this article, the authors propose a new gold standard data set for the validation of two-dimensional/three-dimensional (2D/3D) and 3D/3D image registration algorithms. METHODS A gold standard data set was produced using a fresh cadaver pig head with attached fiducial markers. The authors used several imaging modalities common in diagnostic imaging or radiotherapy, which include 64-slice computed tomography (CT), magnetic resonance imaging using T1, T2, and proton density sequences, and cone beam CT imaging data. Radiographic data were acquired using kilovoltage and megavoltage imaging techniques. The image information reflects both anatomy and reliable fiducial marker information and improves over existing data sets by the level of anatomical detail, image data quality, and soft-tissue content. The markers on the 3D and 2D image data were segmented using ANALYZE 10.0 (AnalyzeDirect, Inc., Kansas City, KN) and an in-house software. RESULTS The projection distance errors and the expected target registration errors over all the image data sets were found to be less than 2.71 and 1.88 mm, respectively. CONCLUSIONS The gold standard data set, obtained with state-of-the-art imaging technology, has the potential to improve the validation of 2D/3D and 3D/3D registration algorithms for image guided therapy.