Philipp Jauer

Towards X-ray free endovascular interventions – using HoloLens for on-line holographic visualisation

A major challenge during endovascular interventions is visualising the position and orientation of the catheter being inserted. This is typically achieved by intermittent X-ray imaging. Since the radiation exposure to the surgeon is considerable, it is desirable to reduce X-ray exposure to the bare minimum needed. Additionally, transferring two-dimensional (2D) X-ray images to 3D locations is challenging. The authors present the development of a real-time navigation framework, which allows a 3D holographic view of the vascular system without any need of radiation. They extract the patients surface and vascular tree from pre-operative computed tomography data and register it to the patient using a magnetic tracking system. The system was evaluated on an anthropomorphic full-body phantom by experienced clinicians using a four-point questionnaire. The average score of the system (maximum of 20) was found to be 17.5. The authors’ approach shows great potential to improve the workflow for endovascular procedures, by simultaneously reducing X-ray exposure. It will also improve the learning curve and help novices to more quickly master the required skills.

Real-time 4D ultrasound visualization with the Voreen framework

Voreen is an open source volume rendering engine which allows interactive visualization of volumetric data sets with high flexibility when integrating new visualization techniques [Meyer-Spradow et al. 2009]. It is designed with minimal overhead, so that, even for difficult processing networks, high rendering speeds can be achieved. While Voreen is well-equipped to display Cartesian volumes, volumes in polar coordinates, such as raw ultrasound beam data, currently have to be interpolated and converted to Cartesian coordinates to be displayed. This is highly time-consuming on a CPU while interpolation and volume rendering can be done on a graphics card in real-time [Sumanaweera 2004].

Robust inverse kinematics by configuration control for redundant manipulators with seven DoF

This paper presents an optimized, robust inverse kinematics solution in a closed form for redundant manipulators with seven degrees of freedom and zero link offsets like the KUKA LBR iiwa lightweight robot. The computation allows for full range elbow self-motion manifold control, defined by an intuitive angle parameter. Furthermore, by using common configuration parameters, all possible solutions for a single effector pose are taken into account and full arm configuration control is realized. The algorithm was evaluated looking at configuration reliability and continuity for an exemplary, challenging path. The results show that commanded arm configurations are not changing spontaneously inside a trajectory. Even for varying elbow self-motion manifold angles, the joint trajectories stay continuous and the presented algorithm provides consistent solutions. In a second evaluation, the algorithm was implemented in three programming languages and analyzed with respect to computing times. The tests demonstrate short runtimes and overall real-time capability.

MO‐D‐144‐02: Ultrasound Transducer Localization Using the CyberKnife's X‐Ray System

PURPOSE 4D ultrasound has become an alternative for image guidance and motion compensation in radiosurgery. Nevertheless, a two-step localization has to be performed when using ultrasound. In addition to target localization inside the ultrasound volume, the transducer itself has to be localized and the target position has to be transformed into treatment coordinates. METHODS The CyberKnife (Accuray Inc.) features a stereo X-ray system which is used for patient and fiducial marker localization. Accessing only the raw X-ray images, we designed a software package for additional marker detection. To measure the physical camera parameters, X-ray phantoms were designed and a non-orthogonal stereo camera calibration was performed on 50 calibration X-ray pairs. Algorithms were developed for three-dimensional single marker localization and six-dimensional localization of marker geometries. Different X-ray marker geometries were designed and attached to the ultrasound transducer. The final system was evaluated and compared to the on-board localization system using a 6-axis industrial robot to position different marker geometries at 150 randomly distributed positions over the acquisition volume. RESULTS Two marker geometries with 20 and 50mm base lengths were localized using the on-board and newly developed software. The mean translational error for the on-board localization is 0.202mm, for the new system 0.218mm. The rotational error of the on-board system could be reduced from 0.212 to 0.053 degrees for the 50mm marker and from 0.272 to 0.076 degrees for the 20mm marker geometry. CONCLUSION We have shown that ultrasound transducers can be localized using the CyberKnife x-ray system. The rotational accuracy of the localization could be increased by a factor of four, which is important for high marker-to-ultrasound-target distances. Furthermore, using the full area of the flat-panel detectors without pre-processing steps the tracking volume was increased by 70 percent, which helps detecting patient/fiducials and transducer at the same time.

Global Data Storage for Collision Avoidance in Robotic Sailboat Racing – the World Server Approach

Collision avoidance is very important for autonomous sailing with many boats, e.g., during races. However, collision detection based on sensor data is complicated by the sails and boat motion. Particularly small boats cannot be equipped with sophisticated sensors, e.g., due to weight and power limitations. One approach to overcome this problem is to collect and store data from all participating vessels in a central data store. This World Server then provides the data to all boats, i.e., all participants in a race have access to a global view of the race situation. We present our basic server implementation and first test results indicating that the approach allows implementing and testing collision avoidance without the need for bulky and expensive sensors.

GPU-based real-time 3D workspace generation of arbitrary serial manipulators

This paper presents a method to solve - in real time - the three dimensional workspace generation problem for arbitrary serial manipulators. Our approach is based on Monte Carlo simulation, to process a high number of forward kinematics with randomly chosen joint values. This results in an asymptotic coverage of the reachable workspace. Additionally, collision detection is integrated to consider obstacles within the manipulators environment. The method is implemented on the graphics processing unit (GPU), such that an extremely high number of workspace points can be processed in parallel. Tests have shown that this approach is capable to generate acceptable workspace coverage within milliseconds. Furthermore, the workspace is held as a three dimensional texture volume on the graphics memory, allowing for instant visualisation of the workspace during the generation process without the need for further time-intensive data exchange.

An improved tracking framework for ultrasound probe localization in image-guided radiosurgery

Abstract Real-time target localization with ultrasound holds high potential for image guidance and motion compensation in radiosurgery due to its non-invasive image acquisition free from ionizing radiation. However, a two-step localization has to be performed when integrating ultrasound into the existing radiosurgery workflow. In addition to target localization inside the ultrasound volume, the probe itself has to be localized in order to transform the target position into treatment room coordinates. By adapting existing camera calibration tools, we have developed a method to extend the stereoscopic X-ray tracking system of a radiosurgery platform in order to locate objects such as marker geometries with six degrees of freedom. The calibration was performed with 0.1 mm reprojection error. By using the full area of the flat-panel detectors without pre-processing the extended software increased the tracking volume and resolution by up to 80%, substantially improving patient localization and marker detectability. Furthermore, marker-tracking showed sub-millimeter accuracy and rotational errors below 0.1°. This demonstrates that the developed extension framework can accurately localize marker geometries using an integrated X-ray system, establishing the link for the integration of real-time ultrasound image guidance into the existing system.

Robots with seven degrees of freedom: Is the additional DoF worth it?

This paper presents a detailed dexterity analysis of three different kinematics, comparing six-DoF with seven-DoF robots. The KUKA LBR iiwa 7 lightweight robot represents the seven-DoF kinematics, while the KUKA KR 10 is chosen as a common six-DoF robot with comparable workspace dimensions. Furthermore, the LBR iiwa is simulated as a six-jointed robot to allow for a direct comparison of seven- and six-DoF kinematics. Due to this simulation it is possible to isolate the effects of the additional 7th DoF for the first time. The results show that the average dexterity is increased by 16.8% due to the additional 7th joint. Compared to the highest average dexterity of the seven-DoF manipulator, the six-DoF KR 10 outperforms this value by 7.6%.

SU‐G‐JeP3‐08: Robotic System for Ultrasound Tracking in Radiation Therapy

PURPOSE For safe and accurate real-time tracking of tumors for IGRT using 4D ultrasound, it is necessary to make use of novel, high-end force-sensitive lightweight robots designed for human-machine interaction. Such a robot will be integrated into an existing robotized ultrasound system for non-invasive 4D live tracking, using a newly developed real-time control and communication framework. METHODS The new KUKA LWR iiwa robot is used for robotized ultrasound real-time tumor tracking. Besides more precise probe contact pressure detection, this robot provides an additional 7th link, enhancing the dexterity of the kinematic and the mounted transducer. Several integrated, certified safety features create a safe environment for the patients during treatment. However, to remotely control the robot for the ultrasound application, a real-time control and communication framework has to be developed. Based on a client/server concept, client-side control commands are received and processed by a central server unit and are implemented by a client module running directly on the robots controller. Several special functionalities for robotized ultrasound applications are integrated and the robot can now be used for real-time control of the image quality by adjusting the transducer position, and contact pressure. The framework was evaluated looking at overall real-time capability for communication and processing of three different standard commands. RESULTS Due to inherent, certified safety modules, the new robot ensures a safe environment for patients during tumor tracking. Furthermore, the developed framework shows overall real-time capability with a maximum average latency of 3.6 ms (Minimum 2.5 ms; 5000 trials). CONCLUSION The novel KUKA LBR iiwa robot will advance the current robotized ultrasound tracking system with important features. With the developed framework, it is now possible to remotely control this robot and use it for robotized ultrasound tracking applications, including image quality control and target tracking.

GPU-based real-time generation of large ultrasound volumes from freehand 3D sweeps

Abstract In the recent past, 3D ultrasound has been gaining relevance in many biomedical applications. One main limitation, however, is that typical ultrasound volumes are either very poorly resolved or only cover small areas. We have developed a GPU-accelerated method for live fusion of freehand 3D ultrasound sweeps to create one large volume. The method has been implemented in CUDA and is capable of generating an output volume with 0.5 mm resolution in real time while processing more than 45 volumes per second, with more than 300.000 voxels per volume. First experiments indicate that large structures like a whole forearm or high-resolution volumes of smaller structures like the hand can be combined efficiently. It is anticipated that this technology will be helpful in pediatric surgery where X-ray or CT imaging is not always possible.