Markus Gall

HTC Vive MeVisLab integration via OpenVR for medical applications

Virtual Reality, an immersive technology that replicates an environment via computer-simulated reality, gets a lot of attention in the entertainment industry. However, VR has also great potential in other areas, like the medical domain, Examples are intervention planning, training and simulation. This is especially of use in medical operations, where an aesthetic outcome is important, like for facial surgeries. Alas, importing medical data into Virtual Reality devices is not necessarily trivial, in particular, when a direct connection to a proprietary application is desired. Moreover, most researcher do not build their medical applications from scratch, but rather leverage platforms like MeVisLab, MITK, OsiriX or 3D Slicer. These platforms have in common that they use libraries like ITK and VTK, and provide a convenient graphical interface. However, ITK and VTK do not support Virtual Reality directly. In this study, the usage of a Virtual Reality device for medical data under the MeVisLab platform is presented. The OpenVR library is integrated into the MeVisLab platform, allowing a direct and uncomplicated usage of the head mounted display HTC Vive inside the MeVisLab platform. Medical data coming from other MeVisLab modules can directly be connected per drag-and-drop to the Virtual Reality module, rendering the data inside the HTC Vive for immersive virtual reality inspection.

Integration of the HTC Vive into the medical platform MeVisLab

Virtual Reality (VR) is an immersive technology that replicates an environment via computer-simulated reality. VR gets a lot of attention in computer games but has also great potential in other areas, like the medical domain. Examples are planning, simulations and training of medical interventions, like for facial surgeries where an aesthetic outcome is important. However, importing medical data into VR devices is not trivial, especially when a direct connection and visualization from your own application is needed. Furthermore, most researcher don’t build their medical applications from scratch, rather they use platforms, like MeVisLab, Slicer or MITK. The platforms have in common that they integrate and build upon on libraries like ITK and VTK, further providing a more convenient graphical interface to them for the user. In this contribution, we demonstrate the usage of a VR device for medical data under MeVisLab. Therefore, we integrated the OpenVR library into MeVisLab as an own module. This enables the direct and uncomplicated usage of head mounted displays, like the HTC Vive under MeVisLab. Summarized, medical data from other MeVisLab modules can directly be connected per drag-and-drop to our VR module and will be rendered inside the HTC Vive for an immersive inspection.

Computer-aided planning and reconstruction of cranial 3D implants

In this contribution, a prototype for semiautomatic computer-aided planning and reconstruction of cranial 3D Implants is presented. The software prototype guides the user through the workflow, beginning with loading and mirroring the patients head to obtain an initial curvature of the cranial implant. However, naïve mirroring is not sufficient for an implant, because human heads are in general too asymmetric. Thus, the user can perform Laplacian smoothing, followed by Delaunay triangulation, for generating an aesthetic looking and well-fitting implant. Finally, our software prototype allows to save the designed 3D model of the implant as a STL-file for 3D printing. The 3D printed implant can be used for further pre-interventional planning or even as the final implant for the patient. In summary, our findings show that a customized MeVisLab prototype can be an alternative to complex commercial planning software, which may not be available in a clinic.In this contribution, a prototype for semiautomatic computer-aided planning and reconstruction of cranial 3D Implants is presented. The software prototype guides the user through the workflow, beginning with loading and mirroring the patients head to obtain an initial curvature of the cranial implant. However, naïve mirroring is not sufficient for an implant, because human heads are in general too asymmetric. Thus, the user can perform Laplacian smoothing, followed by Delaunay triangulation, for generating an aesthetic looking and well-fitting implant. Finally, our software prototype allows to save the designed 3D model of the implant as a STL-file for 3D printing. The 3D printed implant can be used for further pre-interventional planning or even as the final implant for the patient. In summary, our findings show that a customized MeVisLab prototype can be an alternative to complex commercial planning software, which may not be available in a clinic.

Interactive reconstructions of cranial 3D implants under MeVisLab as an alternative to commercial planning software

In this publication, the interactive planning and reconstruction of cranial 3D Implants under the medical prototyping platform MeVisLab as alternative to commercial planning software is introduced. In doing so, a MeVisLab prototype consisting of a customized data-flow network and an own C++ module was set up. As a result, the Computer-Aided Design (CAD) software prototype guides a user through the whole workflow to generate an implant. Therefore, the workflow begins with loading and mirroring the patients head for an initial curvature of the implant. Then, the user can perform an additional Laplacian smoothing, followed by a Delaunay triangulation. The result is an aesthetic looking and well-fitting 3D implant, which can be stored in a CAD file format, e.g. STereoLithography (STL), for 3D printing. The 3D printed implant can finally be used for an in-depth pre-surgical evaluation or even as a real implant for the patient. In a nutshell, our research and development shows that a customized MeVisLab software prototype can be used as an alternative to complex commercial planning software, which may also not be available in every clinic. Finally, not to conform ourselves directly to available commercial software and look for other options that might improve the workflow.

Multi-threaded integration of HTC-Vive and MeVisLab

This work presents how Virtual Reality (VR) can easily be integrated into medical applications via a plugin for a medical image processing framework called MeVisLab. A multi-threaded plugin has been developed using OpenVR, a VR library that can be used for developing vendor and platform independent VR applications. The plugin is tested using the HTC Vive, a head-mounted display developed by HTC and Valve Corporation.

Algorithmic evaluation of lower jawbone segmentations

The lower jawbone (or mandible), is due to its exposure to complex biomechanical forces the largest and strongest facial bone in humans. In this publication, an algorithmic evaluation of lower jawbone segmentation with a cellular automata algorithm called GrowCut is presented. For an evaluation, the algorithmic segmentation results were compared with slice-by-slice segmentations from two specialized physicians, which is considered to assess the given ground truth. As a result, pure manual slice-by-slice outlining took on average 39 minutes (minimum 35 minutes and maximum 46 minutes). This stands in strong contrast to an algorithmic segmentation which needed only about one minute for an initialization, hence needing just a fraction of the manual contouring time. At the same time, the algorithmic segmentations could achieve an acceptable Dice Similarity Score (DSC) of nearly ninety percent when compared to the ground truth slice-by-slice segmentations generated by the physicians. This stands in direct comparison to somewhat above ninety percent Dice Score between the two manual segmentations of the jawbones. In summary, this contribution shows that an algorithmic GrowCut segmentation can be an alternative to the very time consuming manual slice-by-slice outlining in the clinical practice.

Interactive Planning of Miniplates

In this contribution, a novel method for computer aided surgery planning of facial defects by using models of purchasable MedArtis Modus 2.0 miniplates is proposed. Implants of this kind, which belong to the osteosynthetic material, are commonly used for treating defects in the facial area. By placing them perpendicular on the defect, the miniplates are fixed on the healthy bone, bent with respect to the surface, to stabilize the defective area. Our software is able to fit a selection of the most common implant models to the surgeons desired position in a 3D computer model. The fitting respects the local surface curvature and adjusts direction and position in any desired way. Conventional methods use Computed Tomography (CT) scans to generate STereoLithic (STL) models serving as bending template for the implants or use a bending tool during the surgery for readjusting the implant several times. Both approaches lead to undesirable expenses in time. With our visual planning tool, surgeons are able to pre-plan the final implant within just a few minutes. The resulting model can be stored in STL format, which is the commonly used format for 3D printing. With this technology, surgeons are able to print the implant just in time or use it for generating a bending tool, both leading to an exactly bent miniplate.

Computer-aided position planning of miniplates to treat facial bone defects

In this contribution, a software system for computer-aided position planning of miniplates to treat facial bone defects is proposed. The intra-operatively used bone plates have to be passively adapted on the underlying bone contours for adequate bone fragment stabilization. However, this procedure can lead to frequent intra-operatively performed material readjustments especially in complex surgical cases. Our approach is able to fit a selection of common implant models on the surgeon’s desired position in a 3D computer model. This happens with respect to the surrounding anatomical structures, always including the possibility of adjusting both the direction and the position of the used osteosynthesis material. By using the proposed software, surgeons are able to pre-plan the out coming implant in its form and morphology with the aid of a computer-visualized model within a few minutes. Further, the resulting model can be stored in STL file format, the commonly used format for 3D printing. Using this technology, surgeons are able to print the virtual generated implant, or create an individually designed bending tool. This method leads to adapted osteosynthesis materials according to the surrounding anatomy and requires further a minimum amount of money and time.

Interaktive Planung von Gesichtsimplantaten

In diesem Beitrag wird eine neue Methode zur computerunterst utzten Behandlungsplanung von knochernen Gesichtsschadelbr uchen unter der Verwendung von Miniplatten vorgestellt. Diese Art von Implantaten wird verwendet, um Knochenbruche im Gesicht zu behandeln. Nach dem derzeitigen Stand der Technik verwendete Methoden wie die Plattenadaption an stereolithischen Modellen oder auf Basis einer computerunterstutzten Planung weisen allerdings eine geringere Flexibilitat, Mehrkosten oder hygienische Risiken auf. Mit der hier vorgestellten Software ist es den Chirurgen moglich, das Resultat vorab in nur wenigen Minuten an einem computervisualisierten Modell zu planen und anschliesend als STL-Datenformat zu exportieren, um es so in der zukunftstrachtigen 3D-Drucktechnologie verwenden zu konnen. Dadurch werden Chirurgen in die Lage gesetzt, das generierte Implantat oder eine entsprechende Biegevorlage flexibel fur jeden visualisierten Defekt im Behandlungszentrum prazise innerhalb weniger Stunden zu erstellen.