Dzhoshkun I. Shakir

NiftyNet: a deep-learning platform for medical imaging

Highlights • An open-source platform is implemented based on TensorFlow APIs for deep learning in medical imaging domain.• A modular implementation of the typical medical imaging machine learning pipeline facilitates (1) warm starts with established pre-trained networks, (2) adapting existing neural network architectures to new problems, and (3) rapid prototyping of new solutions.• Three deep-learning applications, including segmentation, regression, image generation and representation learning, are presented as concrete examples illustrating the platform’s key features.

GIFT-Cloud

Highlights • A platform for sharing medical imaging data between clinicians and researchers.• Extensible system connects three hospitals and two universities.• Simple for end users with low impact on hospital IT systems.• Automated anonymisation of pixel data and metadata at the clinical site.• Maintains subject data groupings while preserving anonymity.

Real-time mosaicing of fetoscopic videos using SIFT

Fetoscopic laser photo-coagulation of the placental vascular anastomoses remains the most effective therapy for twin-to-twin transfusion syndrome (TTTS) in monochorionic twin pregnancies. However, to ensure the success of the intervention, complete photo-coagulation of all anastomoses is needed. This is made difficult by the limited field of view of the fetoscopic video guidance, which hinders the surgeons ability to locate all the anastomoses. A potential solution to this problem is to expand the field of view of the placental surface by creating a mosaic from overlapping fetoscopic images. This mosaic can then be used for anastomoses localization and spatial orientation during surgery. However, this requires accurate and fast algorithms that can operate within the real-time constraints of fetal surgery. In this work, we present an image mosaicing framework that leverages the parallelism of modern GPUs and can process clinical fetoscopic images in real-time. Initial qualitative results on ex-vivo placental images indicate that the proposed framework can generate clinically useful mosaics from fetoscopic videos in real-time.

Freehand SPECT reconstructions using look up tables

Nuclear imaging is a commonly used tool in todays diagnostics and therapy planning. For interventional use however it suffers from drawbacks which limit its application. Freehand SPECT was developed to overcome these limitations and to provide 3D functional imaging during an intervention. It combines a nuclear probe with an optical tracking system to obtain its position and orientation in space synchronized with its reading. This information can be used to compute a 3D tomographic reconstruction of an activity distribution. However, as there is no fixed geometry the system matrix has to be computed on-the-fly, using ad-hoc models of the detection process. One solution for such a model is a reference look up table of previously acquired measurements of a single source at different angles and distances. In this work two look up tables with a one and four millimeter step size between the entries were acquired. Twelve datasets of a phantom with two hollow spheres filled with a solution of Tc99wm were acquired with the Freehand SPECT system. Reconstructions with the look up tables and two analytical models currently in use were performed with these datasets and compared with each other. The finely sampled look up table achieved the qualitatively best reconstructions, while one of the analytical models showed the best positional accuracy.

A Combined EM and Visual Tracking Probabilistic Model for Robust Mosaicking: Application to Fetoscopy

Twin-to-Twin Transfusion Syndrome (TTTS) is a progressive pregnancy complication in which inter-twin vascular connections in the shared placenta result in a blood flow imbalance between the twins. The most effective therapy is to sever these connections by laser photo-coagulation. However, the limited field of view of the fetoscope hinders their identification. A potential solution is to augment the surgeons view by creating a mosaic image of the placenta. State-of-the-art mosaicking methods use feature-based approaches, which have three main limitations: (i) they are not robust against corrupt data e.g. blurred frames, (ii) temporal information is not used, (iii) the resulting mosaic suffers from drift. We introduce a probabilistic temporal model that incorporates electromagnetic and visual tracking data to achieve a robust mosaic with reduced drift. By assuming planarity of the imaged object, the nRT decomposition can be used to parametrize the state vector. Finally, we tackle the non-linear nature of the problem in a numerically stable manner by using the Square Root Unscented Kalman Filter. We show an improvement in performance in terms of robustness as well as a reduction of the drift in comparison to state-of-the-art methods in synthetic, phantom and ex vivo datasets.

Towards intra-operative PET for head and neck cancer: lymph node localization using high-energy probes

We present a novel approach for intra-operative localization of lymph nodes and metastases in the head and neck region using the radio-tracer [18F]FDG. By combining an optical tracking system with a high-energy gamma probe to detect 511keV annihilation gammas, we enable intra-operative PET to visualize activity distributions. Detection of these gammas is modeled ad-hoc analytically, taking into account several factors affecting the detection process. This allows us to iteratively reconstruct the radio-tracer distribution within a localized volume of interest. As a feasibility study we analyze clinical data of 7 patients with tumors in the head and neck region, and derive a realistic neck phantom configuration with [18F]FDG-filled lesions mimicking tumors and lymph nodes. We demonstrate the capabilities and limitations of our approach using that neck phantom. We also outline possible improvements to make our method clinically viable towards less invasive surgeries.

Detection models for freehand SPECT reconstruction

Nuclear imaging modalities are commonly used tools in todays diagnostics and therapy planning. However for interventional use they suffer from drawbacks which limit their application. Freehand SPECT was developed to provide 3D functional imaging during interventions. It combines a nuclear detector with an optical tracking system to obtain its position and orientation in space and synchronizes this with the detector readings. This information can be used to compute a 3D tomographic reconstruction of an activity distribution of a nuclear tracer. As there is no fixed geometry, the system matrix has to be computed on the fly. This is done with models of the detection process for completely arbitrary freehand acquisitions. The accuracy of the reconstructions is highly dependent on the used models of the detection process. Different models of the detection process were developed and evaluated in this work, in particular two analytical models as well as lookup tables generated from either real measurements or Monte Carlo simulations. We showed that it is possible to perform acceptable reconstructions with a simple but efficient analytical model. The use of lookup tables to generate the system matrix in Freehand SPECT is a fast solution with good accuracy.

Retrieval and registration of long-range overlapping frames for scalable mosaicking of in vivo fetoscopy

PurposeThe standard clinical treatment of Twin-to-Twin transfusion syndrome consists in the photo-coagulation of undesired anastomoses located on the placenta which are responsible to a blood transfer between the two twins. While being the standard of care procedure, fetoscopy suffers from a limited field-of-view of the placenta resulting in missed anastomoses. To facilitate the task of the clinician, building a global map of the placenta providing a larger overview of the vascular network is highly desired.MethodsTo overcome the challenging visual conditions inherent to in vivo sequences (low contrast, obstructions or presence of artifacts, among others), we propose the following contributions: (1) robust pairwise registration is achieved by aligning the orientation of the image gradients, and (2) difficulties regarding long-range consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word strategy, which identifies overlapping frames of the sequence to be registered regardless of their respective location in time.ResultsIn addition to visual difficulties, in vivo sequences are characterised by the intrinsic absence of gold standard. We present mosaics motivating qualitatively our methodological choices and demonstrating their promising aspect. We also demonstrate semi-quantitatively, via visual inspection of registration results, the efficacy of our registration approach in comparison with two standard baselines.ConclusionThis paper proposes the first approach for the construction of mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual challenges during registration and long-range temporal consistency are proposed, offering first positive results on in vivo data for which standard mosaicking techniques are not applicable.

Two new ad-hoc models of detection physics and their evaluation for navigated beta probe surface imaging

Intra-operative surface imaging with navigated beta probes in conjunction with positron-emitting radiotracers like 18F-FDG has been shown to enable control of tumor resection borders. We showed previously that employing iterative reconstruction (MLEM) in conjunction with an ad-hoc model of the detection physics (based on solid-angle geometry, SA) improves the image quality. In this study, we sampled the beta probe readings of a point source using a precision step-motor to generate a look-up-table (LUT) model. We also generated a simplified geometrical model (SG) based on this data set. To see how these two models influence the image quality compared to the old SA model, we reconstructed images from sparsely sampled datasets of a phantom with three hotspots using each model. The images yielded 76% (SA), 81% (SG), and 81% (LUT) mean NCC compared to the ground truth. The SG and LUT models, however, could resolve the hotspots better in the datasets where the detector-to-phantom distance was larger. Additionally, we compared the deviations of the SA and SG analytical models to the measured LUT model, where we found that the SG model gives estimates substantially closer to the actual beta probe readings than the previous SA model.

Evaluation of an ad hoc model of detection physics for navigated beta-probe surface imaging

Intraoperative surface imaging with navigated beta-probes has been shown to be a possibility to enable control of tumor resection borders. By employing ad hoc models of the detection physics the image quality can be improved. Our model computes the amount of radiation from a single point source that reaches the detector, with the solid angle subtended by the detector on the source, assuming perfect shielding. The sensitivity of the detector to the source due to the angle between the detector axis and the source-to-detector vector is also considered. A set of experiments was performed with three sources (two 10x10mm2 and one 20x10mm2 pieces of cellulose saturated with FDG) on a plate as phantom. Five sets of measurements were taken, three of them at a distance of 10mm from the plate und two at 30mm. At both distances one measurement set was taken in a random manner and the other ones systematically covering the whole area. The same experiments were simulated with our model and the GATE simulation framework. The resulting measurements from the experiments and simulations were then used to perform a reconstruction of the sources. The real measurements were compared to those simulated with our model and GATE, with a mean NCC of 80.64% for our model and 70.14% for GATE. In the reconstructions of the real measurements the sources were visually quite well separated, however the reconstructions of the measurements simulated by the model show that there is still room for further improvement.