Germain Forestier

A Virtual Imaging Platform for Multi-Modality Medical Image Simulation

This paper presents the Virtual Imaging Platform (VIP), a platform accessible at http://vip.creatis.insa-lyon.fr to facilitate the sharing of object models and medical image simulators, and to provide access to distributed computing and storage resources. A complete overview is presented, describing the ontologies designed to share models in a common repository, the workίow template used to integrate simulators, and the tools and strategies used to exploit computing and storage resources. Simulation results obtained in four image modalities and with different models show that VIP is versatile and robust enough to support large simulations. The platform currently has 200 registered users who consumed 33 years of CPU time in 2011.

Classification of surgical processes using dynamic time warping

In the creation of new computer-assisted intervention systems, Surgical Process Models (SPMs) are an emerging concept used for analyzing and assessing surgical interventions. SPMs represent Surgical Processes (SPs) which are formalized as symbolic structured descriptions of surgical interventions using a pre-defined level of granularity and a dedicated terminology. In this context, one major challenge is the creation of new metrics for the comparison and the evaluation of SPs. Thus, correlations between these metrics and pre-operative data are used to classify surgeries and highlight specific information on the surgery itself and on the surgeon, such as his/her level of expertise. In this paper, we explore the automatic classification of a set of SPs based on the Dynamic Time Warping (DTW) algorithm. DTW is used to compute a similarity measure between two SPs that focuses on the different types of activities performed during surgery and their sequencing, by minimizing time differences. Indeed, it turns out to be a complementary approach to the classical methods that only focus on differences in the time and the number of activities. Experiments were carried out on 24 lumbar disk herniation surgeries to discriminate the surgeons level of expertise according to a prior classification of SPs. Supervised and unsupervised classification experiments have shown that this approach was able to automatically identify groups of surgeons according to their level of expertise (senior and junior), and opens many perspectives for the creation of new metrics for comparing and evaluating surgeries.

Faster and more accurate classification of time series by exploiting a novel dynamic time warping averaging algorithm

A concerted research effort over the past two decades has heralded significant improvements in both the efficiency and effectiveness of time series classification. The consensus that has emerged in the community is that the best solution is a surprisingly simple one. In virtually all domains, the most accurate classifier is the nearest neighbor algorithm with dynamic time warping as the distance measure. The time complexity of dynamic time warping means that successful deployments on resource-constrained devices remain elusive. Moreover, the recent explosion of interest in wearable computing devices, which typically have limited computational resources, has greatly increased the need for very efficient classification algorithms. A classic technique to obtain the benefits of the nearest neighbor algorithm, without inheriting its undesirable time and space complexity, is to use the nearest centroid algorithm. Unfortunately, the unique properties of (most) time series data mean that the centroid typically does not resemble any of the instances, an unintuitive and underappreciated fact. In this paper we demonstrate that we can exploit a recent result by Petitjean et al. to allow meaningful averaging of “warped” time series, which then allows us to create super-efficient nearest “centroid” classifiers that are at least as accurate as their more computationally challenged nearest neighbor relatives. We demonstrate empirically the utility of our approach by comparing it to all the appropriate strawmen algorithms on the ubiquitous UCR Benchmarks and with a case study in supporting insect classification on resource-constrained sensors.

Unsupervised Trajectory Segmentation for Surgical Gesture Recognition in Robotic Training

Dexterity and procedural knowledge are two critical skills that surgeons need to master to perform accurate and safe surgical interventions. However, current training systems do not allow us to provide an in-depth analysis of surgical gestures to precisely assess these skills. Our objective is to develop a method for the automatic and quantitative assessment of surgical gestures. To reach this goal, we propose a new unsupervised algorithm that can automatically segment kinematic data from robotic training sessions. Without relying on any prior information or model, this algorithm detects critical points in the kinematic data that define relevant spatio-temporal segments. Based on the association of these segments, we obtain an accurate recognition of the gestures involved in the surgical training task. We, then, perform an advanced analysis and assess our algorithm using datasets recorded during real expert training sessions. After comparing our approach with the manual annotations of the surgical gestures, we observe 97.4% accuracy for the learning purpose and an average matching score of 81.9% for the fully automated gesture recognition process. Our results show that trainees workflow can be followed and surgical gestures may be automatically evaluated according to an expert database. This approach tends toward improving training efficiency by minimizing the learning curve.

Multi-site study of surgical practice in neurosurgery based on surgical process models

Surgical Process Modelling (SPM) was introduced to improve understanding the different parameters that influence the performance of a Surgical Process (SP). Data acquired from SPM methodology is enormous and complex. Several analysis methods based on comparison or classification of Surgical Process Models (SPMs) have previously been proposed. Such methods compare a set of SPMs to highlight specific parameters explaining differences between populations of patients, surgeons or systems. In this study, procedures performed at three different international University hospitals were compared using SPM methodology based on a similarity metric focusing on the sequence of activities occurring during surgery. The proposed approach is based on Dynamic Time Warping (DTW) algorithm combined with a clustering algorithm. SPMs of 41 Anterior Cervical Discectomy (ACD) surgeries were acquired at three Neurosurgical departments; in France, Germany, and Canada. The proposed approach distinguished the different surgical behaviors according to the location where surgery was performed as well as between the categorized surgical experience of individual surgeons. We also propose the use of Multidimensional Scaling to induce a new space of representation of the sequences of activities. The approach was compared to a time-based approach (e.g. duration of surgeries) and has been shown to be more precise. We also discuss the integration of other criteria in order to better understand what influences the way the surgeries are performed. This first multi-site study represents an important step towards the creation of robust analysis tools for processing SPMs. It opens new perspectives for the assessment of surgical approaches, tools or systems as well as objective assessment and comparison of surgeons expertise.

Automatic phase prediction from low-level surgical activities

PurposeAnalyzing surgical activities has received a growing interest in recent years. Several methods have been proposed to identify surgical activities and surgical phases from data acquired in operating rooms. These context-aware systems have multiple applications including: supporting the surgical team during the intervention, improving the automatic monitoring, designing new teaching paradigms.MethodsIn this paper, we use low-level recordings of the activities that are performed by a surgeon to automatically predict the current (high-level) phase of the surgery. We augment a decision tree algorithm with the ability to consider the local context of the surgical activities and a hierarchical clustering algorithm.ResultsExperiments were performed on 22 surgeries of lumbar disk herniation. We obtained an overall precision of 0.843 in detecting phases of 51,489 single activities. We also assess the robustness of the method with regard to noise.ConclusionWe show that using the local context allows us to improve the results compared with methods only considering single activity. Experiments show that the use of the local context makes our method very robust to noise and that clustering the input data first improves the predictions.

Surgical data science for next-generation interventions

Interventional healthcare will evolve from an artisanal craft based on the individual experiences, preferences and traditions of physicians into a discipline that relies on objective decision-making on the basis of large-scale data from heterogeneous sources.Lena Maier-Hein, Swaroop Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, Makoto Hashizume, Darko Katic, Hannes Kenngott, Michael Kranzfelder, Anand Malpani, Keno März, Thomas Neumuth, Nicolas Padoy, Carla Pugh, Nicolai Schoch, Danail Stoyanov, Russell Taylor, Martin Wagner, Gregory D. Hager, Pierre Jannin

Multisource images analysis using collaborative clustering

The development of very high-resolution (VHR) satellite imagery has produced a huge amount of data. The multiplication of satellites which embed different types of sensors provides a lot of heterogeneous images. Consequently, the image analyst has often many different images available, representing the same area of the Earth surface. These images can be from different dates, produced by different sensors, or even at different resolutions. The lack of machine learning tools using all these representations in an overall process constraints to a sequential analysis of these various images. In order to use all the information available simultaneously, we propose a framework where different algorithms can use different views of the scene. Each one works on a different remotely sensed image and, thus, produces different and useful information. These algorithms work together in a collaborative way through an automatic and mutual refinement of their results, so that all the results have almost the same number of clusters, which are statistically similar. Finally, a unique result is produced, representing a consensus among the information obtained by each clustering method on its own image. The unified result and the complementarity of the single results (i.e., the agreement between the clustering methods as well as the disagreement) lead to a better understanding of the scene. The experiments carried out on multispectral remote sensing images have shown that this method is efficient to extract relevant information and to improve the scene understanding.

OntoVIP: An ontology for the annotation of object models used for medical image simulation

This paper describes works carried out in the Virtual Imaging Platform (VIP) project to create a comprehensive conceptualization of object models used in medical image simulation and suitable for the major imaging modalities and simulators. The goal is to create an application ontology that can be used to annotate the models in the VIP platforms model repository, to facilitate their sharing and reuse. Such annotations allow making the anatomical, physiological and pathophysiological content of the object models explicit.

Multi-modality medical image simulation of biological models with the Virtual Imaging Platform (VIP)

This paper describes a framework for the integration of medical image simulators in the Virtual Imaging Platform (VIP). Simulation is widely involved in medical imaging but its availability is hampered by the heterogeneity of software interfaces and the required amount of computing power. To address this, VIP defines a simulation workflow template which transforms object models from the IntermediAte Model Format (IAMF) into native simulator formats and parallelizes the simulation computation. Format conversions, geometrical scene definition and physical parameter generation are covered. The core simulator executables are directly embedded in the simulation workflow, enabling data parallelism exploitation without modifying the simulator. The template is instantiated on simulators of the four main medical imaging modalities, namely Positron Emission Tomography, Ultrasound imaging, Magnetic Resonance Imaging and Computed Tomography. Simulation examples and performance results on the European Grid Infrastructure are shown.