Stamatia Giannarou

Probabilistic Tracking of Affine-Invariant Anisotropic Regions

Despite a wide range of feature detectors developed in the computer vision community over the years, direct application of these techniques to surgical navigation has shown significant difficulties due to the paucity of reliable salient features coupled with free--form tissue deformation and changing visual appearance of surgical scenes. The aim of this paper is to propose a novel probabilistic framework to track affine-invariant anisotropic regions under contrastingly different visual appearances during Minimally Invasive Surgery (MIS). The theoretical background of the affine-invariant anisotropic feature detector is presented and a real-time implementation exploiting the computational power of the GPU is proposed. An Extended Kalman Filter (EKF) parameterization scheme is used to adaptively adjust the optimal templates of the detected regions, enabling accurate identification and matching of the tracked features. For effective tracking verification, spatial context and region similarity have also been incorporated. They are used to boost the prediction of the EKF and recover potential tracking failure due to drift or false positives. The proposed framework is compared to the existing methods and their respective performance is evaluated with in vivo video sequences recorded from robotic-assisted MIS procedures, as well as real-world scenes.

From medical images to minimally invasive intervention: Computer assistance for robotic surgery

Minimally invasive surgery has been established as an important way forward in surgery for reducing patient trauma and hospitalization with improved prognosis. The introduction of robotic assistance enhances the manual dexterity and accuracy of instrument manipulation. Further development of the field in using pre- and intra-operative imaging guidance requires the integration of the general anatomy of the patient with clear pathologic indications and geometrical information for preoperative planning and intra-operative manipulation. It also requires effective visualization and the recreation of haptic and tactile sensing with dynamic active constraints to improve consistency and safety of the surgical procedures. This paper describes key technical considerations of tissue deformation tracking, 3D reconstruction, subject-specific modeling, image guidance and augmented reality for robotic assisted minimally invasive surgery. It highlights the importance of adapting preoperative surgical planning according to intra-operative data and illustrates how dynamic information such as tissue deformation can be incorporated into the surgical navigation framework. Some of the recent trends are discussed in terms of instrument design and the usage of dynamic active constraints and human-robot perceptual docking for robotic assisted minimally invasive surgery.

Effect of aging on lung structure in vivo: assessment with densitometric and fractal analysis of high-resolution computed tomography data.

Purpose: To test the hypothesis that there is a difference between the lung computed tomography (CT) microstructure of asymptomatic older individuals and that of young individuals as evaluated by objective indices of complexity and density. Materials and Methods: Two study groups of nonsmoking urban-dwelling individuals over 75 years and under 55 years were prospectively identified. Thirty-three consecutive volunteers (21 older than 75 y and 12 less than 55 y) were included, and CTs were performed with concurrent pulmonary function testing. Pulmonary regions of interest (ROIs) were evaluated with fractal dimension (FD) analysis (an index of complexity), mean lung density (MLD), and percentage of pixels with lung density (LD) less than thresholds of −910 HU and −950 HU. The Student t test and the Mann-Whitney test were used to evaluate for differences in mean values between groups. The Pearson correlation coefficient was used to correlate mean FD value and LD data with pulmonary function. Results: Significant correlations of ROI MLD, LD −910 HU, and LD −950 HU with age and sex were shown (P=0.029–0.003). The ROI mean FD value was greater in younger individuals compared with older individuals (76.5±1.7 vs. 70.3±1.2; P=0.004). There was a correlation between Kco (gas-diffusing capacity adjusted for alveolar volume) and mean FD value (P=0.006) and MLD (P=0.015). Conclusion: The lung parenchyma of nonsmoking older urban-dwelling asymptomatic individuals has significantly different CT density and complexity compared with younger individuals.

Optical Biopsy Mapping for Minimally Invasive Cancer Screening

The quest for providing tissue characterization and functional mapping during minimally invasive surgery (MIS) has motivated the development of new surgical tools that extend the current functional capabilities of MIS. Miniaturized optical probes can be inserted into the instrument channel of standard endoscopes to reveal tissue cellular and subcellular microstructures, allowing excision-free optical biopsy. One of the limitations of such a point based imaging and tissue characterization technique is the difficulty of tracking probed sites in vivo. This prohibits large area surveillance and integrated functional mapping. The purpose of this paper is to present an image-based tracking framework by combining a semi model-based instrument tracking method with vision-based simultaneous localization and mapping. This allows the mapping of all spatio-temporally tracked biopsy sites, which can then be re-projected back onto the endoscopic video to provide a live augmented view in vivo, thus facilitating re-targeting and serial examination of potential lesions. The proposed method has been validated on phantom data with known ground truth and the accuracy derived demonstrates the strength and clinical value of the technique. The method facilitates a move from the current point based optical biopsy towards large area multi-scale image integration in a routine clinical environment.

Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions

PurposeAdvances in technology and computing play an increasingly important role in the evolution of modern surgical techniques and paradigms. This article reviews the current role of machine learning (ML) techniques in the context of surgery with a focus on surgical robotics (SR). Also, we provide a perspective on the future possibilities for enhancing the effectiveness of procedures by integrating ML in the operating room.MethodsThe review is focused on ML techniques directly applied to surgery, surgical robotics, surgical training and assessment. The widespread use of ML methods in diagnosis and medical image computing is beyond the scope of the review. Searches were performed on PubMed and IEEE Explore using combinations of keywords: ML, surgery, robotics, surgical and medical robotics, skill learning, skill analysis and learning to perceive.ResultsStudies making use of ML methods in the context of surgery are increasingly being reported. In particular, there is an increasing interest in using ML for developing tools to understand and model surgical skill and competence or to extract surgical workflow. Many researchers begin to integrate this understanding into the control of recent surgical robots and devices.ConclusionML is an expanding field. It is popular as it allows efficient processing of vast amounts of data for interpreting and real-time decision making. Already widely used in imaging and diagnosis, it is believed that ML will also play an important role in surgery and interventional treatments. In particular, ML could become a game changer into the conception of cognitive surgical robots. Such robots endowed with cognitive skills would assist the surgical team also on a cognitive level, such as possibly lowering the mental load of the team. For example, ML could help extracting surgical skill, learned through demonstration by human experts, and could transfer this to robotic skills. Such intelligent surgical assistance would significantly surpass the state of the art in surgical robotics. Current devices possess no intelligence whatsoever and are merely advanced and expensive instruments.

Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery

Reliable feature tracking is important for accurate tissue deformation recovery, 3D anatomical registration and navigation in computer assisted minimally invasive surgical procedures. Despite a wide range of feature detectors developed in the computer vision community, direct application of these approaches to surgical navigation has shown significant difficulties due to the paucity of reliable feature landmarks coupled with free-form tissue deformation and contrastingly different visual appearances of changing surgical scenes. The purpose of this paper is to introduce an affine-invariant feature detector based on anisotropic features to ensure reliable and persistent feature tracking. A novel scale-space representation is proposed for scale adaptation based on the strength of the anisotropic pattern whereas affine adaptation relies on its intrinsic Fourier properties with an efficient spatial implementation based on the second moment matrix. The proposed detector is compared against the current state-of-the-art feature detectors and their respective performance is evaluated with in vivo video sequences recorded from robotic assisted minimally invasive surgical procedures.

Surgical data science for next-generation interventions

Interventional healthcare will evolve from an artisanal craft based on the individual experiences, preferences and traditions of physicians into a discipline that relies on objective decision-making on the basis of large-scale data from heterogeneous sources.Lena Maier-Hein, Swaroop Vedula, Stefanie Speidel, Nassir Navab, Ron Kikinis, Adrian Park, Matthias Eisenmann, Hubertus Feussner, Germain Forestier, Stamatia Giannarou, Makoto Hashizume, Darko Katic, Hannes Kenngott, Michael Kranzfelder, Anand Malpani, Keno März, Thomas Neumuth, Nicolas Padoy, Carla Pugh, Nicolai Schoch, Danail Stoyanov, Russell Taylor, Martin Wagner, Gregory D. Hager, Pierre Jannin

Pathological Site Retargeting under Tissue Deformation Using Geometrical Association and Tracking

Recent advances in microscopic detection techniques include fluorescence spectroscopy, fibred confocal microscopy and optical coherence tomography. These methods can be integrated with miniaturised probes to assist endoscopy, thus enabling diseases to be detected at an early and pre-invasive stage, forgoing the need for histopathological samples and off-line analysis. Since optical-based biopsy does not leave visible marks after sampling, it is important to track the biopsy sites to enable accurate retargeting and subsequent serial examination. In this paper, a novel approach is proposed for pathological site retargeting in gastroscopic examinations. The proposed method is based on affine deformation modelling with geometrical association combined with cascaded online learning and tracking. It provides online in vivo retargeting, and is able to track pathological sites in the presence of tissue deformation. It is also robust to partial occlusions and can be applied to a range of imaging probes including confocal laser endomicroscopy.

Content-based surgical workflow representation using probabilistic motion modeling

Succinct content-based representation of minimally invasive surgery (MIS) video is important for efficient surgical workflow analysis and modeling of instrument-tissue interaction. Current approaches to video representation are not well suited to MIS as they do not fully capture the underlying tissue deformation nor provide reliable feature tracking. The aim of this paper is to propose a novel framework for content-based surgical scene representation, which simultaneously identifies key surgical episodes and encodes motion of tracked salient features. The proposed method does not require pre-segmentation of the scene and can be easily combined with 3D scene reconstruction techniques to provide further scene representation without the need of going back to the raw data.

Object recognition in the ovary: Quantification of oocytes from microscopic images

The ovary is a female organ that houses a fixed supply of germ cells (oocytes). The absolute number of oocytes at any given stage can be a useful indicator of fertility. Obtaining accurate assessments of the oocyte reserve in humans and experimental models can be time consuming and error prone. In this paper a new approach to facilitate oocyte counting in microscope images of mouse ovaries is presented. The mouse vasa homolog (MVH), an oocyte-specific protein, was labeled in microscope sections and used to develop an algorithm that can identify, count and estimate the size and coordinates of the oocytes. We use this automated approach to generate comparable data with conventional methods of oocyte counting.