Esra Ataer-Cansizoglu

Tracking an RGB-D Camera Using Points and Planes

Planes are dominant in most indoor and outdoor scenes and the development of a hybrid algorithm that incorporates both point and plane features provides numerous advantages. In this regard, we present a tracking algorithm for RGB-D cameras using both points and planes as primitives. We show how to extend the standard prediction-and-correction framework to include planes in addition to points. By fitting planes, we implicitly take care of the noise in the depth data that is typical in many commercially available 3D sensors. In comparison with the techniques that use only points, our tracking algorithm has fewer failure modes, and our reconstructed model is compact and more accurate. The tracking algorithm is supported by relocalization and bundle adjustment processes to demonstrate a real-time simultaneous localization and mapping (SLAM) system using a hand-held or robot-mounted RGB-D camera. Our experiments show large-scale indoor reconstruction results as point-based and plane-based 3D models, and demonstrate an improvement over the point-based tracking algorithms using a benchmark for RGB-D cameras.

Expert Diagnosis of Plus Disease in Retinopathy of Prematurity From Computer-Based Image Analysis

IMPORTANCE Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions. OBJECTIVE To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis. DESIGN, SETTING, AND PARTICIPANTS A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts. MAIN OUTCOMES AND MEASURES The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD. RESULTS Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable to that of 11 expert physicians (mean 87%, range 79%-99%). CONCLUSIONS AND RELEVANCE Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

Plus disease diagnosis in retinopathy of prematurity: vascular tortuosity as a function of distance from optic disk.

Purpose: To examine vascular tortuosity as a function of distance from the optic disk in infants with retinopathy of prematurity. Methods: Thirty-four wide-angle retinal images from infants with retinopathy of prematurity were reviewed by 22 experts. A reference standard for each image was defined as the diagnosis (plus vs. not plus) given by the majority of experts. Tortuosity, defined as vessel length divided by straight line distance between vessel end points, was calculated as a function of distance from the disk margin for arteries and veins using computer-based methods developed by the authors. Results: Mean cumulative tortuosity increased with distance from the disk margin, both in 13 images with plus disease (P = 0.007 for arterial tortuosity [n = 62 arteries], P < 0.001 for venous tortuosity [n = 58 veins] based on slope of best fit line by regression), and in 21 images without plus disease (P < 0.001 for arterial tortuosity [n = 94 arteries], P <0 .001 for venous tortuosity [n = 85 veins]). Images with plus disease had significantly higher vascular tortuosity than images without plus disease (P < 0.05), up to 7.0 disk diameters from the optic disk margin. Conclusion: Vascular tortuosity was higher peripherally than centrally, both in images with and without plus disease, suggesting that peripheral retinal features may be relevant for retinopathy of prematurity diagnosis.

Analysis of Underlying Causes of Inter-expert Disagreement in Retinopathy of Prematurity Diagnosis: Application of Machine Learning Principles

OBJECTIVE Inter-expert variability in image-based clinical diagnosis has been demonstrated in many diseases including retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and is a major cause of childhood blindness. In order to better understand the underlying causes of variability among experts, we propose a method to quantify the variability of expert decisions and analyze the relationship between expert diagnoses and features computed from the images. Identification of these features is relevant for development of computer-based decision support systems and educational systems in ROP, and these methods may be applicable to other diseases where inter-expert variability is observed. METHODS The experiments were carried out on a dataset of 34 retinal images, each with diagnoses provided independently by 22 experts. Analysis was performed using concepts of Mutual Information (MI) and Kernel Density Estimation. A large set of structural features (a total of 66) were extracted from retinal images. Feature selection was utilized to identify the most important features that correlated to actual clinical decisions by the 22 study experts. The best three features for each observer were selected by an exhaustive search on all possible feature subsets and considering joint MI as a relevance criterion. We also compared our results with the results of Cohens Kappa [36] as an inter-rater reliability measure. RESULTS The results demonstrate that a group of observers (17 among 22) decide consistently with each other. Mean and second central moment of arteriolar tortuosity is among the reasons of disagreement between this group and the rest of the observers, meaning that the group of experts consider amount of tortuosity as well as the variation of tortuosity in the image. CONCLUSION Given a set of image-based features, the proposed analysis method can identify critical image-based features that lead to expert agreement and disagreement in diagnosis of ROP. Although tree-based features and various statistics such as central moment are not popular in the literature, our results suggest that they are important for diagnosis.

Calibration of Non-overlapping Cameras Using an External SLAM System

We present a simple method for calibrating a set of cameras that may not have overlapping field of views. We reduce the problem of calibrating the non-overlapping cameras to the problem of localizing the cameras with respect to a global 3D model reconstructed with a simultaneous localization and mapping (SLAM) system. Specifically, we first reconstruct such a global 3D model by using a SLAM system using an RGB-D sensor. We then perform localization and intrinsic parameter estimation for each camera by using 2D-3D correspondences between the camera and the 3D model. Our method locates the cameras within the 3D model, which is useful for visually inspecting camera poses and provides a model-guided browsing interface of the images. We demonstrate the advantages of our method using several indoor scenes.

Dealing with inter-expert variability in retinopathy of prematurity: A machine learning approach

BACKGROUND AND OBJECTIVE Understanding the causes of disagreement among experts in clinical decision making has been a challenge for decades. In particular, a high amount of variability exists in diagnosis of retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and a major cause of childhood blindness. A possible cause of variability, that has been mostly neglected in the literature, is related to discrepancies in the sets of important features considered by different experts. In this paper we propose a methodology which makes use of machine learning techniques to understand the underlying causes of inter-expert variability. METHODS The experiments are carried out on a dataset consisting of 34 retinal images, each with diagnoses provided by 22 independent experts. Feature selection techniques are applied to discover the most important features considered by a given expert. Those features selected by each expert are then compared to the features selected by other experts by applying similarity measures. Finally, an automated diagnosis system is built in order to check if this approach can be helpful in solving the problem of understanding high inter-rater variability. RESULTS The experimental results reveal that some features are mostly selected by the feature selection methods regardless the considered expert. Moreover, for pairs of experts with high percentage agreement among them, the feature selection algorithms also select similar features. By using the relevant selected features, the classification performance of the automatic system was improved or maintained. CONCLUSIONS The proposed methodology provides a handy framework to identify important features for experts and check whether the selected features reflect the pairwise agreements/disagreements. These findings may lead to improved diagnostic accuracy and standardization among clinicians, and pave the way for the application of this methodology to other problems which present inter-expert variability.

Contour-based shape representation using principal curves

Extraction and representation of contours are challenging problems and are crucial for many image processing applications. In this study, given a membership function that returns the score of a point belonging to a contour, we propose a method for contour representation based on the principal curve (PC) of this function. The proposed method provides a piecewise linear representation of the contour with fewer points while preserving shape. Varied experiments are conducted, including lung boundary representation in CT images and shape representation in handwritten images. The results show that the technique provides accurate shape representation.

Observer and feature analysis on diagnosis of retinopathy of prematurity

Retinopathy of prematurity (ROP) is a disease affecting low-birth weight infants and is a major cause of childhood blindness. However, human diagnoses is often subjective and qualitative. We propose a method to analyze the variability of expert decisions and the relationship between the expert diagnoses and features. The analysis is based on Mutual Information and Kernel Density Estimation on features. The experiments are carried out on a dataset of 34 retinal images diagnosed by 22 experts. The results show that a group of observers decide consistently with each other and there are popular features that have a high correlation with labels.

Level sets for retinal vasculature segmentation using seeds from ridges and edges from phase maps

In this paper, we present a novel modification to level set based automatic retinal vasculature segmentation approaches. The method introduces ridge sample extraction for sampling the vasculature centerline and phase map based edge detection for accurate region boundary detection. Segmenting the vasculature in fundus images has been generally challenging for level set methods employing classical edge-detection methodologies. Furthermore, initialization with seed points determined by sampling vessel centerlines using ridge identification makes the method completely automated. The resulting algorithm is able to segment vasculature in fundus imagery accurately and automatically. Quantitative results supplemented with visual ones support this observation. The methodology could be applied to the broader class of vessel segmentation problems encountered in medical image analytics.

Structure-based level set method for automatic retinal vasculature segmentation

Segmentation of vasculature in retinal fundus image by level set methods employing classical edge detection methodologies is a tedious task. In this study, a revised level set-based retinal vasculature segmentation approach is proposed. During preprocessing, intensity inhomogeneity on the green channel of input image is corrected by utilizing all image channels, generating more efficient results compared to methods utilizing only one (green) channel. A structure-based level set method employing a modified phase map is introduced to obtain accurate skeletonization and segmentation of the retinal vasculature. The seed points around vessels are selected and the level sets are initialized automatically. Furthermore, the proposed method introduces an improved zero-level contour regularization term which is more appropriate than the ones introduced by other methods for vasculature structures. We conducted the experiments on our own dataset, as well as two publicly available datasets. The results show that the proposed method segments retinal vessels accurately and its performance is comparable to state-of-the-art supervised/unsupervised segmentation techniques.