Michael Chiang

Evaluation of Screening for Retinopathy of Prematurity by ROPtool or a Lay Reader.

PURPOSEnTo determine if (1) tortuosity assessment by a computer program (ROPtool, developed at the University of North Carolina, Chapel Hill, and Duke University, and licensed by FocusROP) that traces retinal blood vessels and (2) assessment by a lay reader are comparable with assessment by a panel of 3 retinopathy of prematurity (ROP) experts for remote clinical grading of vascular abnormalities such as plus disease.nnnDESIGNnValidity and reliability analysis of diagnostic tools.nnnPARTICIPANTSnThree hundred thirty-five fundus images of prematurely born infants.nnnMETHODSnThree hundred thirty-five fundus images of prematurely born infants were obtained by neonatal intensive care unit nurses. A panel of 3 ROP experts graded 84 images showing vascular dilatation, tortuosity, or both and 251 images showing no evidence of vascular abnormalities. These images were sent electronically to an experienced lay reader who independently graded them for vascular abnormalities. The images also were analyzed using the ROPtool, which assigns a numerical value to the level of vascular abnormality and tortuosity present in each of 4 quadrants or sectors. The ROPtool measurements of vascular abnormalities were graded and compared with expert panel grades with a receiver operating characteristic (ROC) curve. Grades between human readers were cross-tabulated. The area under the ROC curve was calculated for the ROPtool, and sensitivity and specificity were computed for the lay reader.nnnMAIN OUTCOME MEASURESnMeasurements of vascular abnormalities by ROPtool and grading of vascular abnormalities by 3 ROP experts and 1 experienced lay reader.nnnRESULTSnThe ROC curve for ROPtools tortuosity assessment had an area under the ROC curve of 0.917. Using a threshold value of 4.97 for the second most tortuous quadrant, ROPtools sensitivity was 91% and its specificity was 82%. Lay reader sensitivity and specificity were 99% and 73%, respectively, and had high reliability (κ, 0.87) in repeated measurements.nnnCONCLUSIONSnROPtool had very good accuracy for detection of vascular abnormalities suggestive of plus disease when compared with expert physician graders. The lay readers results showed excellent sensitivity andxa0good specificity when compared with those of the expert graders. These options for remote reading of images to detect vascular abnormalities deserve consideration in the quest to use telemedicine with remotexa0reading for efficient delivery of high-quality care and to detect infants requiring bedside examination.

Analysis of Underlying Causes of Inter-expert Disagreement in Retinopathy of Prematurity Diagnosis: Application of Machine Learning Principles

OBJECTIVEnInter-expert variability in image-based clinical diagnosis has been demonstrated in many diseases including retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and is a major cause of childhood blindness. In order to better understand the underlying causes of variability among experts, we propose a method to quantify the variability of expert decisions and analyze the relationship between expert diagnoses and features computed from the images. Identification of these features is relevant for development of computer-based decision support systems and educational systems in ROP, and these methods may be applicable to other diseases where inter-expert variability is observed.nnnMETHODSnThe experiments were carried out on a dataset of 34 retinal images, each with diagnoses provided independently by 22 experts. Analysis was performed using concepts of Mutual Information (MI) and Kernel Density Estimation. A large set of structural features (a total of 66) were extracted from retinal images. Feature selection was utilized to identify the most important features that correlated to actual clinical decisions by the 22 study experts. The best three features for each observer were selected by an exhaustive search on all possible feature subsets and considering joint MI as a relevance criterion. We also compared our results with the results of Cohens Kappa [36] as an inter-rater reliability measure.nnnRESULTSnThe results demonstrate that a group of observers (17 among 22) decide consistently with each other. Mean and second central moment of arteriolar tortuosity is among the reasons of disagreement between this group and the rest of the observers, meaning that the group of experts consider amount of tortuosity as well as the variation of tortuosity in the image.nnnCONCLUSIONnGiven a set of image-based features, the proposed analysis method can identify critical image-based features that lead to expert agreement and disagreement in diagnosis of ROP. Although tree-based features and various statistics such as central moment are not popular in the literature, our results suggest that they are important for diagnosis.

Dealing with inter-expert variability in retinopathy of prematurity: A machine learning approach

BACKGROUND AND OBJECTIVEnUnderstanding the causes of disagreement among experts in clinical decision making has been a challenge for decades. In particular, a high amount of variability exists in diagnosis of retinopathy of prematurity (ROP), which is a disease affecting low birth weight infants and a major cause of childhood blindness. A possible cause of variability, that has been mostly neglected in the literature, is related to discrepancies in the sets of important features considered by different experts. In this paper we propose a methodology which makes use of machine learning techniques to understand the underlying causes of inter-expert variability.nnnMETHODSnThe experiments are carried out on a dataset consisting of 34 retinal images, each with diagnoses provided by 22 independent experts. Feature selection techniques are applied to discover the most important features considered by a given expert. Those features selected by each expert are then compared to the features selected by other experts by applying similarity measures. Finally, an automated diagnosis system is built in order to check if this approach can be helpful in solving the problem of understanding high inter-rater variability.nnnRESULTSnThe experimental results reveal that some features are mostly selected by the feature selection methods regardless the considered expert. Moreover, for pairs of experts with high percentage agreement among them, the feature selection algorithms also select similar features. By using the relevant selected features, the classification performance of the automatic system was improved or maintained.nnnCONCLUSIONSnThe proposed methodology provides a handy framework to identify important features for experts and check whether the selected features reflect the pairwise agreements/disagreements. These findings may lead to improved diagnostic accuracy and standardization among clinicians, and pave the way for the application of this methodology to other problems which present inter-expert variability.

Observer and feature analysis on diagnosis of retinopathy of prematurity

Retinopathy of prematurity (ROP) is a disease affecting low-birth weight infants and is a major cause of childhood blindness. However, human diagnoses is often subjective and qualitative. We propose a method to analyze the variability of expert decisions and the relationship between the expert diagnoses and features. The analysis is based on Mutual Information and Kernel Density Estimation on features. The experiments are carried out on a dataset of 34 retinal images diagnosed by 22 experts. The results show that a group of observers decide consistently with each other and there are popular features that have a high correlation with labels.

A GMM-based feature extraction technique for the automated diagnosis of Retinopathy of Prematurity

Retinopathy of Prematurity (ROP) is an ophthalmic disease that is a leading cause of childhood blindness throughout the world. Accurate diagnosis of ROP is vital to identify infants who require treatment, which can prevent blindness. Arterial tortuosity and venous dilation in the retina are important signs of ROP, so it is necessary to extract these features from points on the vessels or vessel segments. Then, an image is represented with statistics such as minimum, maximum or mean of these values. However, these statistics provide biased estimates as an image contains both healthy and abnormal vessels. In this work, we present a novel feature extraction technique that represents each image with the parameters of a two-component Gaussian Mixture Model (GMM). Using these features, we performed classification experiments on a manually segmented retinal image dataset consisting of 77 images. The results show that GMM-based features outperform other features that are based on classical statistics, with accuracy over 90%. Moreover, if the features are extracted from the whole image without distinguishing veins and arteries, proposed features provide better performance compared to using traditional statistics.

Analysis of shape assumptions in 3D reconstruction of retina from multiple fundus images

Utilizing priors about the shape of retinal surface is important for accurate reconstruction. We present a detailed analysis of geometrical shape priors in the 3D reconstruction of retina. We first approximate the retinal surface either as a sphere inspired by the actual shape of the eyeball, or as a plane inspired by the 2D mosaicing approaches. Based on this approximation, we perform an initial camera localization with a 2D-to-3D registration procedure. Then, parameters of the surface and the camera poses are refined through a nonlinear least squares optimization using different shape priors. The resulting 3D model and camera poses can be used for intuitively visualizing the retinal images with a model-guided browsing interface.