R.V. Paul Chan

Expert Diagnosis of Plus Disease in Retinopathy of Prematurity From Computer-Based Image Analysis

IMPORTANCEnPublished definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions.nnnOBJECTIVEnTo identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis.nnnDESIGN, SETTING, AND PARTICIPANTSnA computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts.nnnMAIN OUTCOMES AND MEASURESnThe primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD.nnnRESULTSnAccuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6-disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2- to 3-disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable to that of 11 expert physicians (mean 87%, range 79%-99%).nnnCONCLUSIONS AND RELEVANCEnExperts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

Diagnostic Discrepancies in Retinopathy of Prematurity Classification

PURPOSEnTo identify the most common areas for discrepancy in retinopathy of prematurity (ROP) classification between experts.nnnDESIGNnProspective cohort study.nnnPARTICIPANTSnA total of 281 infants were identified as part of a multicenter, prospective, ROP cohort study from 7 participating centers. Each site had participating ophthalmologists who provided the clinical classification after routine examination using binocular indirect ophthalmoscopy (BIO) and obtained wide-angle retinal images, which were independently classified by 2 study experts.nnnMETHODSnWide-angle retinal images (RetCam; Clarity Medical Systems, Pleasanton, CA) were obtained from study subjects, and 2 experts evaluated each image using a secure web-based module. Image-based classifications for zone, stage, plus disease, and overall disease category (no ROP, mild ROP, type II or pre-plus, and type I) were compared between the 2 experts and with the clinical classification obtained by BIO.nnnMAIN OUTCOME MEASURESnInter-expert image-based agreement and image-based versus ophthalmoscopic diagnostic agreement using absolute agreement and weighted kappa statistic.nnnRESULTSnA total of 1553 study eye examinations from 281 infants were included in the study. Experts disagreed on the stage classification in 620 of 1553 comparisons (40%), plus disease classification (including pre-plus) in 287 of 1553 comparisons (18%), zone in 117 of 1553 comparisons (8%), and overall ROP category in 618 of 1553 comparisons (40%). However, agreement for presence versus absence of type 1 disease was >95%. There were no differences between image-based and clinical classification except for zone III disease.nnnCONCLUSIONSnThe most common area of discrepancy in ROP classification is stage, although inter-expert agreement for clinically significant disease, such as presence versus absence of type 1 and type 2 disease, is high. There were no differences between image-based grading and clinical examination in the ability to detect clinically significant disease. This study provides additional evidence that image-based classification of ROP reliably detects clinically significant levels of ROP with high accuracy compared with the clinical examination.

Plus Disease in Retinopathy of Prematurity: Improving Diagnosis by Ranking Disease Severity and Using Quantitative Image Analysis.

PURPOSEnTo determine expert agreement on relative retinopathy of prematurity (ROP) disease severity and whether computer-based image analysis can model relative disease severity, and to propose consideration of a more continuous severity score for ROP.nnnDESIGNnWe developed 2 databases of clinical images of varying disease severity (100 images and 34 images) as part of the Imaging and Informatics in ROP (i-ROP) cohort study and recruited expert physician, nonexpert physician, and nonphysician graders to classify and perform pairwise comparisons on both databases.nnnPARTICIPANTSnSix participating expert ROP clinician-scientists, each with a minimum of 10 years of clinical ROP experience and 5 ROP publications, and 5 image graders (3 physicians and 2 nonphysician graders) who analyzed images that were obtained during routine ROP screening in neonatal intensive care units.nnnMETHODSnImages in both databases were ranked by average disease classification (classification ranking), by pairwise comparison using the Elo rating method (comparison ranking), and by correlation with the i-ROP computer-based image analysis system.nnnMAIN OUTCOME MEASURESnInterexpert agreement (weighted κ statistic) compared with the correlation coefficient (CC) between experts on pairwise comparisons and correlation between expert rankings and computer-based image analysis modeling.nnnRESULTSnThere was variable interexpert agreement on diagnostic classification of disease (plus, preplus, or normal) among the 6 experts (mean weighted κ, 0.27; range, 0.06-0.63), but good correlation between experts on comparison ranking of disease severity (mean CC, 0.84; range, 0.74-0.93) on the set of 34 images. Comparison ranking provided a severity ranking that was in good agreement with ranking obtained by classification ranking (CC, 0.92). Comparison ranking on the larger dataset by both expert and nonexpert graders demonstrated good correlation (mean CC, 0.97; range, 0.95-0.98). The i-ROP system was able to model this continuous severity with good correlation (CC, 0.86).nnnCONCLUSIONSnExperts diagnose plus disease on a continuum, with poor absolute agreement on classification but good relative agreement on disease severity. These results suggest that the use of pairwise rankings and a continuous severity score, such as that provided by the i-ROP system, may improve agreement on disease severity in the future.

Plus Disease in Retinopathy of Prematurity: A Continuous Spectrum of Vascular Abnormality as a Basis of Diagnostic Variability.

PURPOSEnTo identify patterns of interexpert discrepancy in plus disease diagnosis in retinopathy of prematurity (ROP).nnnDESIGNnWe developed 2 datasets of clinical images as part of the Imaging and Informatics in ROP study and determined a consensus reference standard diagnosis (RSD) for each image based on 3 independent image graders and the clinical examination results. We recruited 8 expert ROP clinicians to classify these images and compared the distribution of classifications between experts and the RSD.nnnPARTICIPANTSnEight participating experts with more than 10 years of clinical ROP experience and more than 5 peer-reviewed ROP publications who analyzed images obtained during routine ROP screening in neonatal intensive care units.nnnMETHODSnExpert classification of images of plus disease in ROP.nnnMAIN OUTCOME MEASURESnInterexpert agreement (weighted κ statistic) and agreement and bias on ordinal classification between experts (analysis of variance [ANOVA]) and the RSD (percent agreement).nnnRESULTSnThere was variable interexpert agreement on diagnostic classifications between the 8 experts and the RSD (weighted κ, 0-0.75; mean, 0.30). The RSD agreement ranged from 80% to 94% for the dataset of 100 images and from 29% to 79% for the dataset of 34 images. However, when images were ranked in order of disease severity (by average expert classification), the pattern of expert classification revealed a consistent systematic bias for each expert consistent with unique cut points for the diagnosis of plus disease and preplus disease. The 2-way ANOVA model suggested a highly significant effect of both image and user on the average score (dataset A: P < 0.05 and adjusted R2xa0= 0.82; and dataset B: P < 0.05 and adjusted R2xa0=xa00.6615).nnnCONCLUSIONSnThere is wide variability in the classification of plus disease by ROP experts, which occurs because experts have different cut points for the amounts of vascular abnormality required for presence of plus and preplus disease. This has important implications for research, teaching, and patient care for ROP and suggests that a continuous ROP plus disease severity score may reflect more accurately the behavior of expert ROP clinicians and may better standardize classification in the future.

Practice Patterns in Retinopathy of Prematurity Treatment for Disease Milder Than Recommended by Guidelines

PURPOSEnTo characterize the frequency of and clinical indications for which experts treat retinopathy of prematurity (ROP) milder than type 1 disease, the recommended threshold for treatment from established consensus guidelines.nnnDESIGNnDescriptive analysis.nnnMETHODSnsetting: Multicenter.nnnSTUDY POPULATIONnA database of 1444 eyes generated prospectively from all babies screened for ROP at 1 of 6 major ROP centers whose parents provided informed consent.nnnINTERVENTIONnRetrospective review of the database and charts to identify all patients treated for ROP milder than type 1.nnnMAIN OUTCOME MEASUREnIndication(s) for treatment.nnnRESULTSnA total of 137 eyes of 70 infants were treated for ROP. Of these 137 eyes, 13 (9.5%) were treated despite a clinical diagnosis milder than type 1 ROP. Indications for treatment included active ROP with the fellow eye being treated for type 1 ROP (2 eyes, 15.4%); concerning structural changes (9 eyes, 69.2%), including tangential traction with temporal vessel straightening concerning for macular dragging (8 eyes, 61.5%) and thick stage 3 membranes with anteroposterior traction concerning for progression to stage 4 ROP (3 eyes, 23.1%); persistent ROP at an advanced postmenstrual age (4 eyes, 30.8%); and/or vitreous hemorrhage (3 eyes, 23.1%).nnnCONCLUSIONSnExperts in this study occasionally recommended treatment in eyes with disease less than type 1 ROP. This study has important clinical implications and highlights the role of individual clinical judgment in situations not covered by evidence-based treatment guidelines.

Automated Diagnosis of Plus Disease in Retinopathy of Prematurity Using Deep Convolutional Neural Networks

Importance Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide. The decision to treat is primarily based on the presence of plus disease, defined as dilation and tortuosity of retinal vessels. However, clinical diagnosis of plus disease is highly subjective and variable. Objective To implement and validate an algorithm based on deep learning to automatically diagnose plus disease from retinal photographs. Design, Setting, and Participants A deep convolutional neural network was trained using a data set of 5511 retinal photographs. Each image was previously assigned a reference standard diagnosis (RSD) based on consensus of image grading by 3 experts and clinical diagnosis by 1 expert (ie, normal, pre–plus disease, or plus disease). The algorithm was evaluated by 5-fold cross-validation and tested on an independent set of 100 images. Images were collected from 8 academic institutions participating in the Imaging and Informatics in ROP (i-ROP) cohort study. The deep learning algorithm was tested against 8 ROP experts, each of whom had more than 10 years of clinical experience and more than 5 peer-reviewed publications about ROP. Data were collected from July 2011 to December 2016. Data were analyzed from December 2016 to September 2017. Exposures A deep learning algorithm trained on retinal photographs. Main Outcomes and Measures Receiver operating characteristic analysis was performed to evaluate performance of the algorithm against the RSD. Quadratic-weighted &kgr; coefficients were calculated for ternary classification (ie, normal, pre–plus disease, and plus disease) to measure agreement with the RSD and 8 independent experts. Results Of the 5511 included retinal photographs, 4535 (82.3%) were graded as normal, 805 (14.6%) as pre–plus disease, and 172 (3.1%) as plus disease, based on the RSD. Mean (SD) area under the receiver operating characteristic curve statistics were 0.94 (0.01) for the diagnosis of normal (vs pre–plus disease or plus disease) and 0.98 (0.01) for the diagnosis of plus disease (vs normal or pre–plus disease). For diagnosis of plus disease in an independent test set of 100 retinal images, the algorithm achieved a sensitivity of 93% with 94% specificity. For detection of pre–plus disease or worse, the sensitivity and specificity were 100% and 94%, respectively. On the same test set, the algorithm achieved a quadratic-weighted &kgr; coefficient of 0.92 compared with the RSD, outperforming 6 of 8 ROP experts. Conclusions and Relevance This fully automated algorithm diagnosed plus disease in ROP with comparable or better accuracy than human experts. This has potential applications in disease detection, monitoring, and prognosis in infants at risk of ROP.

Diagnostic Accuracy of Ophthalmoscopy vs Telemedicine in Examinations for Retinopathy of Prematurity

Importance Examinations for retinopathy of prematurity (ROP) are typically performed using binocular indirect ophthalmoscopy. Telemedicine studies have traditionally assessed the accuracy of telemedicine compared with ophthalmoscopy as a criterion standard. However, it is not known whether ophthalmoscopy is truly more accurate than telemedicine. Objective To directly compare the accuracy and sensitivity of ophthalmoscopy vs telemedicine in diagnosing ROP using a consensus reference standard. Design, Setting, and Participants This multicenter prospective study conducted between July 1, 2011, and November 30, 2014, at 7 neonatal intensive care units and academic ophthalmology departments in the United States and Mexico included 281 premature infants who met the screening criteria for ROP. Exposures Each examination consisted of 1 eye undergoing binocular indirect ophthalmoscopy by an experienced clinician followed by remote image review of wide-angle fundus photographs by 3 independent telemedicine graders. Main Outcomes and Measures Results of both examination methods were combined into a consensus reference standard diagnosis. The agreement of both ophthalmoscopy and telemedicine was compared with this standard, using percentage agreement and weighted &kgr; statistics. Results Among the 281 infants in the study (127 girls and 154 boys; mean [SD] gestational age, 27.1 [2.4] weeks), a total of 1553 eye examinations were classified using both ophthalmoscopy and telemedicine. Ophthalmoscopy and telemedicine each had similar sensitivity for zone I disease (78% [95% CI, 71%-84%] vs 78% [95% CI, 73%-83%]; Pu2009>u2009.99 [nu2009=u2009165]), plus disease (74% [95% CI, 61%-87%] vs 79% [95% CI, 72%-86%]; Pu2009=u2009.41 [nu2009=u200950]), and type 2 ROP (stage 3, zone I, or plus disease: 86% [95% CI, 80%-92%] vs 79% [95% CI, 75%-83%]; Pu2009=u2009.10 [nu2009=u2009251]), but ophthalmoscopy was slightly more sensitive in identifying stage 3 disease (85% [95% CI, 79%-91%] vs 73% [95% CI, 67%-78%]; Pu2009=u2009.004 [nu2009=u2009136]). Conclusions and Relevance No difference was found in overall accuracy between ophthalmoscopy and telemedicine for the detection of clinically significant ROP, although, on average, ophthalmoscopy had slightly higher accuracy for the diagnosis of zone III and stage 3 ROP. With the caveat that there was variable accuracy between examiners using both modalities, these results support the use of telemedicine for the diagnosis of clinically significant ROP.

Retinopathy of Prematurity: A Review of Risk Factors and their Clinical Significance

Retinopathy of prematurity (ROP) is a retinal vasoproliferative disease that affects premature infants. Despite improvements in neonatal care and management guidelines, ROP remains a leading cause of childhood blindness worldwide. Current screening guidelines are primarily based on two risk factors: birth weight and gestational age;xa0however, many investigators have suggested other risk factors, including maternal factors, prenatal and perinatal factors, demographics, medical interventions, comorbidities of prematurity, nutrition, and genetic factors. We review the existing literature addressing various possible ROP risk factors. Although there have been contradictory reports, and the risk may vary between different populations, understanding ROP risk factors is essential to develop predictive models, to gain insights into pathophysiology of retinal vascular diseases and diseases of prematurity, and to determine future directions in management of and research in ROP.

Fully automated disease severity assessment and treatment monitoring in retinopathy of prematurity using deep learning

Retinopathy of prematurity (ROP) is a disease that affects premature infants, where abnormal growth of the retinal blood vessels can lead to blindness unless treated accordingly. Infants considered at risk of severe ROP are monitored for symptoms of plus disease, characterized by arterial tortuosity and venous dilation at the posterior pole, with a standard photographic definition. Disagreement among ROP experts in diagnosing plus disease has driven the development of computer-based methods that classify images based on hand-crafted features extracted from the vasculature. However, most of these approaches are semi-automated, which are time-consuming and subject to variability. In contrast, deep learning is a fully automated approach that has shown great promise in a wide variety of domains, including medical genetics, informatics and imaging. Convolutional neural networks (CNNs) are deep networks which learn rich representations of disease features that are highly robust to variations in acquisition and image quality. In this study, we utilized a U-Net architecture to perform vessel segmentation and then a GoogLeNet to perform disease classification. The classifier was trained on 3,000 retinal images and validated on an independent test set of patients with different observed progressions and treatments. We show that our fully automated algorithm can be used to monitor the progression of plus disease over multiple patient visits with results that are consistent with the experts’ consensus diagnosis. Future work will aim to further validate the method on larger cohorts of patients to assess its applicability within the clinic as a treatment monitoring tool.

Changes in relative position of choroidal versus retinal vessels in preterm infants

Purpose The purpose of this study was to characterize a novel finding that relative positions of choroidal and retinal vessels change over time in preterm infants and to identify factors associated with this finding using quantitative analysis. Methods Fundus images were obtained prospectively through a retinopathy of prematurity (ROP) cohort study. Images were excluded if choroidal vessels could not be identified. Changes in relative position of characteristic choroidal landmarks with respect to retinal vessels between two time points 5 to 7 weeks apart were measured. Univariate and multivariate regression analyses were performed to identify associated factors with the amount of change. Results The discovery and replication cohorts included 45 and 58 patients, respectively. Ninety-two of them (89%) were non-Hispanic Caucasians. Changes in relative position of choroidal versus retinal vessels were detected in all eyes of the discovery and replication cohorts (mean amount = 0.42 ± 0.12 and 0.35 ± 0.12 mm, respectively). On combined multiple regression analysis of the two cohorts, type 1 ROP, higher postmenstral age at the first time point, and shorter distance from optic disc to choroidal landmark were significantly associated with less change in relative position. Conclusions Choroidal vessels grow anteriorly with respect to retinal vessels at posterior pole in preterm infants, suggesting relatively faster peripheral growth of choroidal versus retinal vessels. Eyes with severe ROP showed less difference in growth, which might represent alterations in choroidal development due to advanced ROP. These findings may contribute to better understanding about the physiology of choroidal development and involvement in ROP.