Lucy I. Mudie

Crowdsourcing to Evaluate Fundus Photographs for the Presence of Glaucoma

Purpose: To assess the accuracy of crowdsourcing for grading optic nerve images for glaucoma using Amazon Mechanical Turk before and after training modules. Materials and Methods: Images (n=60) from 2 large population studies were graded for glaucoma status and vertical cup-to-disc ratio (VCDR). In the baseline trial, users on Amazon Mechanical Turk (Turkers) graded fundus photos for glaucoma and VCDR after reviewing annotated example images. In 2 additional trials, Turkers viewed a 26-slide PowerPoint training or a 10-minute video training and passed a quiz before being permitted to grade the same 60 images. Each image was graded by 10 unique Turkers in all trials. The mode of Turker grades for each image was compared with an adjudicated expert grade to determine accuracy as well as the sensitivity and specificity of Turker grading. Results: In the baseline study, 50% of the images were graded correctly for glaucoma status and the area under the receiver operating characteristic (AUROC) was 0.75 [95% confidence interval (CI), 0.64-0.87]. Post-PowerPoint training, 66.7% of the images were graded correctly with AUROC of 0.86 (95% CI, 0.78-0.95). Finally, Turker grading accuracy was 63.3% with AUROC of 0.89 (95% CI, 0.83-0.96) after video training. Overall, Turker VCDR grades for each image correlated with expert VCDR grades (Bland-Altman plot mean difference=−0.02). Conclusions: Turkers graded 60 fundus images quickly and at low cost, with grading accuracy, sensitivity, and specificity, all improving with brief training. With effective education, crowdsourcing may be an efficient tool to aid in the identification of glaucomatous changes in retinal images.

Autorefraction-Based Prescription and Mailed Delivery of Eyeglasses

In the United States, approximately 8.2 million people older than 40 years of age have visual impairment resulting from uncorrected refractive error (RE). Subjective refraction (SR) performed by a licensed eye care professional remains the gold standard for correcting RE. However, trained experts often are expensive and underused by poorer populations. Difficulties with transportation, affordability, and coverage gaps are considerable barriers to accessing eye care services among disadvantaged individuals with visual impairment. Given that modern autorefractors provide an efficient and reliable objective measure of RE, autorefraction-based prescription of eyeglasses could reduce barriers to eye care access. The Screening to Prevent Glaucoma scheme is an ongoing community-based program aimed at streamlining screening approaches for glaucoma and other eye diseases in underserved populations in Baltimore. As part of the screening algorithm, participants undergo vision testing and autorefraction. Herein, we report the outcome of providing spectacles based on autorefraction results. Screening Eye Examination. Eye examinations performed by trained screeners included assessment of presenting visual acuity (VA), fundus photography, visual field assessment, and measurement of intraocular pressure. Presenting VA was tested using an iPod application (iPoD touch; Apple Inc., Cupertino, CA; developed by Manu Lakkur, based in San Francisco, CA), and if found to be worse than 20/40 in at least 1 eye, participants underwent autorefraction in both eyes using a Topcon KR800S autorefractor (Topcon Medical Systems, Inc., Oakland, CA). Objective refraction measurements were obtained for each eye, followed by machine-corrected VA measurements where the smallest line with 3 of 5 characters read correctly was considered the best-corrected VA. Autorefraction-Based Eyeglass Prescription. After an initial feasibility study, which demonstrated that autorefractiongenerated refractions were highly accurate compared with SRs, stringent eligibility criteria (Table S1, available at www.aaojournal.org) were developed and used from December 2015 through September 2016 to provide prescription glasses to individuals whose only vision problem identified on screening was RE. Participants demonstrating improvement with autorefraction to at least 20/30 could choose between receiving free corrective eyeglasses via mail or attending a free eye examination at Wilmer Eye Institute, Baltimore, Maryland (including free eyeglasses). Of 58 participants who were eligible for this study, 55 participants (94.8%) opted to receive autorefraction-based mail-in eyeglasses and 3 participants (0.05%) opted for an in-person eye examination at Wilmer Eye Institute. Mean age was 61.7 years (standard deviation [SD], 8.1 years), approximately half were women (52.7%), and most were black (92.7%). Based on autorefraction, mean right eye spherical equivalent, J0, and J45 were 0.23 diopters (D; SD, 1.86 D), 0.17 D (SD, 0.51 D), and 0.04 D (SD, 0.33 D), respectively. After correction of RE, subjective VA in the right eye obtained from autorefraction improved to 20/20 or better in 36 participants (65.5%), 20/25 in 12 participants (21.8%), and 20/30 in 7 participants (12.7%). Participants were provided plastic and metal frames to choose from. Given a choice, 36 participants (65.5%) chose to receive single-vision lenses and 19 participants (32.5%) preferred bifocals. Progressives were not offered. For participants requesting bifocals, segment heights were measured and they were prescribed near reading power measurements using a standard age nomogram. Interpupillary distance was measured using a digital pupillometer (Gizmo Supply Co., Fountain Valley, CA). Glasses were prepared at Wilmer Eye Institute and mailed to participants along with a survey designed to capture satisfaction with the autorefraction-based mail-in eyeglasses (Table S2, available at www.aaojournal.org). Participants were asked to test the new glasses for 2 weeks before answering and returning the satisfaction survey via mail. Thirty-nine of 55 participants (71%) returned satisfaction surveys. Nonrespondents (n 1⁄4 16) did not differ significantly from respondents in terms of age, race, or gender (P > 0.05 for all, t test). Thirty-three of 39 participants (85%) were extremely satisfied and 36 of 39 participants (92%) were extremely or moderately satisfied with the mailed eyeglasses (Table 3). The only participant who stated that she did not feel like autorefractionbased glasses improved her vision reported being unhappy with the lined bifocals that she had chosen and believed that she might have instead chosen single-vision glasses if she had had a chance to try them on at a clinic first. Participants who were dissatisfied or neutral did not differ from those who were satisfied in terms of presenting visual acuity, VA after autorefraction, or spherical equivalent (P > 0.05 for all, t test). This positive response to autorefraction-based spectacles and participants’ preference for this method of delivery suggest that this model could help to reduce the burden of uncorrected RE. There exists a pattern of underuse of eye care among low-income and minority populations. In a previous community-based eye screening study conducted in Baltimore, only 41% of participants scheduled for a definitive eye examination attended the visit, despite being offered free examinations and transportation. Although we recognize that SR is the gold standard for RE correction, our model of community screenings coupled with autorefraction-based mailed spectacles has the potential to reach those not seeking care. Our study did use strict eligibility criteria. In particular, all participants had to have spherical power between 6.00 andþ2.00 D and astigmatism of less than þ2.75 D, so our results cannot be generalized to those with higher RE. Likewise, our approach applies

Improving consensus scoring of crowdsourced data using the rasch model: Development and refinement of a diagnostic instrument

Background Diabetic retinopathy (DR) is a leading cause of vision loss in working age individuals worldwide. While screening is effective and cost effective, it remains underutilized, and novel methods are needed to increase detection of DR. This clinical validation study compared diagnostic gradings of retinal fundus photographs provided by volunteers on the Amazon Mechanical Turk (AMT) crowdsourcing marketplace with expert-provided gold-standard grading and explored whether determination of the consensus of crowdsourced classifications could be improved beyond a simple majority vote (MV) using regression methods. Objective The aim of our study was to determine whether regression methods could be used to improve the consensus grading of data collected by crowdsourcing. Methods A total of 1200 retinal images of individuals with diabetes mellitus from the Messidor public dataset were posted to AMT. Eligible crowdsourcing workers had at least 500 previously approved tasks with an approval rating of 99% across their prior submitted work. A total of 10 workers were recruited to classify each image as normal or abnormal. If half or more workers judged the image to be abnormal, the MV consensus grade was recorded as abnormal. Rasch analysis was then used to calculate worker ability scores in a random 50% training set, which were then used as weights in a regression model in the remaining 50% test set to determine if a more accurate consensus could be devised. Outcomes of interest were the percent correctly classified images, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) for the consensus grade as compared with the expert grading provided with the dataset. Results Using MV grading, the consensus was correct in 75.5% of images (906/1200), with 75.5% sensitivity, 75.5% specificity, and an AUROC of 0.75 (95% CI 0.73-0.78). A logistic regression model using Rasch-weighted individual scores generated an AUROC of 0.91 (95% CI 0.88-0.93) compared with 0.89 (95% CI 0.86-92) for a model using unweighted scores (chi-square P value<.001). Setting a diagnostic cut-point to optimize sensitivity at 90%, 77.5% (465/600) were graded correctly, with 90.3% sensitivity, 68.5% specificity, and an AUROC of 0.79 (95% CI 0.76-0.83). Conclusions Crowdsourced interpretations of retinal images provide rapid and accurate results as compared with a gold-standard grading. Creating a logistic regression model using Rasch analysis to weight crowdsourced classifications by worker ability improves accuracy of aggregated grades as compared with simple majority vote.

Comparison of self-refraction using a simple device, USee, with manifest refraction in adults

Background The USee device is a new self-refraction tool that allows users to determine their own refractive error. We evaluated the ease of use of USee in adults, and compared the refractive error correction achieved with USee to clinical manifest refraction. Methods Sixty adults with uncorrected visual acuity <20/30 and spherical equivalent between –6.00 and +6.00 diopters completed manifest refraction and self-refraction. Results Subjects had a mean (±SD) age of 53.1 (±18.6) years, and 27 (45.0%) were male. Mean (±SD) spherical equivalent measured by manifest refraction and self-refraction were –0.90 D (±2.53) and –1.22 diopters (±2.42), respectively (p = 0.001). The proportion of subjects correctable to ≥20/30 in the better eye was higher for manifest refraction (96.7%) than self-refraction (83.3%, p = 0.005). Failure to achieve visual acuity ≥20/30 with self-refraction in right eyes was associated with increasing age (per year, OR: 1.05; 95% CI: 1.00–1.10) and higher cylindrical power (per diopter, OR: 7.26; 95% CI: 1.88–28.1). Subjectively, 95% of participants thought USee was easy to use, 85% thought self-refraction correction was better than being uncorrected, 57% thought vision with self-refraction correction was similar to their current corrective lenses, and 53% rated their vision as “very good” or “excellent” with self-refraction. Conclusion Self-refraction provides acceptable refractive error correction in the majority of adults. Programs targeting resource-poor settings could potentially use USee to provide easy on-site refractive error correction.

Dual sensory impairment: The association between glaucomatous vision loss and hearing impairment and function.

Background Hearing impairment, vision impairment, and dual impairment (both hearing and vision impairment), have been independently associated with functional and cognitive decline. In prior studies of dual impairment, vision impairment is generally not defined or defined by visual acuity alone. Glaucoma is a leading cause of blindness and does not affect visual acuity until late in the disease; instead, visual field loss is used to measure vision impairment from glaucoma. Objective To examine the effect of glaucomatous visual field loss and hearing impairment on function. Design Cross-sectional. Setting Hospital-based clinic in Baltimore, Maryland. Subjects 220 adults, ≥55 years presenting to the glaucoma clinic. Methods Vision impairment was defined as mean deviation on visual field testing worse than -5 decibels in the better eye, and hearing impairment was defined as pure tone average worse than 25 decibels on threshold audiometry testing in the better ear. Standardized questionnaires were used to assess functional status. Results Five participants were excluded for incomplete testing, leaving 32 with vision impairment only, 63 with hearing impairment only, 42 with dual impairment, and 78 controls with no hearing impairment or vision impairment. Participants with dual impairment were more likely to be older and non-White. Dual impairment was associated with significantly more severe driving limitation and more difficulty with communication compared to those without sensory impairment when adjusted for age, race, gender and number of comorbidities. Conclusion Older individuals with glaucoma and hearing loss seem to have generally poorer functioning than those with single sensory loss. Health professionals should consider visual field loss as a type of vision impairment when managing patients with dual impairment.

Nationwide Prevalence of Self-Reported Serious Sensory Impairments and Their Associations with Self-Reported Cognitive and Functional Difficulties

PURPOSE To estimate the nationwide prevalence of self-reported serious vision impairment (SVI), serious hearing impairment (SHI), and serious dual sensory impairment (DSI; i.e., concomitant SVI and SHI) and to characterize their associations with self-reported cognitive, independent living, self-care, and ambulatory difficulties. DESIGN The American Community Survey (ACS) is a nationwide cross-sectional survey administered by the United States Census Bureau. PARTICIPANTS The 2011-2015 ACS sample contains data on 7 210 535 individuals 45 years of age or older. METHODS Descriptive statistics for each of the 4 mutually exclusive sensory impairment categories no sensory impairment (NSI), SVI, SHI, and serious DSI were calculated using the weighted sample. Adjusted odds ratios using several logistic regressions were calculated using the unweighted sample to measure the magnitude of associations between sensory impairment status and the outcome difficulties. MAIN OUTCOME MEASURES Self-reported cognitive, independent living, self-care, and ambulatory difficulty. RESULTS Among individuals 45 years of age or older, the estimated nationwide prevalence of self-reported SVI alone is 2.8%, that of SHI alone is 6.0%, and that of serious DSI is 1.6%. The prevalence of each sensory impairment increases with age. A greater proportion of American Indians or Alaskan Natives experience SVI (4.8%), SHI (8.5%), and serious DSI (3.7%) than any other race or ethnic group (P < 0.001). Individuals reporting serious DSI are more likely to report cognitive impairment, independent living difficulty, self-care difficulty, and difficulty ambulating than individuals with NSI across all age groups (all P < 0.001). Furthermore, serious DSI is associated with greater cognitive and functional difficulties than SVI or SHI alone, and SVI alone has a greater association with cognitive and functional difficulties than SHI alone. CONCLUSIONS The nationwide prevalence of self-reported serious sensory impairment increases with age and is distributed unequally among different racial and ethnic groups. Any sensory impairment is associated with greater cognitive and functional difficulties than NSI. Additionally, serious DSI is associated with greater difficulties than SVI or SHI alone, and SVI alone is more serious than SHI alone in each of the 4 cognitive and functional difficulties.

In Plain Sight: Reading Outcomes of Providing Eyeglasses to Disadvantaged Children

ABSTRACT Many disadvantaged students with refractive errors, such as myopia (nearsightedness) and hyperopia (farsightedness), do not have eyeglasses, and their reduced vision may impact reading proficiency. Providing eyeglasses may increase their reading success. This article reports the findings of a study in Baltimore City in which disadvantaged second and third graders were assessed for vision problems. Of 317 students, 182 were given glasses. Those who needed glasses were given two pairs, one for home and one for school, as well as replacements if glasses were lost or broken. School staff assisted in ensuring that students wore their glasses, storing them safely, and replacing glasses when necessary. Students who received glasses improved more on Woodcock reading measures than those who never needed glasses (ES = +0.16, p < .03). The study demonstrates the potential of providing eyeglasses to disadvantaged students who need them to improve their reading performance.

Pediatric ophthalmology and childhood reading difficulties. Overview of reading development and assessments for the pediatric ophthalmologist

Reading difficulties are common in the pediatric population, and large socioeconomic disparities exist. In the United States 46% of white children achieved expected reading proficiency by the end of fourth grade, while only 21% of Hispanic and 18% of African American children were reading at the expected level. Reading is an involved cognitive process with many subskills; likewise, development of reading proficiency is a complex and continuous process. Failure to achieve reading proficiency or even early difficulty with reading can affect a childs academic performance for years to come. Some studies suggest reading proficiency may be related to later success in life. Although many problems with reading are not related to vision, a vision assessment is recommended for children with reading difficulties and a suspected vision problem. The process of reading development as well as the varied educational assessments of reading are presented here for pediatric ophthalmologists.