GP157\u2005Quality, trustworthiness, readability, and accuracy of medical information regarding common pediatric emergency medicine-related complaints on the web

Abstract

Study Objectives To analyze the adequacy of information on the internet directed at parents regarding common acute pediatric complaints. Methods Authors searched three internet search engines for four common acute pediatric complaints (child + fever, vomiting, cough, abdomen pain) assessing the first 20 results for each query. Website readability was evaluated using a composite of the Flesch-Kincaid Grade Level, Gunning Fog Scale, Simple Measure of Gobbledygook Score and Coleman-Liau Index. Quality and trustworthiness were independently assessed by two reviewers using JAMA Benchmark Criteria and National Library of Medicine (NLM) Trustworthy scores. Accuracy was independently assessed by two board-certified physicians (emergency medicine, pediatric emergency medicine) who analyzed text with website and author identifiers deleted. Accuracy was calculated by dividing the number of correct by the total number of correct and incorrect medical statements. Disagreements regarding physician accuracy were settled by a third physician. Accuracy was defined as ≥ 95% correct, readability as an 8th grade or lower reading level, high quality as ≥ 3 JAMA criteria, and trustworthiness as an NLM total score ≥ 3 with all sub-scores > 0. Features of accurate and inaccurate websites were compared using chi-square analysis for categorical data, and Mann-Whitney U for continuous and ordinal data. Association between website accuracy and search rank order was measured using Spearman’s correlation coefficient. Inter-rater reliability of website accuracy, JAMA criteria and NLM scores was measured using Cohen’s kappa (k). Results Ninety-six websites that were duplicates or directed at health professionals were excluded, leaving 144 evaluable websites. Of these, 60 (42%) were readable, 49 (34%) were certified as reliable by the Health on the Net (HON) Foundation, 38 (26%) had high quality JAMA criteria (k = 0.68), and 44 (31%) had reliable NLM trustworthy scores (k = 0.66). Physicians graded 87 websites (60%) as accurate (k = 0.94). Professional medical organizations (hospitals, academic societies, governments) more frequently published accurate websites compared to individuals and non-professional websites. (74% vs. 46%, p < 0.01). There was no correlation between accuracy and search rank order (rho = - 0.05, -0.21 to 0.12, 95% CI). There was no association between accuracy and physician authorship, quality, trustworthiness, readability, article age or HON certification. Conclusion Most studied websites had poor quality, readability, and trustworthiness. Many websites were also inaccurate. Because inadequate web-based information might adversely influence parents’ medical decisions, measures should be taken to ensure information related to acute pediatric complaints is of high quality, readable, trustworthy, and accurate.