Elmer V. Bernstam

Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites.

Abstract Objectives: To determine the characteristics of popular breast cancer related websites and whether more popular sites are of higher quality. Design: The search engine Google was used to generate a list of websites about breast cancer. Google ranks search results by measures of link popularity—the number of links to a site from other sites. The top 200 sites returned in response to the query “breast cancer” were divided into “more popular” and “less popular” subgroups by three different measures of link popularity: Google rank and number of links reported independently by Google and by AltaVista (another search engine). Main outcome measures: Type and quality of content. Results: More popular sites according to Google rank were more likely than less popular ones to contain information on ongoing clinical trials (27% v 12%, P=0.01), results of trials (12% v 3%, P=0.02), and opportunities for psychosocial adjustment (48% v 23%, P<0.01). These characteristics were also associated with higher number of links as reported by Google and AltaVista. More popular sites by number of linking sites were also more likely to provide updates on other breast cancer research, information on legislation and advocacy, and a message board service. Measures of quality such as display of authorship, attribution or references, currency of information, and disclosure did not differ between groups. Conclusions: Popularity of websites is associated with type rather than quality of content. Sites that include content correlated with popularity may best meet the publics desire for information about breast cancer. What is already known on this topic Patients are using the world wide web to search for health information Breast cancer is one of the most popular search topics Characteristics of popular websites may reflect the information needs of patients What this study adds Type rather than quality of content correlates with popularity of websites Measures of quality correlate with accuracy of medical information

Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use?

OBJECTIVE To find and assess quality-rating instruments that can be used by health care consumers to assess websites displaying health information. DATA SOURCES Searches of PubMed, the World Wide Web (using five different search engines), reference tracing from identified articles, and a review of the of the American Medical Informatics Associations annual symposium proceedings. REVIEW METHODS Sources were examined for availability, number of elements, objectivity, and readability. RESULTS A total of 273 distinct instruments were found and analyzed. Of these, 80 (29%) made evaluation criteria publicly available and 24 (8.7%) had 10 or fewer elements (items that a user has to assess to evaluate a website). Seven instruments consisted of elements that could all be evaluated objectively. Of these seven, one instrument consisted entirely of criteria with acceptable interobserver reliability (kappa> or =0.6); another instrument met readability standards. CONCLUSIONS There are many quality-rating instruments, but few are likely to be practically usable by the intended audience.

Accuracy and self correction of information received from an internet breast cancer list: content analysis.

Abstract Objectives To determine the prevalence of false or misleading statements in messages posted by internet cancer support groups and whether these statements were identified as false or misleading and corrected by other participants in subsequent postings. Design Analysis of content of postings. Setting Internet cancer support group Breast Cancer Mailing List. Main outcome measures Number of false or misleading statements posted from 1 January to 23 April 2005 and whether these were identified and corrected by participants in subsequent postings. Results 10 of 4600 postings (0.22%) were found to be false or misleading. Of these, seven were identified as false or misleading by other participants and corrected within an average of four hours and 33 minutes (maximum, nine hours and nine minutes). Conclusions Most posted information on breast cancer was accurate. Most false or misleading statements were rapidly corrected by participants in subsequent postings.

Caveats for the use of operational electronic health record data in comparative effectiveness research.

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

A Day in the Life of PubMed: Analysis of a Typical Day's Query Log

OBJECTIVE To characterize PubMed usage over a typical day and compare it to previous studies of user behavior on Web search engines. DESIGN We performed a lexical and semantic analysis of 2,689,166 queries issued on PubMed over 24 consecutive hours on a typical day. MEASUREMENTS We measured the number of queries, number of distinct users, queries per user, terms per query, common terms, Boolean operator use, common phrases, result set size, MeSH categories, used semantic measurements to group queries into sessions, and studied the addition and removal of terms from consecutive queries to gauge search strategies. RESULTS The size of the result sets from a sample of queries showed a bimodal distribution, with peaks at approximately 3 and 100 results, suggesting that a large group of queries was tightly focused and another was broad. Like Web search engine sessions, most PubMed sessions consisted of a single query. However, PubMed queries contained more terms. CONCLUSION PubMeds usage profile should be considered when educating users, building user interfaces, and developing future biomedical information retrieval systems.

Commonly cited website quality criteria are not effective at identifying inaccurate online information about breast cancer

Consumers increasingly consult the Internet for breast cancer information. Concerned about accuracy, multiple organizations developed quality criteria for online content. However, the effectiveness of these tools is unknown. The authors determined whether existing quality criteria can identify inaccurate breast cancer information online.

What is biomedical informatics

Biomedical informatics lacks a clear and theoretically-grounded definition. Many proposed definitions focus on data, information, and knowledge, but do not provide an adequate definition of these terms. Leveraging insights from the philosophy of information, we define informatics as the science of information, where information is data plus meaning. Biomedical informatics is the science of information as applied to or studied in the context of biomedicine. Defining the object of study of informatics as data plus meaning clearly distinguishes the field from related fields, such as computer science, statistics and biomedicine, which have different objects of study. The emphasis on data plus meaning also suggests that biomedical informatics problems tend to be difficult when they deal with concepts that are hard to capture using formal, computational definitions. In other words, problems where meaning must be considered are more difficult than problems where manipulating data without regard for meaning is sufficient. Furthermore, the definition implies that informatics research, teaching, and service should focus on biomedical information as data plus meaning rather than only computer applications in biomedicine.

Using citation data to improve retrieval from MEDLINE

OBJECTIVE To determine whether algorithms developed for the World Wide Web can be applied to the biomedical literature in order to identify articles that are important as well as relevant. DESIGN AND MEASUREMENTS A direct comparison of eight algorithms: simple PubMed queries, clinical queries (sensitive and specific versions), vector cosine comparison, citation count, journal impact factor, PageRank, and machine learning based on polynomial support vector machines. The objective was to prioritize important articles, defined as being included in a pre-existing bibliography of important literature in surgical oncology. RESULTS Citation-based algorithms were more effective than noncitation-based algorithms at identifying important articles. The most effective strategies were simple citation count and PageRank, which on average identified over six important articles in the first 100 results compared to 0.85 for the best noncitation-based algorithm (p < 0.001). The authors saw similar differences between citation-based and noncitation-based algorithms at 10, 20, 50, 200, 500, and 1,000 results (p < 0.001). Citation lag affects performance of PageRank more than simple citation count. However, in spite of citation lag, citation-based algorithms remain more effective than noncitation-based algorithms. CONCLUSION Algorithms that have proved successful on the World Wide Web can be applied to biomedical information retrieval. Citation-based algorithms can help identify important articles within large sets of relevant results. Further studies are needed to determine whether citation-based algorithms can effectively meet actual user information needs.

Matching identifiers in electronic health records: implications for duplicate records and patient safety

Objective To quantify the percentage of records with matching identifiers as an indicator for duplicate or potentially duplicate patient records in electronic health records in five different healthcare organisations, describe the patient safety issues that may arise, and present solutions for managing duplicate records or records with matching identifiers. Methods For each institution, we retrieved deidentified counts of records with an exact match of patient first and last names and dates of birth and determined the number of patient records existing for the top 250 most frequently occurring first and last name pairs. We also identified methods for managing duplicate records or records with matching identifiers, reporting the adoption rate of each across institutions. Results The occurrence of matching first and last name in two or more individuals ranged from 16.49% to 40.66% of records; inclusion of date of birth reduced the rates to range from 0.16% to 15.47%. The number of records existing for the most frequently occurring name at each site ranged from 41 to 2552. Institutions varied widely in the methods they implemented for preventing, detecting and removing duplicate records, and mitigating resulting errors. Conclusions The percentage of records having matching patient identifiers is high in several organisations, indicating that the rate of duplicate records or records may also be high. Further efforts are necessary to improve management of duplicate records or records with matching identifiers and minimise the risk for patient harm.

Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed?

PURPOSE Many criteria have been developed to rate the quality of online health information. To effectively evaluate quality, consumers must use quality criteria that can be reliably assessed. However, few instruments have been validated for inter-rater agreement. Therefore, we assessed the degree to which two raters could reliably assess 22 popularly cited quality criteria on a sample of 42 complementary and alternative medicine Web sites. METHODS We determined the degree of inter-rater agreement by calculating the percentage agreement, Cohens kappa, and prevalence- and bias-adjusted kappa (PABAK). RESULTS Our un-calibrated analysis showed poor inter-rater agreement on eight of the 22 quality criteria. Therefore, we created operational definitions for each of the criteria, decreased the number of assessment choices and defined where to look for the information. As a result 18 of the 22 quality criteria were reliably assessed (inter-rater agreement > or = 0.6). CONCLUSIONS We conclude that even with precise definitions, some commonly used quality criteria cannot be reliably assessed. However, inter-rater agreement can be improved with precise operational definitions.