Thomas R. Eng

Published criteria for evaluating health related web sites: review

Abstract Objective: To review published criteria for specifically evaluating health related information on the world wide web, and to identify areas of consensus. Design: Search of world wide web sites and peer reviewed medical journals for explicit criteria for evaluating health related information on the web, using Medline and Lexis-Nexis databases, and the following internet search engines: Yahoo!, Excite, Altavista, Webcrawler, HotBot, Infoseek, Magellan Internet Guide, and Lycos. Criteria were extracted and grouped into categories. Results: 29 published rating tools and journal articles were identified that had explicit criteria for assessing health related web sites. Of the 165 criteria extracted from these tools and articles, 132 (80%) were grouped under one of 12 specific categories and 33 (20%) were grouped as miscellaneous because they lacked specificity or were unique. The most frequently cited criteria were those dealing with content, design and aesthetics of site, disclosure of authors, sponsors, or developers, currency of information (includes frequency of update, freshness, maintenance of site), authority of source, ease of use, and accessibility and availability. Conclusions: Results suggest that many authors agree on key criteria for evaluating health related web sites, and that efforts to develop consensus criteria may be helpful. The next step is to identify and assess a clear, simple set of consensus criteria that the general public can understand and use. Key messages Many organisations and individuals have published criteria to evaluate health related information on the world wide web A literature and world wide web search found that the most frequently cited criteria were those dealing with content, design and aesthetics of site, disclosure of authors, sponsors, or developers, currency of information, authority of source, and ease of use Criteria related to confidentiality and privacy were only cited by one author Consensus regarding critical criteria for evaluation of web based health information seems to be emerging Our results indicate that many authors agree on key criteria for evaluating health related web sites, and that efforts to develop a set of key criteria may be helpful

INTRODUCTION TO EVALUATION OF INTERACTIVE HEALTH COMMUNICATION APPLICATIONS

Virtually all aspects of society have been altered in some way by advances in computer and communication technologies. In 1997, the information technology industry was the single largest industry in the United States in terms of sales and accounted for 33% of the growth in GDP in 1996.1,2 An estimated 41.5 million U.S. adults were active users of the Internet in 1997,3 and more than 43% of Internet users have used it to research health information.4 At the same time that these new technologies have emerged, consumers seem to be demanding increasing access to a wide range of information, including health information, and social support as a vehicle for recovering from illness. Consumer demand for health information and the availability of new media technologies have spurred substantial interest in interactive health communication (IHC), the interaction of an individual—consumer, patient, caregiver, or professional—with or through an electronic device or communication technology to access or transmit health information or receive guidance and support on a health-related issue.5 Using this definition, IHC encompasses technology-mediated health communication and does not include direct communication such as face-to-face clinician-patient counseling. The panel chose the term IHC because it focuses on the content rather than on the technology that facilitates IHC. The panel uses the term IHC applications to refer to the operational software programs or modules that interface with the end user. This includes health information and support Web sites and clinical decision-support and risk assessment software (which may or may not be online), but does not include applications that focus exclusively on administrative, financial, or clinical data, such as electronic medical records, dedicated clinical telemedicine applications, or expert clinical decision-support systems for providers. Some of these latter applications, however, are integrated with health communication functions. The panel uses the term IHC technologies to refer to the hardware and infrastructure technologies that run or disseminate IHC applications, such as networks, computers, telecommunications equipment and the like. IHC applications are increasingly accessible to the public through the Internet and non-networked technologies, such as stand-alone computers and kiosks.6,7 Their major functions are to: (1) relay information, From the Office of Disease Prevention and Health Promotion, US Department of Health and Human Services (Eng), Washington, DC; University of Wisconsin-Madison (Gustafson), Madison, WI; Dartmouth Medical School (Henderson), Dartmouth, NH; Oregon Health Sciences University (Jimison), Portland, OR; and San Diego State University and the University of California, San Diego (Patrick), San Diego, CA. Address correspondence to: Thomas R. Eng, VMD, MPH, Office of Disease Prevention and Health Promotion, HHS, 200 Independence Avenue, SW, Room 738G, Washington, DC 20201. Address reprint requests to: Mary Jo Deering, PhD, Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services, 200 Independence Avenue, SW, Washington, DC 20201. Other panel members and staff: Linda Adler, MPH, MA, National Member Technology Group, Kaiser Permanente, Oakland, CA; Farrokh Alemi, PhD, Cleveland State University, Cleveland, OH; David Ansley, Consumer Reports, Yonkers, NY; Patricia Flatley Brennan, RN, PhD, FAAN, School of Nursing and College of Engineering, University of Wisconsin-Madison, Madison, WI; Molly Joel Coye, MD, MPH, The Lewin Group, San Francisco, CA; Mary Jo Deering, PhD, Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services, Washington, DC; Albert Mulley Jr, MD, MPP, Massachusetts General Hospital, Boston, MA; John Noell, PhD, Oregon Center for Applied Science, Inc. and Oregon Research Institute, Eugene, OR; Thomas C. Reeves, PhD, University of Georgia, Athens, GA; Thomas N. Robinson, MD, MPH, Stanford University School of Medicine, Palo Alto, CA; and Victor Strecher, PhD, MPH, University of Michigan Comprehensive Cancer Center, Ann Arbor, MI. For example, less than one year after free Medline searches became available on the Web, the number of searches increased 10-fold, and 30% of users were members of the general public (Testimony of Dr. Donald A.B. Lindberg, Director, National Library of Medicine to the House Appropriations Sub-Committee on Labor, HHS and Education, March 18, 1998. Accessed on April 6, 1998. Available from: URL: http://www.nlm.nih.gov/pubs/staffpubs/od/budget99.html For example, a search for the keyword “health” on the World Wide Web using common search engines yielded more than 16 thousand indexed Web sites (www.yahoo.com) and 20 million matching Web pages (altavista.digital.com) on October 28, 1998.

Policy issues relevant to evaluation of interactive health communication applications

This article provides an analysis of policy-related issues associated with the evaluation of interactive health communication (IHC) applications. These include an assessment of the current health and technology policy environment pertinent to public (government, education, public health) and private (medical care providers, purchasers, consumers, IHC developers) IHC stakeholders and discussion of issues likely to merit additional consideration by these stakeholders in the future.

Population Health Technologies Emerging Innovations for the Health of the Public

Abstract At the beginning of the 21st century, we are at the dawn of a possibly unprecedented era of scientific discovery and promise. Emerging technologies, including information and communication technologies, genomics, microelectromechanical systems, robotics, sensors, and nanotechnologies, provide enormous opportunities for population health improvement. Population health technology refers to the application of an emerging technology to improve the health of populations. Emerging technologies present an opportunity for addressing global health challenges—in both developed and developing countries. Health issues ripe for the application of new technologies include disease surveillance and control, environmental monitoring and pollution prevention, food safety, health behavior change, self-care, population screening, and chronic disease and injury prevention and control. If appropriately applied, population health technologies may greatly enhance existing health intervention models. However, potential adverse consequences could arise related to privacy, confidentiality, and security; quality and effectiveness; sustainability; and the technology divide. To ensure the optimal development and diffusion of population health technologies will require balancing these risks and benefits while simultaneously adopting new mechanisms of public and private support for research and development in this potentially important new domain of public health.

Health care providers and purchasers and evaluation of interactive health communication applications

Health care providers and purchasers of health services have an opportunity to improve patient care and potentially save costs through the wise purchase of interactive health communication applications for patients and employees. Purchasing decisions based on evaluation and evidence should drive the design and development of new systems. The cycle of evaluation includes a needs assessment before system development, usability testing during development, and studies of use and outcomes in natural settings. This type of evidence is critical to our understanding of how best to provide health information and decision assistance to patients, employees, and others.

PREVENTION OF SEXUALLY TRANSMITTED DISEASES : A MODEL FOR OVERCOMING BARRIERS BETWEEN MANAGED CARE AND PUBLIC HEALTH

CONTEXT The growth of managed care has spurred re-evaluation of the roles and responsibilities of public health agencies and private health plans for providing public health services. Although rates of curable sexually transmitted diseases (STDs) in the United States are the highest in the developed world, many clinicians and managed care organizations are not systematically providing high-quality, comprehensive STD-related services to their patients and the community. OBJECTIVE To examine issues around managed care and STD prevention as a model for overcoming barriers that impede managed care organizations from providing comprehensive public health services and collaborating with health agencies. SETTING Two-day invitational workshop. PARTICIPANTS Representatives from 18 health plans, 10 public health agencies, 6 academic institutions, 1 purchasing coalition, and 5 other health organizations. RESULTS Major obstacles include: turnover and heterogeneity in the health care system; deficiencies in clinical knowledge and skills; differences in organizational culture and language; low priority of STDs; inadequate public health surveillance data and performance measures; confidentiality concerns; and lack of coverage for sex partners. CONCLUSIONS Potential approaches for addressing these barriers include: requiring that STD-related services be covered by Medicaid managed care programs; implementing performance measures; requiring collaborative activities; promoting education of and outreach to stakeholders; funding of pilot projects; and researching the cost-benefit and cost-effectiveness of STD-related services for various populations.Abstract Context: The growth of managed care has spurred re-evaluation of the roles and responsibilities of public health agencies and private health plans for providing public health services. Although rates of curable sexually transmitted diseases (STDs) in the United States are the highest in the developed world, many clinicians and managed care organizations are not systematically providing high-quality, comprehensive STD-related services to their patients and the community. Objective: To examine issues around managed care and STD prevention as a model for overcoming barriers that impede managed care organizations from providing comprehensive public health services and collaborating with health agencies. Setting: Two-day invitational workshop. Participants: Representatives from 18 health plans, 10 public health agencies, 6 academic institutions, 1 purchasing coalition, and 5 other health organizations. Results: Major obstacles include: turnover and heterogeneity in the health care system; deficiencies in clinical knowledge and skills; differences in organizational culture and language; low priority of STDs; inadequate public health surveillance data and performance measures; confidentiality concerns; and lack of coverage for sex partners. Conclusions: Potential approaches for addressing these barriers include: requiring that STD-related services be covered by Medicaid managed care programs; implementing performance measures; requiring collaborative activities; promoting education of and outreach to stakeholders; funding of pilot projects; and researching the cost-benefit and cost-effectiveness of STD-related services for various populations.CONTEXT The growth of managed care has spurred re-evaluation of the roles and responsibilities of public health agencies and private health plans for providing public health services. Although rates of curable sexually transmitted diseases (STDs) in the United States are the highest in the developed world, many clinicians and managed care organizations are not systematically providing high-quality, comprehensive STD-related services to their patients and the community. OBJECTIVE To examine issues around managed care and STD prevention as a model for overcoming barriers that impede managed care organizations from providing comprehensive public health services and collaborating with health agencies. SETTING Two-day invitational workshop. PARTICIPANTS Representatives from 18 health plans, 10 public health agencies, 6 academic institutions, 1 purchasing coalition, and 5 other health organizations. RESULTS Major obstacles include: turnover and heterogeneity in the health care system; deficiencies in clinical knowledge and skills; differences in organizational culture and language; low priority of STDs; inadequate public health surveillance data and performance measures; confidentiality concerns; and lack of coverage for sex partners. CONCLUSIONS Potential approaches for addressing these barriers include: requiring that STD-related services be covered by Medicaid managed care programs; implementing performance measures; requiring collaborative activities; promoting education of and outreach to stakeholders; funding of pilot projects; and researching the cost-benefit and cost-effectiveness of STD-related services for various populations.

Immunogenicity of rabies vaccines used during an urban epizootic of rabies in Mexico

From 1 July 1987 to 31 December 1988, 30% of 247 rabid dogs in Hermosillo, Mexico had a positive history of rabies vaccination. Serosurveys suggested that inactivated suckling mouse brain vaccine (INACT-SMBV) and inactivated tissue culture vaccine (INACT-TC) used before and during the epizootic were poor immunogens. Prospective studies showed that only about one-third of dogs vaccinated with INACT-SMBV were seropositive 5 weeks after vaccination. Lack of vaccine potency was the most likely cause of poor immunogenicity. Rabies vaccines should be evaluated periodically by measuring antibody responses in animals. In some circumstances, minimum seroconversion rates and antibody titres in vaccinated animals may be better measures of immunogenicity than relative potency.