Julie J. McGowan

Formative Evaluation: A Critical Component in EHR Implementation

This Viewpoint paper has grown out of a presentation at the American College of Medical Informatics 2007 Winter Symposium, the resulting discussion, and several activities that have coalesced around an issue that most informaticians accept as true but is not commonly considered during the implementation of Electronic Health Records (EHR) outside of academia or research institutions. Successful EHR implementation is facilitated and sometimes determined by formative evaluation, usually focusing on process rather than outcomes. With greater federal funding for the implementation of electronic health record systems in health care organizations unfamiliar with research protocols, the need for formative evaluation assistance is growing. Such assistance, in the form of tools and protocols necessary to do formative evaluation and resulting in successful EHR implementations, should be provided by practicing medical informaticians.

Knowledge sharing using codification and collaboration technologies to improve health care: lessons from the public sector

Knowledge management (KM) enables the public sector to support knowledge transfer across organizations and communities. This case study tells the story of how one U.S. Government agency has been able to support change within the health-care industry to adopt and use information and communication technologies. The study focuses on the role and use of codification and collaboration technologies in KM practice. The study also describes the agencys emphasis on evaluation of these techniques in support of continuous quality improvement of KM practice. Building on previous work in KM, the study extends the traditional dialectic on codification and collaboration, blurring the lines between formal and informal forms and suggesting that both approaches may be necessary to achieve desired impacts on government and societal challenges.

Linking health information technology to patient safety and quality outcomes: a bibliometric analysis and review

Objective. To assess the scholarly output of grants funded by the Agency for Healthcare Research and Quality (AHRQ) that published knowledge relevant to the impact of health information technologies on patient safety and quality of care outcomes. Study design We performed a bibliometric analysis of the identified scholarly articles, their journals, and citations. In addition, we performed a qualitative review of the full-text articles and grant documents. Data collection/extraction methods Papers published by AHRQ-funded investigators were retrieved from MEDLINE, journal impact factors were extracted from the 2010 Thompson Reuters Journal Citation Report, citations were retrieved from ISIs Web of Knowledge and Google Scholar. Principal findings. Seventy-two articles met the criteria for review. Most articles addressed one or more of AHRQs outcome goals and focus priorities. The average impact factor for the journals was 4.005 (range: 0.654–28.899). The articles, and their respective grants, represented a broad range of health information technologies. Conclusions. This set of AHRQ-funded research projects addressed the goals and priorities of AHRQ, indicating notable contributions to the scientific knowledge base on the impact of information system use in healthcare.

The future of health IT innovation and informatics: a report from AMIA's 2010 policy meeting

While much attention has been paid to the short-term impact that widespread adoption of health information technology (health IT) will have on the healthcare system, there is a corresponding need to look at the long-term effects that extant policies may have on health IT system resilience, innovation, and related ethical, social/legal issues. The American Medical Informatics Associations 2010 Health Policy Conference was convened to further the national discourse on the issues surrounding these longer-term considerations. Conference participants self-selected into three broad categories: resilience in healthcare and health IT; ethical, legal, and social challenges; and innovation, adoption, and sustainability. The discussions about problem areas lead to findings focusing on the lack of encouragement for long-term IT innovation that may result from current health IT policies; the potential impact of uneven adoption of health IT based on the exclusions of the current financial incentives; the weaknesses of contingency and risk mitigation planning that threaten system resilience; and evolving standards developed in response to challenges relating to the security, integrity, and availability of electronic health information. This paper discusses these findings and also offers recommendations that address the interwoven topics of innovation, resilience, and adoption. The goal of this paper is to encourage public and private sector organizations that have a role in shaping health information policy to increase attention to developing a national strategy that assures that health IT innovation and resilience are not impeded by shorter-term efforts to implement current approaches emphasizing adoption and meaningful use of electronic health records.

A model for assessing information retrieval and application skills of medical students.

Purpose To develop and evaluate a model for assessing information retrieval and application skills, and to compare the performances on the assessment exercises of students who were and were not instructed in these skills. Method The authors developed a set of four examination stations, each with multiple subtasks, and administered the exams to students at two medical schools. Students at one school had intensive instruction in literature searching and filtering skills for information quality (instructed group), and those at the other school had minimal instruction in these areas (uninstructed group). The stations addressed pediatrics content and the skills of searching Medline and the World Wide Web, evaluating research articles, evaluating the accuracy of information from the Web, and using the information to make recommendations to patients. The authors determined the psychometric characteristics of the stations and compared the performances of the two groups of students. Results Students in the instructed group performed significantly better and with less variability than the uninstructed group on four tasks and no differently on seven tasks. There was no task on which the uninstructed group performed significantly better than the instructed group. Conclusion The prototype stations showed predictable differences across curricula, indicating that they have promise as assessment tools for the essential skills of information retrieval and application.

The Pervasiveness of Telemedicine: Adoption With or Without a Research Base

Arguably, telemedicine could claim that it is the oldest of the health information technologies, dating from the first telephone consultation in the 1800s. Modern telemedicine encompasses a variety of technologies and uses, ranging from direct patient care to both patient and provider education. Because of this, the definition of telemedicine remains elusive but its disparate applications belong in this special issue of Journal of General Internal Medicine (JGIM).

Proposed Curricular Objectives to Teach Physicians Competence in Using the World Wide Web

As the World Wide Web becomes more ubiquitous, physicians are increasingly using the information it provides as part of medical practice. The Web can be a valuable information resource for patient education and decision support or it can be a troubling source of misinformation for providers and patients alike. Attempts have been made to apply quality standards to Web sites, but these have been only moderately successful. Medical schools are designing curricula to confront the content of the medical information explosion, but they also have an obligation to teach students how to quality filter and critically analyze the sources of this information, particularly when it is delivered over the Web. The authors suggest some of the concepts that need to be considered when educating the next generation of what they have termed “information-astute physicians.”

Potential Unintended Consequences of Health Information Exchange

ABSTRACTAccountable models of care delivery demand that health care provider organizations be able to exchange clinical data about the patient. The “Meaningful Use” program is helping to advance health information exchange by requiring physicians and hospitals to exchange clinical data about patients in order to qualify for incentive payments for electronic health records. Early studies demonstrate that the ability to exchange clinical data among provider organizations has the potential to improve clinical care. However, as with any technology, there is a risk of unintended consequences from health information exchange. This manuscript outlines seven aspects of health information exchange that, if not managed properly, may lead to unintended consequences. These categories are: (1) the desire for complete, accurate and timely data for decision making, (2) data management and presentation, (3) assuring routine use of health information exchange, (4) consideration of patient perceptions and concerns, (5) reputational and financial concerns, (6) technical issues and (7) administrative aspects of health information exchange. Education about the capabilities and limitations of health information exchange, along with checklists to support proper implementation and assure that systems are being used as planned, can mitigate risks and help to realize the promise of this powerful technology.

Evaluating Healthcare Information Technology Outside of Academia: Observations from the National Resource Center for Healthcare Information Technology at the Agency for Healthcare Research and Quality

The National Resource Center for Health Information Technology (NRC) was formed in the fall of 2004 as part of the Agency for Healthcare Research and Quality (AHRQ) health IT portfolio to support its grantees. One of the core functions of the NRC was to assist grantees in their evaluation efforts of Health IT. This manuscript highlights some common challenges experienced by health IT project teams at nonacademic institutions, including inappropriately scoped and resourced evaluation efforts, inappropriate choice of metrics, inadequate planning for data collection and analysis, and lack of consideration of qualitative methodologies. Many of these challenges can be avoided or overcome. The strategies adopted by various AHRQ grantees and the lessons learned from their projects should become part of the toolset for current and future implementers of health IT as the nation moves rapidly towards its widespread adoption.

Use of social network analysis tools to validate a resources infrastructure for interinstitutional translational research: a case study*

QUESTION How can knowledge management and innovative technology, cornerstones of library practice, be leveraged to validate the progress of Clinical and Translational Science Awards? SETTING The Indiana Clinical and Translational Sciences Institute (Indiana CTSI) promotes interdisciplinary research across academic institutions. METHODS Using social networking tools and knowledge management skills enabled the department of knowledge informatics and translation to create a visualization of utilization of resources across different Indiana CTSI programs and coauthorship and citation patterns. RESULTS Contacts with different resources per investigator increased; every targeted program was shown to be linked to another. Analysis of publications established a baseline to further analyze the scientific contribution of Indiana CTSI projects. CONCLUSION Knowledge management and social networking utilities validated the efficacy of the Indiana CTSI resources infrastructure and demonstrated visualization of collaboration. The bibliometric analysis of publications provides a basis for assessing longer-term contributions of support to scientific discovery and transdisciplinary science.