Deborah J. Cohen

Evaluative Criteria for Qualitative Research in Health Care: Controversies and Recommendations

PURPOSE We wanted to review and synthesize published criteria for good qualitative research and develop a cogent set of evaluative criteria. METHODS We identified published journal articles discussing criteria for rigorous research using standard search strategies then examined reference sections of relevant journal articles to identify books and book chapters on this topic. A cross-publication content analysis allowed us to identify criteria and understand the beliefs that shape them. RESULTS Seven criteria for good qualitative research emerged: (1) carrying out ethical research; (2) importance of the research; (3) clarity and coherence of the research report; (4) use of appropriate and rigorous methods; (5) importance of reflexivity or attending to researcher bias; (6) importance of establishing validity or credibility; and (7) importance of verification or reliability. General agreement was observed across publications on the first 4 quality dimensions. On the last 3, important divergent perspectives were observed in how these criteria should be applied to qualitative research, with differences based on the paradigm embraced by the authors. CONCLUSION Qualitative research is not a unified field. Most manuscript and grant reviewers are not qualitative experts and are likely to embrace a generic set of criteria rather than those relevant to the particular qualitative approach proposed or reported. Reviewers and researchers need to be aware of this tendency and educate health care researchers about the criteria appropriate for evaluating qualitative research from within the theoretical and methodological framework from which it emerges.

Bridging Primary Care Practices and Communities to Promote Healthy Behaviors

BACKGROUND Primary care practices able to create linkages with community resources may be more successful at helping patients to make and sustain health behavior changes. METHODS Health behavior-change interventions in eight practice-based research networks were examined. Data were collected July 2005-October 2007. A comparative analysis of the data was conducted to identify and understand strategies used for linking primary care practices with community resources. RESULTS Intervention practices developed three strategies to initiate and/or implement linkages with community resources: pre-identified resource options, referral guides, and people external to the practice who offered support and connection to resources. To initiate linkages, practices required the capacity to identify patients, make referrals, and know area resources. Linkage implementation could still be defeated if resources were not available, accessible, affordable, and perceived as valuable. Linkages were facilitated by boundary-spanning strategies that compensated for the lack of infrastructure between practices and resources, and by brokering strategies that identified interested community partners and aided mutually beneficial connections with them. Linkages were stronger when they incorporated practice or resource abilities to motivate the patient, such as brief counseling or postreferral outreach. Further, data suggested that sustaining linkages requires continuous attention and ongoing communication between practices and resources. CONCLUSIONS Creating linkages between primary care practices and community resources has the potential to benefit both patients and clinicians and to lessen the burden on the U.S. healthcare system resulting from poor health behaviors. Infrastructure support and communication systems must be developed to foster sustainable linkages between practices and local resources.

Implementing Health Behavior Change in Primary Care: Lessons From Prescription for Health

PURPOSE Our objective was to identify themes that emerged from the evaluation of 17 interventions funded by the Robert Wood Johnson Foundation’s Prescription for Health that aimed to enhance adherence to healthy behaviors in the primary care setting. METHODS We performed a content analysis of diary data from this 16-month initiative. Other data sources used to complement this analysis include funded grant applications and field notes from interviews with investigative teams and a limited number of site visits. Participants were 17 practice-based research networks (PBRNs) that had projects funded during Round 1 of Prescription for Health. RESULTS Five themes emerged regarding implementation of health behavior change: (1) health behavior change resources are enthusiastically received by practices and patients, and when given a choice, patients prefer methods of assistance that involve personal contact; (2) practice extenders require extensive training, as well as careful case management and support, in order to function fully and avoid burnout; (3) integrating behavior change tools into the primary care setting requires time, effort, and often specialized expertise; (4) even simple interventions require practice change, and use of a practice change model to guide implementation efforts is crucial; and (5) research philosophy and project management approaches vary across PBRNs and have implications for the potential sustainability of an intervention. CONCLUSIONS A more versatile, multifaceted solution involving new tools, technologies, and multidisciplinary care teams is needed in order to integrate health behavior change into everyday primary care routines. Even the best interventions require a model to articulate how to integrate an innovation into practices.

Online Diaries for Qualitative Evaluation: Gaining Real-Time Insights

Interactive online diaries are a novel approach for evaluating project implementation and provide real-time communication between evaluation staff members and those implementing a program. Evaluation teams can guide the lines of inquiry and ensure that data are captured as implementation develops. When used together with conventional evaluation strategies, interactive online diaries allow for an in-depth understanding of project implementation, as well as the challenges program staff members confront and the solutions they develop. Interactive online diaries also can help evaluators address challenges such as self-reporting bias, documenting project evolution, and capturing implementers’ ongoing insights as they develop. These insights might otherwise be lost to the evaluation process. The authors describe the development and use of this online approach in the evaluation of a foundation-sponsored program to improve the provision of preventive care in physicians’ offices. The program included 17 practice-based research networks and their participating primary care practices.

Electronic Medical Records Are Not Associated With Improved Documentation in Community Primary Care Practices

The adoption of electronic medical records (EMRs) in ambulatory settings has been widely recommended. It is hoped that EMRs will improve care; however, little is known about the effect of EMR use on care quality in this setting. This study compares EMR versus paper medical record documentation of basic health history and preventive service indicators in 47 community-based practices. Differences in practice-level documentation rates between practices that did and did not use an EMR were examined using the Kruskal—Wallis nonparametric test and robust regression, adjusting for practice-level covariates. Frequency of documentation of health history and preventive service indicator items were similar in the 2 groups of practices. Although EMRs provide the capacity for more robust record keeping, the community-based practices here do not use EMRs to their full capacity. EMR usage does not guarantee more systematic record keeping and thus may not lead to improved quality in the community practice setting.