Marianna B. Shershneva

Commitment to Practice Change: An Evaluator’s Perspective

A commitment to practice change (CTC) approach may be used in educational program evaluation to document practice changes, examine the educational impact relative to the instructional focus, and improve understanding of the learning-to-change continuum. The authors reviewed various components and procedures of this approach and discussed some practical aspects of its application using an example of a study evaluating a presentation on menopausal care for primary care physicians. The CTC approach is a valuable evaluation tool, but it requires supplementation with other data to have a complete picture of the impact of education on practice. From the evaluation perspective, the self-reported nature of the CTC data is a major limitation of this method.

Peering inside the clock: Using success case method to determine how and why practice-based educational interventions succeed†

Introduction: No educational method or combination of methods will facilitate implementation of clinical practice guidelines in all clinical contexts. To develop an empirical basis for aligning methods to contexts, we need to move beyond “Does it work?” to also ask “What works for whom and under what conditions?” This study employed Success Case Method to understand how 3 performance improvement CME activities contributed to implementation of tobacco cessation practice guidelines in 9 outpatient practices. Methods: Success criteria were applied to clinical data from 93 practices, generating a pool of 14 success cases; 9 were recruited into the study. We conducted semistructured telephone interviews with 1 to 4 informants in each practice. Individual case reports were developed summarizing changes made, what was done to effect the changes, relevant contextual factors, and contributions of the educational interventions to change. A cross‐case analysis followed. Results: Twenty informants were interviewed. Practice changes varied in number and degree. Implementation mechanisms included acquisition of new knowledge and skills, making improving cessation practice an active goal, engaging the clinical team, adopting a more proactive approach with smokers, and making smokers and clinical practice performance more visible. Contextual factors influencing the implementation process were also identified. Discussion: The study shows that (1) the appropriate target of an educational intervention may be a team rather than an individual, (2) implementing even relatively simple practice guidelines can be a complex process, and (3) change requires scientific and practical knowledge. A richer understanding of implementation mechanisms and contextual factors is needed to guide educational planning.

Evaluation of a Collaborative Program on Smoking Cessation: Translating Outcomes Framework Into Practice

Introduction: Although evaluating at multiple outcome levels has been proposed for continuing education activities and programs, it is a complex undertaking and is not done routinely, especially in collaborative, multicomponent programs. This article reports on strategies used and results obtained in an evaluation project that examined multiple outcomes of a US‐based collaborative, multicomponent smoking cessation educational program for clinicians. Methods: Evaluation was organized conceptually around the 6 levels of an outcomes‐based evaluation model and was conducted using registration data, postactivity evaluations, clinical vignettes with questions assessing knowledge and competence in participants and in a comparison group, a commitment to change approach, data from patient charts to assess clinician compliance on 8 performance measures, and tobacco cessation rates. Additional methods included a success case method study of 9 practices participating in performance improvement (PI) activities and assessment of partner collaboration using a written survey and interviews. Results: The program reached more than 43,000 clinicians who participated in a variety of activities. Participants indicated a high level of satisfaction with the programs educational activities and demonstrated higher scores than a comparison group on 6 of 7 competencies. The majority of participants who responded to commitment to change questions reported intended and implemented practice changes consistent with desired outcomes. Performance outcomes of 3 PI activities varied, with greater improvements observed in 1 activity (9.0% to 36.2% improvement across 8 measures). Lower performance outcomes, but a smoking quit rate of 46.8%, was observed in 2 other PI activities. Discussion: The program had an overall positive impact on the measured variables for clinicians and their patients. Use of 1 outcomes assessment framework acceptable to all members of the collaborative, common measures and evaluation techniques, and centralized data repositories contributed to the success of the program evaluation reported here and is recommended to others who are considering a collaborative program evaluation.

Typical Time to Treatment of Patients With Lung Cancer in a Multisite, US-Based Study

INTRODUCTION The importance of high-quality, timely lung cancer care and the need to have indicators to measure timeliness are increasingly discussed in the United States. This study explored when and why delays occur in lung cancer care and compared timeliness between two states with divergent disease incidence. METHODS Patients with small-cell or non-small-cell lung cancer were recruited through cancer centers, outpatient clinics, and community approaches, and interviewed over the phone. Statistical analysis of patient-reported dates included descriptive statistics and comparing time intervals between states and across the sites with Mann-Whitney U tests. Additionally, data from patients with longer timelines were qualitatively analyzed to identify possible reasons for delays. RESULTS On the basis of the dates reported by 275 patients, the median time from first presentation to a clinician to treatment was 52 days; 29% of patients experienced a wait of 90 days or more. Median times for key intervals were 36.5 days from abnormal radiograph to treatment, 9.5 days from initial presentation to specialist referral, 15 days from patient informed of diagnosis to first therapy, and 16 days from referral to treatment to first therapy. More than one quarter of patients perceived delays in care. No significant differences in length of time intervals were identified between states. Monitoring of small nodules, missed diagnosis, and other reasons for longer timelines were documented. CONCLUSION Results defined typical time to treatment of patients with lung cancer across a variety of health systems and should facilitate establishing metrics for determining timeliness of lung cancer care.

Smoking cessation: a community-based approach to continuing medical education

ABSTRACTContinuing medical education can help close the gaps between current and desired tobacco cessation practices. This paper reports a case of an innovative community-based continuing education approach implemented by a multi-organizational initiative aimed at increasing smoking cessation rates among adults in the USA. The approach involved collaborative partnerships with healthcare professionals and other stakeholders in 14 communities where smoking cessation was an established priority. The centralized evidence-based educational curriculum was delivered locally to more than 15,600 clinicians. Evaluation provided evidence of positive impact on clinicians, healthcare systems, and communities. A collaborative, community-based approach to continuing medical education has potential to increase tobacco cessation rates by leveraging efforts of multiple stakeholders operating at the community level into more effective and sustainable tobacco cessation projects. Future research is needed to study effectiveness of and appropriate evaluation frameworks for this approach.

Facilitating multiorganizational smoking cessation knowledge translation through on-line toolkit for educators and clinicians

Effective approaches are needed to facilitate knowledge translation into clinical practice. In 2008, 9 US-based organizations, including 3 universities, launched the Cease Smoking Today ~CS2day! initiative to enhance the implementation of a recently updated tobacco cessation guideline and improve clinical practice of helping adults quit smoking. These goals were to be achieved through more than 140 continuingeducation activities for physicians, pharmacists, and other health care professionals over a period of 2 years. As part of this initiative we developed an on-line Toolkit—a compilation of educational and practice-oriented resources for educators and clinicians.