P. Adam Kelly

Perspective: Guidelines for Reporting Team-Based Learning Activities in the Medical and Health Sciences Education Literature

Medical and health sciences educators are increasingly employing team-based learning (TBL) in their teaching activities. TBL is a comprehensive strategy for developing and using self-managed learning teams that has created a fertile area for medical education scholarship. However, because this method can be implemented in a variety of ways, published reports about TBL may be difficult to understand, critique, replicate, or compare unless authors fully describe their interventions. The authors of this article offer a conceptual model and propose a set of guidelines for standardizing the way that the results of TBL implementations are reported and critiqued. They identify and articulate the seven core design elements that underlie the TBL method and relate them to educational principles that maximize student engagement and learning within teams. The guidelines underscore important principles relevant to many forms of small-group learning. The authors suggest that following these guidelines when writing articles about TBL implementations should help standardize descriptive information in the medical and health sciences education literature about the essential aspects of TBL activities and allow authors and reviewers to successfully replicate TBL implementations and draw meaningful conclusions about observed outcomes.

Overtreatment of asymptomatic bacteriuria: Identifying provider barriers to evidence-based care

BACKGROUNDnInappropriate use of antibiotics to treat asymptomatic bacteriuria (ASB) is a significant contributor to antibiotic overuse in hospitalized patients despite evidence-based guidelines on ASB management. We surveyed whether accurate knowledge of how to manage catheter-associated urine cultures was associated with level of training, familiarity with ASB guidelines, and various cognitive-behavioral constructs.nnnMETHODSnWe used a survey to measure respondents knowledge of how to manage catheter-associated bacteriuria, familiarity with the content of the relevant Infectious Diseases Society of America guidelines, and cognitive-behavioral constructs. The survey was administered to 169 residents and staff providers.nnnRESULTSnThe mean knowledge score was 57.5%, or slightly over one-half of the questions answered correctly. The overall knowledge score improved significantly with level of training (P < .0001). Only 42% of respondents reported greater than minimal recall of ASB guideline contents. Self-efficacy, behavior, risk perceptions, social norms, and guideline familiarity were individually correlated with knowledge score (P < .01). In multivariable analysis, behavior, risk perception, and year of training were correlated with knowledge score (P < .05).nnnCONCLUSIONSnKnowledge of how to manage catheter-associated bacteriuria according to evidence-based guidelines increases with experience. Addressing both knowledge gaps and relevant cognitive biases early in training may decrease the inappropriate use of antibiotics to treat ASB.

Development and validation of an algorithm to recalibrate mental models and reduce diagnostic errors associated with catheter-associated bacteriuria.

BackgroundOvertreatment of catheter-associated bacteriuria is a quality and safety problem, despite the availability of evidence-based guidelines. Little is known about how guidelines-based knowledge is integrated into clinicians’ mental models for diagnosing catheter-associated urinary tract infection (CA-UTI). The objectives of this research were to better understand clinicians’ mental models for CA-UTI, and to develop and validate an algorithm to improve diagnostic accuracy for CA-UTI.MethodsWe conducted two phases of this research project. In phase one, 10 clinicians assessed and diagnosed four patient cases of catheter associated bacteriuria (n= 40 total cases). We assessed the clinical cues used when diagnosing these cases to determine if the mental models were IDSA guideline compliant. In phase two, we developed a diagnostic algorithm derived from the IDSA guidelines. IDSA guideline authors and non-expert clinicians evaluated the algorithm for content and face validity. In order to determine if diagnostic accuracy improved using the algorithm, we had experts and non-experts diagnose 71 cases of bacteriuria.ResultsOnly 21 (53%) diagnoses made by clinicians without the algorithm were guidelines-concordant with fair inter-rater reliability between clinicians (Fleiss’ kappa = 0.35, 95% Confidence Intervals (CIs) = 0.21 and 0.50). Evidence suggests that clinicians’ mental models are inappropriately constructed in that clinicians endorsed guidelines-discordant cues as influential in their decision-making: pyuria, systemic leukocytosis, organism type and number, weakness, and elderly or frail patient. Using the algorithm, inter-rater reliability between the expert and each non-expert was substantial (Cohen’s kappa = 0.72, 95% CIs = 0.52 and 0.93 between the expert and non-expert #1 and 0.80, 95% CIs = 0.61 and 0.99 between the expert and non-expert #2).ConclusionsDiagnostic errors occur when clinicians’ mental models for catheter-associated bacteriuria include cues that are guidelines-discordant for CA-UTI. The understanding we gained of clinicians’ mental models, especially diagnostic errors, and the algorithm developed to address these errors will inform interventions to improve the accuracy and reliability of CA-UTI diagnoses.

Predicting implementation from organizational readiness for change: a study protocol

BackgroundThere is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment.ObjectivesOur objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias.Methods and DesignWe will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique.DiscussionWe propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs.

A hospital-site controlled intervention using audit and feedback to implement guidelines concerning inappropriate treatment of catheter- associated asymptomatic bacteriuria

BackgroundCatheter-associated urinary tract infection (CAUTI) is one of the most common hospital-acquired infections. However, many cases treated as hospital-acquired CAUTI are actually asymptomatic bacteriuria (ABU). Evidence-based guidelines recommend that providers neither screen for nor treat ABU in most catheterized patients, but there is a significant gap between these guidelines and clinical practice. Our objectives are (1) to evaluate the effectiveness of an audit and feedback intervention for increasing guideline-concordant care concerning catheter-associated ABU and (2) to measure improvements in healthcare providers knowledge of and attitudes toward the practice guidelines associated with the intervention.Methods/DesignThe study uses a controlled pre/post design to test an intervention using audit and feedback of healthcare providers to improve their compliance with ABU guidelines. The intervention and the control sites are two VA hospitals. For objective 1 we will review medical records to measure the clinical outcomes of inappropriate screening for and treatment of catheter-associated ABU. For objective 2 we will survey providers knowledge and attitudes. Three phases of our protocol are proposed: the first 12-month phase will involve observation of the baseline incidence of inappropriate screening for and treatment of ABU at both sites. This surveillance for clinical outcomes will continue at both sites throughout the study. Phase 2 consists of 12 months of individualized audit and feedback at the intervention site and guidelines distribution at both sites. The third phase, also over 12 months, will provide unit-level feedback at the intervention site to assess sustainability. Healthcare providers at the intervention site during phase 2 and at both sites during phase 3 will complete pre/post surveys of awareness and familiarity (knowledge), as well as of acceptance and outcome expectancy (attitudes) regarding the relevant practice guidelines.DiscussionOur proposal to bring clinical practice in line with published guidelines has significant potential to decrease overdiagnosis of CAUTI and associated inappropriate antibiotic use. Our study will also provide information about how to maximize effectiveness of audit and feedback to achieve guideline adherence in the inpatient setting.Trial RegistrationNCT01052545

Impact of resident duty hour limits on safety in the intensive care unit: a national survey of pediatric and neonatal intensivists.

Objective: Resident duty-hour regulations potentially shift the workload from resident to attending physicians. We sought to understand how current or future regulatory changes might impact safety in academic pediatric and neonatal intensive care units. Design: Web-based survey. Setting: U.S. academic pediatric and neonatal intensive care units. Subjects: Attending pediatric and neonatal intensivists. Interventions: We evaluated perceptions on four intensive care unit safety-related risk measures potentially affected by current duty-hour regulations: 1) attending physician and resident fatigue; 2) attending physician workload; 3) errors (self-reported rates by attending physicians or perceived resident error rates); and 4) safety culture. We also evaluated perceptions of how these risks would change with further duty-hour restrictions. Measurements and Main Results: We administered our survey between February and April 2010 to 688 eligible physicians, of whom 360 (52.3%) responded. Most believed that resident error rates were unchanged or worse (91.9%) and safety culture was unchanged or worse (84.4%) with current duty-hour regulations. Of respondents, 61.9% believed their own work-hours providing direct patient care increased and 55.8% believed they were more fatigued while providing direct patient care. Most (85.3%) perceived no increase in their own error rates currently, but in the scenario of further reduction in resident duty-hours, over half (53.3%) believed that safety culture would worsen and a significant proportion (40.3%) believed that their own error rates would increase. Conclusions: Pediatric intensivists do not perceive improved patient safety from current resident duty-hour restrictions. Policies to further restrict resident duty-hours should consider unintended consequences of worsening certain aspects of intensive care unit safety.

Pre-Implementation Strategies to Adapt and Implement a Veteran Peer Coaching Intervention to Improve Mental Health Treatment Engagement Among Rural Veterans

PURPOSEnTelephone motivational coaching has been shown to increase urban veteran mental health treatment initiation. However, no studies have tested telephone motivational coaching delivered by veteran peers to facilitate mental health treatment initiation and engagement. This study describes pre-implementation strategies with 8 Veterans Affairs (VA) community-based outpatient clinics in the West and Mid-South United States to adapt and implement a multisite pragmatic randomized controlled trial of telephone peer motivational coaching for rural veterans.nnnMETHODSnWe used 2 pre-implementation strategies, Formative Evaluation (FE) research and Evidence-Based Quality Improvement (EBQI) meetings to adapt the intervention to stakeholders needs and cultural contexts. FE data were qualitative, semi-structured interviews with rural veterans and VA clinic staff. Results were rapidly analyzed and presented to stakeholders during EBQI meetings to optimize the intervention implementation.nnnFINDINGSnFE research results showed that VA clinic providers felt overwhelmed by veterans mental health needs and acknowledged limited mental health services at VA clinics. Rural veteran interviews indicated geographical, logistical, and cultural barriers to VA mental health treatment initiation and a preference for self-care to cope with mental health symptoms. EBQI meetings resulted in several intervention adaptations, including veteran study recruitment, peer veteran coach training, and an expanded definition of mental health care outcomes.nnnCONCLUSIONSnAs the VA moves to cultivate community partnerships in order to personalize and expand access to care for rural veterans, pre-implementation processes with engaged stakeholders, such as those described here, can help guide other researchers and clinicians to achieve proactive and veteran-centered health care services.

Creating a Common Curriculum for the DSM-5: Lessons in Collaboration

In 2013, the introduction of the Fifth Edition of the Diagnostic and Statistical Manual ofMental Disorders (DSM-5) created a challenge and an opportunity for psychiatric educators. The challenge consisted of the necessity of revising a standing curriculum for educating medical students and other learners. The opportunity consisted of the stimulus for innovation and collaboration. We decided to take advantage of the challenge of the introduction of the DSM-5 to collaboratively create a new curriculum that could eventually be shared with others. Since some of us were experienced Team-Based Learning practitioners, and others were not, the innovation also created an opportunity to disseminate knowledge about the pedagogy. There are multiple reasons for organizations to work together around a shared goal [1]. When groups collaborate, they can improve decisionmaking, utilize multiple perspectives to solve complex problems, create synergies to enhance creativity and skill development, and pool resources to quicken responsiveness to evolving conditions. The success of a collaborative effort depends on a variety of factors described in published reports, including the environment, membership, process and structure, communication, purpose, and resources. We believed that the environment for our collaboration was ideal because of our shared need to revise our curricula in response to the introduction of the DSM-5. All members of our collaborative knew each other professionally, and several had previous experience working together. While all of the institutions involved in the collaborative had individual resources sufficient to complete a new curriculum, none were capable of autonomously developing the ambitious and high quality product we envisioned. The membership (e.g., collaborative faculty) included experienced clerkship directors and/or educational researchers and thus was capable of meaningfully contributing to the group effort. All members agreed to the process and structure developed by the primary investigator. We established regular communications to facilitate completion of our project via email, conference calls, and face-to-face visits. A clearly defined and shared purpose was developed based on the ensuing publication of the DSM-5. By pooling resources, the collaboration was able to result in a complete curriculum in a relatively short period of time. Following is a description of how we developed the collaboration and some of the lessons learned though our experiences. * Ruth E. Levine rlevine@utmb.edu

Oncologists’ Views on Using Value to Guide Cancer Treatment Decisions

OBJECTIVESnCancer costs have increased substantially in the past decades, prompting specialty societies to urge oncologists to consider value in clinical decision making. Despite oncologists crucial role in guiding cancer care, current literature is sparse with respect to the oncologists views on value. Here, we evaluated oncologists perceptions of the use and measurement of value in cancer care.nnnMETHODSnWe conducted in-depth, open-ended interviews with 31 US oncologists practicing nationwide in various environments. Oncologists discussed the definition, measurement, and implementation of value. Transcripts were analyzed using matrix and thematic analysis.nnnRESULTSnOncologists definitions of value varied greatly. Some described versions of the standard health economic definition of value, that is, cost relative to health outcomes. Many others did not include cost in their definition of value. Oncologists considered patient goals and quality of life as important components of value that they perceived were missing from current value measurement. Oncologists prioritized a patient-centric view of value over societal or other perspectives. Oncologists were inclined to consider the value of a treatment only if they perceived treatment would pose a financial burden to patients. Oncologists had differing opinions regarding who should be responsible for determining whether care is low value but generally felt this should remain within the purview of the oncology community.nnnCONCLUSIONSnOncologists agreed that cost was an important issue, but disagreed about whether cost was involved in value as well as the role of value in guiding treatment. Better clarity and alignment on the definition of and appropriate way to measure value is critical to the success of efforts to improve value in cancer care.

Development of the Perceived Access Inventory: A patient-centered measure of access to mental health care.

According to recent Congressional testimony by the Secretary for Veterans Affairs (VA), improving the timeliness of services is one of five current priorities for VA. A comprehensive access measure, grounded in veterans experience, is essential to support VAs efforts to improve access. In this article, the authors describe the process they used to develop the Perceived Access Inventory (PAI), a veteran-centered measure of perceived access to mental health services. They used a multiphase, mixed-methods approach to develop the PAI. Each phase built on and was informed by preceding phases. In Phase 1, the authors conducted 80 individual, semistructured, qualitative interviews with veterans from 3 geographic regions to elicit the barriers and facilitators they experienced in seeking mental health care. In Phase 2, they generated a preliminary set of 77 PAI items based on Phase 1 qualitative data. In Phase 3, an external expert panel rated the preliminary PAI items in terms of relevance and importance, and provided feedback on format and response options. Thirty-nine PAI items resulted from Phase 3. In Phase 4, veterans gave feedback on the readability and understandability of the PAI items generated in Phase 3. Following completion of these 4 developmental phases, the PAI included 43 items addressing 5 domains: logistics (five items), culture (three items), digital (nine items), systems of care (13 items), and experiences of care (13 items). Future work will evaluate concurrent and predictive validity, test/retest reliability, sensitivity to change, and the need for further item reduction. (PsycINFO Database Record