Herta Fidler

Helping Babies Breathe: global neonatal resuscitation program development and formative educational evaluation.

OBJECTIVES To develop an educational program designed to train health care providers in resource limited settings to carry out neonatal resuscitation. We analyzed facilitator and learner perceptions about the course, examined skill performance, and assessed the quality of instruments used for learner evaluation as part of the formative evaluation of the educational program Helping Babies Breathe. METHODS Multiple stakeholders and a Delphi panel contributed to program development. Training of facilitators and learners occurred in global field test sites. Course evaluations and focus groups provided data on facilitator and learner perceptions. Knowledge and skill assessments included pre/post scores from multiple choice questions (MCQ) and post-training assessment of bag and mask skills, as well as 2 objective structured clinical evaluations (OSCE). RESULTS Two sites (Kenya and Pakistan) trained 31 facilitators and 102 learners. Participants expressed high satisfaction with the program and high self-efficacy with respect to neonatal resuscitation. Assessment of participant knowledge and skills pre/post-program demonstrated significant gains; however, the majority of participants could not demonstrate mastery of bag and mask ventilation on the post-training assessment without additional practice. CONCLUSIONS Participants in a program for neonatal resuscitation in resource-limited settings demonstrated high satisfaction, high self-efficacy and gains in knowledge and skills. Mastery of ventilation skills and integration of skills into case management may not be achievable in the classroom setting without additional practice, continued learning, and active mentoring in the workplace. These findings were used to revise program structure, materials and assessment tools.

Stillbirth and Newborn Mortality in India After Helping Babies Breathe Training

OBJECTIVE: This study evaluated the effectiveness of Helping Babies Breathe (HBB) newborn care and resuscitation training for birth attendants in reducing stillbirth (SB), and predischarge and neonatal mortality (NMR). India contributes to a large proportion of the worlds annual 3.1 million neonatal deaths and 2.6 million SBs. METHODS: This prospective study included 4187 births at >28 weeks’ gestation before and 5411 births after HBB training in Karnataka. A total of 599 birth attendants from rural primary health centers and district and urban hospitals received HBB training developed by the American Academy of Pediatrics, using a train-the-trainer cascade. Pre-post written trainee knowledge, posttraining provider performance and skills, SB, predischarge mortality, and NMR before and after HBB training were assessed by using χ2 and t-tests for categorical and continuous variables, respectively. Backward stepwise logistic regression analysis adjusted for potential confounding. RESULTS: Provider knowledge and performance systematically improved with HBB training. HBB training reduced resuscitation but increased assisted bag and mask ventilation incidence. SB declined from 3.0% to 2.3% (odds ratio [OR] 0.76, 95% confidence interval [CI] 0.59–0.98) and fresh SB from 1.7% to 0.9% (OR 0.54, 95% CI 0.37–0.78) after HBB training. Predischarge mortality was 0.1% in both periods. NMR was 1.8% before and 1.9% after HBB training (OR 1.09, 95% CI 0.80–1.47, P = .59) but unknown status at 28 days was 2% greater after HBB training (P = .007). CONCLUSIONS: HBB training reduced SB without increasing NMR, indicating that resuscitated infants survived the neonatal period. Monitoring and community-based assessment are recommended.

Changing physicians' practices: the effect of individual feedback.

OBJECTIVE To determine whether physicians who received feedback from six peers, six referring/referral physicians, six co-workers, and 25 patients about 55 aspects of their medical practices (e.g., able to reach doctor by phone after office hours) would make changes to their practices based on that feedback. METHOD In an earlier study, 308 physicians were given feedback about 106 aspects of their practices in the form of mean Likert-scale ratings that (1) the peers made on 26 aspects; (2) the referring/referral physicians made on 23 aspects; (3) the co-workers made on 17 aspects; and (4) the patients made on 40 aspects. Three months later 255 of these physicians responded when asked to indicate whether they had contemplated or initiated changes, or whether no change had been necessary, regarding 31 practice aspects, each of which was a summary of one or more of 55 of the original 106 aspects on which they had received ratings. These 55 were considered the aspects most amenable to change over a short period. The physicians were also asked about the educational interventions that they felt would help them make changes. Multivariate analysis of variance was used to see whether the types of changes reported for the specific aspects of practice were associated with the feedback ratings received for those aspects. RESULTS An examination of the responses showed that 83% of the 255 physicians reported having contemplated a change, and 66% reported having initiated a change for at least one aspect of practice. Changes were contemplated most frequently for aspects of practice associated with clinical skills and resource use. Changes were initiated most frequently for aspects of practice associated with communication with patients and support of patients. Physicians who contemplated or initiated changes had lower (i.e., more negative) mean ratings than did physicians who reported that no change was necessary, which suggests that the physicians did use their feedback ratings to decide about changes, although their qualitative comments indicated other sources as well. Printed material was chosen most often as a method of receiving continuing medical education related to making changes in the practice areas examined.

Feasibility and psychometric properties of using peers, consulting physicians, co-workers, and patients to assess physicians

No abstract available.

Changes in performance: a 5-year longitudinal study of participants in a multi-source feedback programme

Objectives  Multi‐source feedback (MSF) enables performance data to be provided to doctors from patients, co‐workers and medical colleagues. This study examined the evidence for the validity of MSF instruments for general practice, investigated changes in performance for doctors who participated twice, 5 years apart, and determined the association between change in performance and initial assessment and socio‐demographic characteristics.

The Development and Testing of a Performance Checklist to Assess Neonatal Resuscitation Megacode Skill

PURPOSE. The purpose of this work was to develop and assess the feasibility, reliability, and validity of a brief performance checklist to evaluate skills during a simulated neonatal resuscitation (“megacode”) for the Neonatal Resuscitation Program of the American Academy of Pediatrics. METHODS. A performance checklist of items was created, validated, and modified in sequential phases involving: an expert committee, review, and feedback by Neonatal Resuscitation Program instructors for feasibility and criticality and use of the performance checklist by Neonatal Resuscitation Program instructors reviewing videotaped megacodes. The final 20-item performance checklist used a 3-point scale and was assessed by student and instructor volunteers. Megacode scores, the NRP multiple-choice examination scores, student assessments of their ability and performance, and sociodemographic descriptors for both students and instructors were collected. Data were analyzed descriptively. In addition, we assessed the megacode score internal consistency reliability, the correlations between megacode and multiple-choice examination scores, and the variance in scores based on instructor and student characteristics. RESULTS. A total of 468 students and 148 instructors volunteered for the study. The instrument was reliable and internally consistent. Students scores were high on most items. There was a significant but low correlation between the megacode score and the written knowledge examination. Instructor and student characteristics had little effect on the variance in scores. CONCLUSIONS. This performance checklist provides a feasible assessment tool. There is evidence for its reliability and validity.

A multi source feedback program for anesthesiologists

PurposeTo assess the feasibility, validity, and reliability of a multi source feedback program for anesthesiologists.MethodsSurveys with 11, 19, 29 and 29 items were developed for patients, coworkers, medical colleagues and self, respectively, using five-point scales with an ‘unable to assess’ category. The items addressed communication skills, professionalism, collegiality, continuing professional development and collaboration. Each anesthesiologist was assessed by eight medical colleagues, eight coworkers, and 30 patients. Feasibility was assessed by response rates for each instrument. Validity was assessed by rating profiles, the percentage of participants unable to assess the physician for each item, and exploratory factor analyses to determine which items grouped together into scales. Cronbach’s alpha and generalizability coefficient analyses assessed reliability.ResultsOne hundred and eighty-six physicians participated. The mean number and percentage return rate of respondents per physician was 17.7 (56.2%) for patients, 7.8 (95.1%) for coworkers, and 7.8 (94.6%) for medical colleagues. The mean ratings ranged from four to five for each item on each scale. There were relatively few items with high percentages of ‘unable to assess’. The factor analyses revealed a two-factor solution for the patient, a two-factor solution for the coworker and a three-factor solution for the medical colleague survey, accounting for at least 70% of the variance. All instruments had a high internal consistency reliability (Cronbach’s α > 0.95). The generalizability coefficients were 0.65 for patients, 0.56 for coworkers and 0.69 for peers.ConclusionIt is feasible to develop multi source feedback instruments for anesthesiologists that are valid and reliable.RésuméObjectifÉvaluer la faisabilité, la validité et la fiabilité ďun programme de rétroaction multisources pour les anesthésiologistes.MéthodeDes sondages comportant 11, 19, 29 et 29 éléments ont été élaborés pour les patients, les collègues de travail, les collègues médecins et nous-mêmes respectivement, en utilisant des échelles en cinq points dont une catégorie «impossible ďévaluer». Les éléments concernaient les habiletés de communications, le professionnalisme, la collégialité, la formation professionnelle continue et la collaboration. Chaque anesthésiologiste était évalué par huit médecins, huit collègues de travail et 30 patients. La faisabilité a été évaluée par les taux de réponses pour chaque instrument. La validité a été évaluée par les profils de cotation, le pourcentage des participants incapables ďévaluer le médecin pour chaque élément et les analyses factorielles exploratrices pour déterminer quels étaient les éléments regroupables avec ľusage ďune échelle. La fiabilité a été évaluée par les analyses du coefficient Alpha de Cronback et de généralisabilité.RésultatsOn a compté 186 médecins participants. Le nombre moyen de répondants et de pourcentage de questionnaires retournés par médecin a été de 17,7 (56,2 %) pour les patients, 7,8 (95,1 %) pour les collègues de travail et 7,8 (94,6 %) pour les collègues médecins. Les scores moyens étaient de quatre ou cinq pour chaque élément de chaque échelle. Il y a eu relativement peu ďéléments avec de hauts pourcentages de réponses «impossible ďévaluer». Les analyses factorielles ont révélé une solution bifactorielle au sondage des patients, une bifactorielle à celui des collègues de travail et une trifactorielle à celui des collègues médicaux, ce qui constitue au moins 70 % de la variance. Tous les instruments avaient une fiabilité de forte cohérence interne (coefficient α de Cronback > 0,95). Les coefficients de généralisabilité ont été de 0,65 pour les patients, 0,56 pour les collègues de travail et 0,69 pour les pairs.ConclusionIl est faisable ďélaborer des instruments valides et fiables de rétroaction multisources pour les anesthésiologistes.

Likelihood of Change: A Study Assessing Surgeon Use of Multisource Feedback Data

Background: Multisource feedback, using questionnaire-based data from patients, coworkers, and medical colleagues, is designed to provide broad-based information about clinical performance to facilitate change. Purpose: To determine and explain the likelihood that surgeons would implement change following receipt of performance data. Methods: Surgeons were surveyed to determine the likelihood they would make changes based on specific feedback about their clinical practices. Results: One hundred fifty-three surgeons (76.5%) responded to the follow-up survey. There was little correlation between performance ratings provided by self or medical colleagues and the likelihood of change. A linear regression analysis indicated that 19.2% of the variance in likelihood to change could be explained by age, time spent reviewing feedback, the gap between self- and other ratings, and surgical specialty. Conclusion: Surgeons made few changes in practice in response to feedback data. Attention needs to be paid to methods that might increase surgeon use of performance data.

Assessment of Pediatricians by a Regulatory Authority

OBJECTIVE. To determine whether it is possible to develop feasible, valid, and reliable multisource feedback data for pediatricians. METHODS. Surveys with 40, 22, 38, and 37 items were developed for assessment of pediatricians by patients, co-workers, medical colleagues, and themselves, respectively, using 5-point scales with an “unable to assess” category. Items addressed key competencies related to communication skills, professionalism, collegiality, continuing professional development, and collaboration. Each pediatrician was assessed by 25 patients, 8 medical colleagues, and 8 co-workers. Feasibility was assessed with response rates for each instrument. Validity was assessed with rating profiles, the percentage of participants unable to assess the physician for each item, and exploratory factor analyses to determine which items grouped together into scales. Cronbachs α and generalizability coefficient analyses assessed reliability. RESULTS. One hundred pediatricians participated. The mean number of respondents per physician was 23.4 (93.6%) for patients, 7.6 (94.8%) for co-workers, and 7.6 (95.5%) for medical colleagues. The mean ratings ranged from 4 to 5 for each item on each scale. Few items had high percentages of “unable to assess” responses. The factor analyses revealed a 4-factor solution for the patient survey, a 3-factor solution for the co-worker survey, and a 4-factor solution for the medical colleague survey, accounting for at least 64% of the variance. All instruments had high internal consistency. The generalizability coefficients were .85 for patients, .87 for co-workers, and .78 for medical colleagues. CONCLUSION. Surveys can be developed to provide feedback data on key competencies.

Permanent small groups: Group dynamics, learning, and change

Introduction: The concept of “communities of practice,” a special facet of social constructivist learning theory, provides a new template against which we can examine the learning that goes on within permanent small groups of physicians. We interviewed participants and facilitators about the dynamics of these groups, their learning in conjunction with these groups, and the role the facilitator played to see the extent to which they captured the essence of communities of practice. Methods: Semistructured interviews were conducted with physicians known to be participants or facilitators of small groups that met regularly. A constant comparative method was used for data gathering and analysis leading to coded themes, catagories, and subcategories. The coding schemas were tested, the analyses were reviewed, and data were recoded as necessary. To ensure accuracy, interviewees were provided with a preliminary copy of the manuscript to ensure that the interpretation of the data was appropriately handled. Results: Interviews were conducted with 10 facilitators and 22 group members representing 24 different groups of physicians. The groups appeared to function as communities of practice in which the members were supportive of each others learning and respectful of one another, reporting little conflict. Members preferred to agree to disagree rather than pursue a “right” answer or consensus. Most of the discussion focused on scientific information and the way in which their colleagues approached common problems. Practice refinement rather than new directions in patient care appeared to be the goal. The facilitators in these groups played a key role in providing administrative support for the group and often the energy needed to sustain them. Discussion: Small groups that meet regularly provide a supportive network to share knowledge and validate clinical experience. There is some evidence that the groups have the potential to become communities of practice but do not actually achieve that level of sharing. Researth needs to be done to determine how these groups could become more powerful as communities of practice and vehicles for more substantive learning and change.