Vanessa Burch

Workplace-based assessment as an educational tool: AMEE Guide No. 31

Background: There has been concern that trainees are seldom observed, assessed, and given feedback during their workplace-based education. This has led to an increasing interest in a variety of formative assessment methods that require observation and offer the opportunity for feedback. Aims: To review some of the literature on the efficacy and prevalence of formative feedback, describe the common formative assessment methods, characterize the nature of feedback, examine the effect of faculty development on its quality, and summarize the challenges still faced. Results: The research literature on formative assessment and feedback suggests that it is a powerful means for changing the behaviour of trainees. Several methods for assessing it have been developed and there is preliminary evidence of their reliability and validity. A variety of factors enhance the efficacy of workplace-based assessment including the provision of feedback that is consistent with the needs of the learner and focused on important aspects of the performance. Faculty plays a critical role and successful implementation requires that they receive training. Conclusions: There is a need for formative assessment which offers trainees the opportunity for feedback. Several good methods exist and feedback has been shown to have a major influence on learning. The critical role of faculty is highlighted, as is the need for strategies to enhance their participation and training.

CRITERIA FOR GOOD ASSESSMENT: CONSENSUS STATEMENT AND RECOMMENDATIONS FROM THE OTTAWA 2010 CONFERENCE

In this article, we outline criteria for good assessment that include: (1) validity or coherence, (2) reproducibility or consistency, (3) equivalence, (4) feasibility, (5) educational effect, (6) catalytic effect, and (7) acceptability. Many of the criteria have been described before and we continue to support their importance here. However, we place particular emphasis on the catalytic effect of the assessment, which is whether the assessment provides results and feedback in a fashion that creates, enhances, and supports education. These criteria do not apply equally well to all situations. Consequently, we discuss how the purpose of the test (summative versus formative) and the perspectives of stakeholders (examinees, patients, teachers-educational institutions, healthcare system, and regulators) influence the importance of the criteria. Finally, we offer a series of practice points as well as next steps that should be taken with the criteria. Specifically, we recommend that the criteria be expanded or modified to take account of: (1) the perspectives of patients and the public, (2) the intimate relationship between assessment, feedback, and continued learning, (3) systems of assessment, and (4) accreditation systems.

Mortality in patients treated for tuberculous pericarditis in sub-Saharan Africa

OBJECTIVE To determine the mortality rate and its predictors in patients with a presumptive diagnosis of tuberculous pericarditis in sub-Saharan Africa. DESIGN Between 1 March 2004 and 31 October 2004, we enrolled 185 consecutive patients with presumed tuberculous pericarditis from 15 referral hospitals in Cameroon, Nigeria and South Africa, and observed them during the 6-month course of antituberculosis treatment for the major outcome of mortality. This was an observational study, with the diagnosis and management of each patient left at the discretion of the attending physician. Using Cox regression, we have assessed the effect of clinical and therapeutic characteristics (recorded at baseline) on mortality during follow-up. RESULTS We obtained the vital status of 174 (94%) patients (median age 33; range 14 - 87 years). The overall mortality rate was 26%. Mortality was higher in patients who had clinical features of HIV infection than in those who did not (40% v. 17%, p=0.001). Independent predictors of death during followup were: (i) a proven non-tuberculosis final diagnosis (hazard ratio (HR) 5.35, 95% confidence interval (CI) 1.76 - 16.25), (ii) the presence of clinical signs of HIV infection (HR 2.28, CI 1.14 - 4.56), (iii) coexistent pulmonary tuberculosis (HR 2.33, CI 1.20 - 4.54), and (iv) older age (HR 1.02, CI 1.01 - 1.05). There was also a trend towards an increase in death rate in patients with haemodynamic instability (HR 1.80, CI 0.90 - 3.58) and a decrease in those who underwent pericardiocentesis (HR 0.34, CI 0.10 - 1.19). CONCLUSION A presumptive diagnosis of tuberculous pericarditis is associated with a high mortality in sub-Saharan Africa. Attention to rapid aetiological diagnosis of pericardial effusion and treatment of concomitant HIV infection may reduce the high mortality associated with the disease.

Clinical characteristics and initial management of patients with tuberculous pericarditis in the HIV era: the Investigation of the Management of Pericarditis in Africa (IMPI Africa) registry

BackgroundThe incidence of tuberculous pericarditis has increased in Africa as a result of the human immunodeficiency virus (HIV) epidemic. However, the effect of HIV co-infection on clinical features and prognosis in tuberculous pericarditis is not well characterised. We have used baseline data of the Investigation of the Management of Pericarditis in Africa (IMPI Africa) registry to assess the impact of HIV co-infection on clinical presentation, diagnostic evaluation, and treatment of patients with suspected tuberculous pericarditis in sub-Saharan Africa.MethodsConsecutive adult patients in 15 hospitals in three countries in sub-Saharan Africa were recruited on commencement of treatment for tuberculous pericarditis, following informed consent. We recorded demographic, clinical, diagnostic and therapeutic information at baseline, and have used the chi-square test and analysis of variance to assess probabilities of significant differences (in these variables) between groups defined by HIV status.ResultsA total of 185 patients were enrolled from 01 March 2004 to 31 October 2004, 147 (79.5%) of whom had effusive, 28 (15.1%) effusive-constrictive, and 10 (5.4%) constrictive or acute dry pericarditis. Seventy-four (40%) had clinical features of HIV infection. Patients with clinical HIV disease were more likely to present with dyspnoea (odds ratio [OR] 3.2, 95% confidence interval [CI] 1.4 to 7.4, P = 0.005) and electrocardiographic features of myopericarditis (OR 2.8, 95% CI 1.1 to 6.9, P = 0.03). In addition to electrocardiographic features of myopericarditis, a positive HIV serological status was associated with greater cardiomegaly (OR 3.89, 95% CI 1.34 to 11.32, P = 0.01) and haemodynamic instability (OR 9.68, 95% CI 2.09 to 44.80, P = 0.0008). However, stage of pericardial disease at diagnosis and use of diagnostic tests were not related to clinical HIV status. Similar results were obtained for serological HIV status. Most patients were treated on clinical grounds, with microbiological evidence of tuberculosis obtained in only 13 (7.0%) patients. Adjunctive corticosteroids were used in 109 (58.9%) patients, with patients having clinical HIV disease less likely to be put on them (OR 0.37, 95% CI 0.20 to 0.68). Seven patients were on antiretroviral drugs.ConclusionPatients with suspected tuberculous pericarditis and HIV infection in Africa have greater evidence of myopericarditis, dyspnoea, and haemodynamic instability. These findings, if confirmed in other studies, may suggest more intensive management of the cardiac disease is warranted in patients with HIV-associated pericardial disease.

A structured assessment of newly qualified medical graduates

Introduction  While there is extensive published experience with the assessment of procedural skills in undergraduate students, this is limited in newly qualified medical graduates at the time of entry to the pre‐registration (internship) year. The few studies that have been published suggest that these skills are frequently deficient when objectively tested. We therefore chose to assess the competence of a group of South African medical graduates on entry to their pre‐registration year.

Does multidisciplinary stroke care improve outcome in a secondary-level hospital in South Africa?

Background and Purpose The improved outcome (survival and function) of stroke patients admitted to multidisciplinary stroke units (SU) in developed countries has not been replicated in developing countries in sub-Saharan Africa. This study documents the outcome of patients admitted to the first multidisciplinary SU opened at a secondary hospital in Cape Town, South Africa. Methods Patient outcomes including in-hospital mortality, resource utilization (length of hospital stay, CT brain scans performed, and tertiary hospital referral), and access to inpatient rehabilitation were recorded for all patients admitted to the hospital for 3 months before initiating multidisciplinary stroke care and for 3 months after implementing multidisciplinary stroke care. Results One hundred and ninety-five patients were studied; 101 of these were managed in the SU. Inpatient mortality decreased from 33% to 16% after initiating multidisciplinary stroke care (P = 0·005). The length of hospital stay increased from a mean (SD, 95% CI) of 5·1 (6·5, 3·8–6·4) days to 6·8 (4·5, 5·9–7·6) days (P = 0·01). Referral to inpatient rehabilitation increased from 5% to 19% (P = 0·04) for those who survived to discharge. The number of CT brain scans performed and the number of referrals to the tertiary academic hospital did not increase significantly. Conclusions Multidisciplinary stroke care was successfully implemented in a resource-constrained secondary-level hospital in South Africa and despite the limitations of the study, the significant reduction in inpatient mortality and increase in referral for inpatient rehabilitation would suggest an improvement in stroke care. Resource utilization in terms of length of hospital stay increased by a mean of 2 days but the number of CT brain scans performed and referral to a tertiary hospital did not increase significantly.

Use of a structured interview to assess portfolio-based learning.

Context  Portfolio‐based learning is a popular educational tool usually examined by document review which is sometimes accompanied by an oral examination. This labour‐intensive assessment method prohibits its use in the resource‐constrained settings typical of developing countries.

Fit for purpose? The appropriate education of health professionals in South Africa

Sixty years ago Sidney and Emily Kark had a vision to make health care accessible to all South Africans, particularly those in rural and under-served areas. They established a comprehensive primary health care model that was replicated in over 40 rural communities before the apartheid government abolished their plan. In the light of these historic initiatives, what have health sciences training institutions achieved in the past 50 years to address the health care priorities of South Africans who are most in need?

Abdominal and pericardial ultrasound in suspected extrapulmonary or disseminated tuberculosis

OBJECTIVE Tuberculosis (TB) in patients with or without advanced HIV infection may present as smear-negative, extrapulmonary and/or disseminated forms. We studied the role of pericardial and abdominal ultrasound examinations in the determination of extrapulmonary or disseminated TB. METHODS A prospective descriptive and analytic cross-sectional study design was used to determine the ultrasound findings of value in patients with subsequently proven TB. Ultrasound examinations were performed on 300 patients admitted to G F Jooste Hospital with suspected extrapulmonary or disseminated TB. OUTCOME MEASURES The presence of hepatomegaly, splenomegaly, lymphadenopathy (location, size and appearance), ascites, pleural effusions, pericardial effusions and/or splenic micro-abscesses was noted. Clinical findings, microbiological and serological data were also recorded, correlated and analysed. RESULTS Complete data sets were available for 267 patients; 91.0% were HIV positive, and 70.0% had World Health Organization clinical stage 4 disease. Active TB (determined by smear or culture) was present in 170 cases (63.7%). Ultrasonically visible abdominal lymphadenopathy over 1 cm in minimum diameter correlated with active TB in 55.3% of cases (odds ratio (OR) 2.6, 95% confidence interval (CI) 1.5 - 4.6, p = 0.0002). Ultrasonographically detected pericardial effusions (OR 2.8, 95% CI 1.6 - 5.0, p < 0.0001), ascites (OR 2.2, 95% CI 1.2 - 4.2, p = 0.005) and splenic lesions (OR 1.9, 95% CI 1.0 - 3.5, p = 0.024) also predicted active TB. CONCLUSION Pericardial and abdominal ultrasound examinations are valuable supplementary investigations in the diagnosis of suspected extrapulmonary or disseminated TB.

Developing an educational research framework for evaluating rural training of health professionals: A case for innovation

Background: World-wide, rural clinical training of undergraduate medical students is looking to transform learning experiences, calling for the adoption of innovative approaches that create spaces for curriculum renewal and new ways of thinking. In order for these teaching models to gain acceptance and credibility among the relevant academic communities, it is critical that they be studied and evaluated. Aim: This article describes an innovative rural education intervention and a concomitant, intentional process that was adopted to establish a research framework within which the intervention will be evaluated. Methods: Key role-players participated in a one-day workshop aimed at developing the framework. A collaborative, structured process that moved through three phases of deliberation and reflection was followed. Results: The documentation and raw data generated during the workshop was used to generate the framework that will serve as a blueprint for ensuring the study and evaluation of the educational innovation. Conclusion: Establishing an educational research framework, by adopting a consultative and collaborative process, provides a vehicle for encouraging a culture of critical accountability that seeks to discern evidence of good practice in the training of health care workers in a rural context.