Paul Duggan

Script concordance testing: From theory to practice: AMEE Guide No. 75

The script concordance test (SCT) is used in health professions education to assess a specific facet of clinical reasoning competence: the ability to interpret medical information under conditions of uncertainty. Grounded in established theoretical models of knowledge organization and clinical reasoning, the SCT has three key design features: (1) respondents are faced with ill-defined clinical situations and must choose between several realistic options; (2) the response format reflects the way information is processed in challenging problem-solving situations; and (3) scoring takes into account the variability of responses of experts to clinical situations. SCT scores are meant to reflect how closely respondents’ ability to interpret clinical data compares with that of experienced clinicians in a given knowledge domain. A substantial body of research supports the SCTs construct validity, reliability, and feasibility across a variety of health science disciplines, and across the spectrum of health professions education from pre-clinical training to continuing professional development. In practice, its performance as an assessment tool depends on careful item development and diligent panel selection. This guide, intended as a primer for the uninitiated in SCT, will cover the basic tenets, theoretical underpinnings, and construction principles governing script concordance testing.

Electronic voting to encourage interactive lectures: a randomised trial

BackgroundElectronic Voting Systems have been used for education in a variety of disciplines. Outcomes from these studies have been mixed. Because results from these studies have been mixed, we examined whether an EVS system could enhance a lectures effect on educational outcomes.MethodsA cohort of 127 Year 5 medical students at the University of Adelaide was stratified by gender, residency status and academic record then randomised into 2 groups of 64 and 63 students. Each group received consecutive 40-minute lectures on two clinical topics. One group received the EVS for both topics. The other group received traditional teaching only. Evaluation was undertaken with two, 15-question multiple-choice questionnaires (MCQ) assessing knowledge and problem solving and undertaken as a written paper immediately before and after the lectures and repeated online 8–12 weeks later. Standardised institutional student questionnaires were completed for each lecture and independent observers assessed student behaviour during the lectures. Lecturers opinions were assessed by a questionnaire developed for this study.ResultsTwo-thirds of students randomised to EVS and 59% of students randomised to traditional lectures attended. One-half of the students in the EVS group and 41% in the traditional group completed all questionnaires. There was no difference in MCQ scores between EVS and traditional lectures (p = 0.785). The cervical cancer lectures showed higher student ranking in favour of EVS in all parameters. The breast cancer lectures showed higher ranking in favour of traditional lectures in 5 of 7 parameters (p < 0.001). The observed higher-order lecturer-students interactions were increased in the EVS lecture for one lecturer and reduced for the other. Both lecturers felt that the EVS lectures were difficult to prepare, that they were able to keep to time in the traditional lectures, that the educational value of both lecture styles was similar, and that they were neutral-to-slightly favourably disposed to continue with the EVS technology. The 2 lecturers disagreed regarding the ease of preparation of the traditional lecture, their ability to keep to time in the EVS lecture, and personal satisfaction with the EVS lecture. The lecturers felt that EVS encouraged student participation and helped identify where students were having difficulty.ConclusionIn this setting, EVS technology used in large group lectures did not offer significant advantages over the more traditional lecture format.

Identifying the quality of life effects of urinary incontinence with depression in an Australian population

BackgroundTo explore the additive effect of urinary incontinence, in people with comorbid depression, on health related quality of life.MethodsMales and females, 15 to 95 years (n = 3010, response rate 70.2%) were interviewed face to face in the 1998 Autumn South Australian Health Omnibus Survey.ResultsSelf-reported urinary incontinence was found in 20.3% (n=610), and depression as defined by the PRIME-MD in 15.2% (n=459) of the survey population. Urinary incontinence with comorbid depression was found in 4.3% of the overall population. Univariate analysis showed that respondents with urinary incontinence and comorbid depression were more likely to be aged between 15 and 34 years and never married when compared to those with incontinence only. Multivariate analysis demonstrated that in people with incontinence, the risk of having comorbid depression was increased by an overall health status of Fair or Poor, or the perception that their incontinence was moderately or very serious. Respondents reporting that they experienced incontinence with comorbid depression scored significantly lower than those experiencing incontinence without depression on all dimensions of the SF-36.The interaction of the presence of incontinence and the presence of depression was significantly associated with the dimensions of physical functioning.ConclusionsDepression and incontinence both reduce QOL. When they occur together there appears to be an additive effect which affects both physical and mental health, perhaps by increasing a person’s negative perceptions of their illness. Clinicians should identify and manage comorbid depression when treating patients who have incontinence to improve their overall QOL.

Summative assessment of 5th year medical students’ clinical reasoning by script concordance test: requirements and challenges

BackgroundThe Script Concordance Test (SCT) has not been reported in summative assessment of students across the multiple domains of a medical curriculum. We report the steps used to build a test for summative assessment in a medical curriculum.MethodsA 51 case, 158-question, multidisciplinary paper was constructed to assess clinical reasoning in 5th-year. 10–16 experts in each of 7 discipline-based reference panels answered questions on-line. A multidisciplinary group considered reference panel data and data from a volunteer group of 6th Years, who sat the same test, to determine the passing score for the 5th Years.ResultsThe mean (SD) scores were 63.6 (7.6) and 68.6 (4.8) for the 6th Year (n = 23, alpha = 0.78) and and 5th Year (n = 132, alpha =0.62) groups (p < 0.05), respectively. The passing score was set at 4 SD from the expert mean. Four students failed.ConclusionsThe SCT may be a useful method to assess clinical reasoning in medical students in multidisciplinary summative assessments. Substantial investment in training of faculty and students and in the development of questions is required.

Utilization of preoperative urodynamic investigations by gynecologists who frequently operate for female urinary incontinence

Our objective was to determine the availability and utilization of urodynamic investigations by gynecologists in the preoperative evaluation of women with urinary incontinence. Gynecologists in the UK, USA, Australia, New Zealand and Canada were asked what urodynamic investigations were required for four clinical scenarios. Analysis was restricted to frequent operators. Urodynamic investigations were available to 70% of frequent operators. For uncomplicated stress incontinence, cystometry was utilized by 72% of subspecialists and 44% of generalists (P<0.001) who had access to urodynamic investigations. For stress incontinence and straining to void, uroflowmetry was utilized by 73% of subspecialists and 46% of generalists (P<0.001) who had access to urodynamic investigations. We concluded that many gynecologists who frequently operate for female urinary incontinence do not have access to urodynamic investigations or do not utilize urodynamic investigations, or utilize investigations in a way that may be inadequate for the clinical problem. There are differences in utilization between subspecialists and generalists that are not explained by access. These observations could be explained by poor understanding or a lack of belief in the value of urodynamic investigations.

Antihypertensive drug effects on placental flow velocity waveforms in pregnant women with severe hypertension.

Summary: Doppler studies of the uterine and umbilical arteries were performed in 24 acutely hypertensive pregnant women. Women were divided into 2 groups: Group 1 (diastolic blood pressure (DBP)>110 mm Hg) received placebo (n = 6) or 10 mg oral nifedipine (n = 9); Group 2 (DBP ≥ 110 mm Hg) received 10 mg oral nifedipine (n = 5) or 10 mg intravenous hydralazine (n = 4). Treatment allocations were random within groups and the investigator and patient were blind to the treatments. Baseline flow velocity waveforms (FVW) and posttreatment FVWs at 30, 60 and 120 minutes were recorded. There was no significant difference between baseline and posttreatment FVW indices within or between groups. Placebo was as effective as nifedipine in lowering blood pressure over 2 hours of study. There were no significant adverse effects of treatment.

The modified essay question: Its exit from the exit examination?

Background: Exit examinations in medicine are ‘high stakes’ examinations and as such must satisfy a number of criteria including psychometric robustness, fairness and reliability in the face of legal or other challenges. Aims: We have undertaken a critical review of the exit examination from the University of Adelaide focussing on the written components. This examination consisted of an objective structure clinical examination (OSCE), a multiple choice question (MCQ) paper and a modified essay question (MEQ) paper. Methods: The two written papers were assessed for item writing flaws and taxonomic level using modified Blooms criteria. Curriculum experts independently assessed adequacy of the examination for validity and fidelity. Results: The overall examination had good fidelity and validity. The results of the MEQ and MCQ were strongly and positively correlated and there was a weak negative correlation between these papers and the OSCE. The MEQ had a higher proportion of questions focussed on recall of knowledge and the questions were more structurally flawed compared with the MCQs. The MEQ re-marking process resulted in lower scores than were awarded by the original, discipline-based expert markers. The MEQ paper failed to achieve its primary purpose of assessing higher cognitive skills. Conclusion: The University of Adelaides MBBS programme has since dropped the MEQ paper from its exit examination and is evaluating in its place the Script Concordance test.

Urodynamic diagnoses and quality of life in women presenting for evaluation of urinary incontinence

Background:  Several population‐based and clinical studies report that stress incontinence has less impact on quality of life (QoL) than urge incontinence and overactive bladder.

Blood Pressure Measurement in Pregnancy: A survey of methods used in teaching hospitals in South Australia

Summary: A voluntary, anonymous 10‐point multiple choice questionnaire was used to assess variability in methods used to measure blood pressure in pregnancy in 5 South Australian teaching hospitals. Medical and midwifery staff working in maternity units attached to teaching hospitals in South Australia were asked to complete a survey of their current practice related to the measurement of blood pressure in pregnant women; 213 replies were received from 440 surveyed (48% response rate). There was a lack of standardization of practice for: positioning of the patient, use of the 4th or 5th Korotkoff sound for diastolic blood pressure, cuff selection, rounding of the measurement, selection of the left or right arm, and period of premeasurement resting. Systematic errors averaging 10–15 mmHg can be expected in measurement of blood pressure in pregnancy due to failure of standardization of method. There is a need for standardization of method which is not being met by present methods of staff training in teaching hospitals.

Psychological perspectives in urinary incontinence: a metasynthesis

Introduction Urinary incontinence with co-morbid depression has been found to have a significant effect on quality of life. Examining the associations between the psychosocial factors related to urinary incontinence and mental health may help in improving the care for patients with these conditions. The aim of this research study is to explore the relationship between mental health status and urinary incontinence, focusing on the role of psychosocial factors. Materials and Methods A search of Medline, CINAHL and SCOPUS databases yielded 15 studies on the topic, and 10 studies were found to be in scope. A metasynthesis using Noblit and Hare’s approach of meta-ethnography was undertaken. This involved a number of steps including determining how studies are related and identifying major themes. Results Three psychosocial aspects of urinary incontinence appear to influence mental health status: living with, management of and attitudes about incontinence. Other smaller component themes included control, seeking help and personal beliefs. Discussion Psychosocial factors appear to mediate the relationship between urinary incontinence and mental health status. An increased awareness of the major psychosocial issues that can influence both incontinence and mental health may contribute to a better management of the condition as well as reduce the burden of the condition on individuals. Conclusion Incontinence and psychological wellbeing are intertwined and this adversely affects a number of aspects of life. Enquiring about the mental health status of those with incontinence should include an assessment of psychosocial factors to help reduce the burden of incontinence. To improve the management of these conditions, further research should investigate whether psychological issues precede incontinence or vice versa.