David E. Steward

Direct, standardized assessment of clinical competence

Summary. Doctor ratings of clerkship performance are often discounted as not accurately reflecting clinical competence. Such ratings are influenced by the following uncontrolled variables: case difficulty; differing rater focus and standards; lack of agreement on what constitutes acceptable performance; and collective patient care responsibility masks individual contributions. Standardized direct measures of clinical competence were developed to control these factors and allow direct comparisons of student performance. Students saw 18 patients representing frequently occurring and important patient problems. Student actions and decisions were recorded and subsequent responses to questions revealed knowledge of pathophysiology, basis for actions, use and interpretation of laboratory investigations, and management. Actions and responses were graded using a pre‐set key. The examination covered 73% of designated clinical competencies. Examination scores corresponded with independent measures of clinical competence. Reliability studies indicated that new cases can be substituted in subsequent years with confidence that scores will maintain similar meaning. Costs are £6.95 per student per case, which is modest considering the quality and quantity of information acquired. Methods described are practical for evaluation of clerks and residents and for licensing and specialty certification examinations.

A survey of medical school teachers to identify basic biomedical concepts medical students should understand

Insights from the cognitive sciences indicate a continuing need for physicians to understand conceptual knowledge from the basic sciences, despite recent concerns regarding the increasing amount of information in medicine and the growing emphasis on performance skills. A 1987 survey of selected basic science and clinical teachers in North American medical schools was undertaken to identify basic biomedical concepts that are important in the practice of medicine and to specify how difficult these are for students to learn, apply, or both. Responses from faculty (nominated by their deans to answer the survey) from 82% of the medical schools indicated considerable agreement between the basic science teachers and clinical teachers on the relative importance of a set of biomedical concepts, and showed relatively minor levels of disagreement on how difficult these concepts are. The judgments of these teachers could prove extremely useful in (1) determining concepts that–because of their importance–should receive special attention in curriculum efforts, and (2) determining concepts that–because of their difficulty–need “special handling.”

Three studies of the effect of multiple standardized patients on intercase reliability of five standardized‐patient examinations

Standardized‐patient (SP) cases have been used in a postclerkship examination to assess clinical competence of five classes of senior medical students at Southern Illinois University School of Medicine. Two or more (multiple) SPs have been used to simulate at least half the cases in each examination administered to date. The present studies were conducted to determine the effect of multiple SPs on intercase reliability. In Study 1, for each class, generalizability analyses were performed to determine the intercase reliabilities for cases using a single SP and for cases using multiple SPs. Study 2 was conducted to compare the intercase reliabilities of those few cases that were used in two different classes, simulated by single SPs in one class and by multiple SPs in the other. Study 3 was undertaken to determine directly by statistical analysis of the multiple‐SP cases only, the contributions of multiple SPs to measurement error and, hence, reliability. The results of all three studies showed little or no...

Effect of using multiple standardized patients to rate interpersonal and communication skills on intercase reliability

Performance‐based examinations that use standardized‐patient (SP) cases have been used to assess the clinical competence of senior medical students in eight classes (1986 to 1993) at Southern Illinois University School of Medicine. Two or more (multiple) SPs have been used to simulate about half the cases in each examination. This study was conducted to assess the effect of multiple SPs on the intercase reliability of SP ratings of interpersonal and communication skills. In general, results showed little effect of multiple SPs on reliability of ratings of interpersonal and communication skills. For example, for ratings on overall patient satisfaction, the generalizability coefficient computed without including measurement error due to multiple SPs was .78, whereas the coefficient computed with measurement error due to multiple SPs included was. 73—suggesting that use of multiple SP raters on the same case reduced reliability by about .05. Similar results were obtained for ratings on the other dimensions. ...

Effect of repeated simulations by standardized patients on intercase reliability

A performance‐based examination consisting of standardized‐patient (SP) cases was given to all senior medical students in five classes at Southern Illinois University School of Medicine. Each class taking the examination consisted of about 70 students, so that each SP case had to be simulated a total of 70 times during the examination. Having 70 simulations of the same case by a given SP naturally raises questions about the ability of the SP to portray the case and to complete checklists consistently throughout the entire examination period. One concern is that, with repeated simulations, SPs might grow tired and become careless in portraying their respective cases and completing checklists, so that measurement error would increase throughout the examination. On the other hand, the practice gained with repeated simulations could increase the consistency of the simulations and checklist completions, so that measurement error would decrease throughout the examination. The present analysis, then, was underta...

PERSPECTIVE: A Proposal to Enhance the Disclosure of Potential Conflict of Interest for Continuing Medical Education Events

Background: Commercial support for continuing medical education (CME) raises questions about potential conflict of interest. Current disclosure methods provide limited, general information about the nature and degree of support, such that potential conflict of interest cannot be studied by researchers or understood by physicians. Summary: This article proposes a semiquantitative method for disclosure of commercial support for CME. The method includes a statement of the degree of support a program receives from all sources and an estimate of how revenue for the program compares to expenses. The new method would standardize the semiquantitative reporting of commercial support for CME programs. Conclusion: The proposed method for disclosing conflict of interest for CME events should facilitate the study and understanding of how commercial support affects these programs.

Implementing a Nutrition and Physical Activity Curriculum in Head Start through an Academic-community Partnership.

BACKGROUND Schools may be an effective avenue for interventions that prevent childhood obesity. I am Moving I am Learning/Choosy Kids© (IMIL/CK) is a curriculum recommended by Head Start (HS) for education in nutrition, physical activity, and healthy lifestyle habits. METHODS We formed an academic-community partnership (ACP), the Springfield Collaborative for Active Child Health, to promote prevention of childhood obesity, in part, to implement the IMIL/CK curriculum in local HS sites. The ACP included a medical school, HS program, public school district, and state health department. RESULTS Community-based participatory research principles helped identify and organize important implementation activities: community engagement, curriculum support, professional teacher training, and evaluation. IMIL/CK was piloted in 1 school then implemented in all local HS sites. All sites were engaged in IMIL/CK professional teacher training, classroom curriculum delivery, and child physical activity assessments. Local HS policy changed to include IMIL/CK in lesson plans and additional avenues of collaboration were initiated. Furthermore, improvements in physical activity and/or maintenance or improvement of healthy weight prevalence was seen in 4 of the 5 years evaluated. CONCLUSIONS An ACP is an effective vehicle to implement and evaluate childhood obesity prevention programming in HS sites.

Gender-specific relationships between socioeconomic disadvantage and obesity in elementary school students

OBJECTIVE To assess the gender-specific effect of socioeconomic disadvantage on obesity in elementary school students. METHODS We evaluated body mass index (BMI) data from 2,648 first- and fourth-grade students (1,377 male and 1,271 female students) in eight elementary schools in Springfield, Illinois, between 2012 and 2014. Other factors considered in analysis were grade level, year of data collection, school, race/ethnicity, gender, and socioeconomic disadvantage (SD). Students were considered SD if they were eligible for free/reduced price lunch, a school-based poverty measure. We performed Fishers exact test or chi-square analysis to assess differences in gender and obesity prevalence by the other factors and gender-stratified logistic regression analysis to determine if SD contributed to increased odds of obesity. RESULTS A higher proportion of SD female students (20.8%) were obese compared to their non-SD peers (15.2%) (p=0.01). Unadjusted and adjusted logistic regression analysis indicated no difference in obesity in SD and non-SD male students. However, in both unadjusted and adjusted analyses, SD female students had higher odds of obesity than their peers. Even after controlling for grade level, school, year of data collection, and race/ethnicity, SD female students had 49% higher odds of obesity than their non-SD classmates (odds ratio:1.49; 95% confidence interval: 1.09-2.04). CONCLUSIONS Obesity was elevated in SD female students, even after controlling for factors such as race/ethnicity, but such an association was not seen in male students. Further study is warranted to determine the cause of this disparity, and interventions should be developed to target SD female students.

The Role of Community Health Needs Assessments in Medicalizing Poverty

The Patient Protection and Affordable Care Act (ACA), passed in 2010, is considered by many to be the most significant healthcare overhaul since the 1960s, but part of its promise — improvement of population health through requirements for non-profit hospitals to provide “community benefit” — has not been met. This paper examines the history of community benefit legislation, how community benefit dollars are allocated, and innovative practices by a few hospitals and communities that are addressing primarily non-medical factors that influence health such as social disadvantage, attitudes, beliefs, risk exposure, and social inequalities.

The Mini Report: a Practical Tool to Address Lung Cancer Disparities in Rural Communities.

Community-based participatory research (CBPR) is an effective way to address cancer disparities in medically underserved populations. Our research demonstrates how CBPR principles were used to develop lung cancer and risk factor mini reports for a network of community coalitions in the Illinois Delta Region, a predominately rural region with high lung cancer disparities in southern Illinois. An academic-community partnership, including a community-based medical school, state public health department, and a healthcare system, used CBPR principles to translate epidemiological, behavioral, and demographic data into understandable, comprehensive, yet concise mini reports for each coalition. A cyclical and iterative process was used to draft, revise, and optimize these mini reports to raise awareness about lung cancer disparities in the community and to provide information to help guide the development of interventions that address these disparities. The use of CBPR principles was a successful way to create mini reports about local lung cancer disparities and risk factors that were usable in individual communities. Local coalitions used the mini reports to educate community members at local meetings, to guide strategic planning, and to disseminate information through their respective websites. Additionally, the process of creating these reports built trust among academic-community partners and provided additional avenues of engagement, such as the involvement of an academic partner in the strategic planning process of a local coalition. Using CBPR processes is an effective way to translate epidemiological data into a community-friendly format to address cancer disparities.