Contextualizing Work-Based Assessments of Faculty and Residents: Is There a Relationship Between the Clinical Practice Environment and Assessments of Learners and Teachers?

Abstract

Purpose Competence is bound to context, yet seldom is environment explicitly considered in work-based assessments. This study explored faculty and residents’ perspectives of the environment during internal medicine clinical teaching unit (CTU) rotations, the extent that each group accounts for environmental factors in assessments, and relationships between environmental factors and assessments. Method From July 2014 to June 2015, 212 residents and 54 faculty across 5 teaching hospitals at University of Toronto rated their CTU environment using a novel Practice Environment Rating Scale (PERS) matched by block and hospital. Faculty-PERS data were paired to In-Training Evaluation Reports (ITERs) of residents supervised during each block, and Resident-PERS data to Resident Assessment of Teaching Effectiveness (RATE) scores of the same faculty. Differences between perceptions and assessments were tested using repeated-measures MANOVAs, ANOVAs, and correlations. Results One-hundred sixty-four residents completed the PERS; residents rated the CTU environment more positively than faculty (3.91/5 vs. 3.29, P < .001). Residents were less likely to report considering environmental factors when assessing faculty (2.70/5) compared with faculty assessing residents (3.40, P < .0001), d = 1.2. Whereas Faculty-PERS ratings did not correlate with ITER scores, Resident-PERS ratings had weak to moderate correlations with RATE scores (overall r = 0.27, P = .001). Conclusions Residents’ perceptions of the environment had small but significant correlations with assessments of faculty. Faculty’s perceptions did not affect assessments of residents, potentially because they reported accounting for environmental factors. Understanding the interplay between environment and assessment is essential to developing valid competency judgments.