David Trumpower

Validation of the interprofessional collaborative competency attainment survey (ICCAS)

Abstract The purpose of this study was to obtain evidence regarding the validity and reliability of an instrument to measure the self-reported competencies of interprofessional care in interprofessional education programs. Five hundred and eighty-four students and clinicians in Canada and New Zealand who were registered in 15 interprofessional education undergraduate, postgraduate, and continuing professional development programs completed the Interprofessional Collaborative Competency Attainment Survey (ICCAS) using a retrospective pre-test/post-test design. Factor analyses showed the presence of two factors in the pre-program items and one factor in the post-program items. Tests conducted provided evidence in support of the validity and reliability of the ICCAS as a self-assessment instrument for interprofessional collaborative practice. Internal consistency was high for items loading on factor 1 (α = 0.96) and factor 2 (α = 0.94) in the pre-program assessment and for the items in the post-program assessment (α = 0.98). The transition from a two factor solution to a single factor structure suggests interventions influence learners’ understanding of interprofessional care by promoting the recognition of the high degree of interrelation among interprofessional care competencies. Scores on the ICCAS are reliable and predict meaningful outcomes with regard to attitudes toward interprofessional competency attainment.

Assessment for Learning Using Digital Knowledge Maps

As the importance of formative assessment, assessment tools that evaluate the structure of one’s knowledge, and computer-based assessment are becoming increasingly recognized, it is worthwhile to consider if and how digital knowledge maps can be utilized as assessment to support learning. In this chapter, we present evidence that digital knowledge maps do indeed meet the criteria deemed necessary for effective formative assessment—they validly assess higher-order knowledge, identify specific conceptual strengths and weaknesses of students, provide feedback that can be used to effectively improve learning, and can be user friendly for both students and teachers. We conclude the chapter with a discussion of several recently developed formative assessment systems meeting these criteria that rely extensively on the use of digital knowledge maps.

Formative Use of Intuitive Analysis of Variance

Students’ informal inferential reasoning (IIR) is often inconsistent with the normative logic underlying formal statistical methods such as Analysis of Variance (ANOVA), even after instruction. In two experiments reported here, students IIR was assessed using an intuitive ANOVA task at the beginning and end of a statistics course. In both experiments, students were provided feedback regarding the normative logic underlying ANOVA and how their reasoning compared with it. Additionally, students in Experiment 2 were given an assignment in which they analyzed and interpreted other students’ performance on the intuitive ANOVA task. Results indicate that the feedback combined with the assignment (which required active explanation of both normative and non-normative reasoning applied to the task) led to more normative inferential reasoning at the end of the course, whereas providing feedback alone did not. Implications are discussed for using the intuitive ANOVA task as a formative classroom tool to help students improve their conceptual understanding of ANOVA.

The Interprofessional Collaborative Competency Attainment Survey (ICCAS): A replication validation study

ABSTRACT This study replicates a validation of the Interprofessional Collaboration Competency Attainment Survey (ICCAS), a 20-item self-report instrument designed to assess behaviours associated with patient-centred, team-based, collaborative care. We appraised the content validity of the ICCAS for a foundation course in interprofessional collaboration, investigated its internal (factor) structure and concurrent validity, and compared results with those obtained previously by ICCAS authors. Self-assessed competency ratings were obtained from a broad spectrum of pre-licensure, health professions students (n = 785) using a retrospective, pre-/post-design. Moderate to large effect sizes emerged for 16 of 20 items. Largest effects (1.01, 0.94) were for competencies emphasized in the course; the smallest effect (0.35) was for an area not directly taught. Positive correlations were seen between all individual item change scores and a separate item assessing overall change, and item-total correlations were moderate to strong. Exploratory factor analysis was used to understand the interrelationship of ICCAS items. Principal component analysis identified a single factor (Cronbach’s alpha = 0.96) accounting for 85% of the total variance—slightly higher than the 73% reported previously. Findings suggest strong overlaps in the proposed constructs being assessed; use of a total average score is justifiable for assessment and evaluation.