HsingChi von Bergmann

Creating a test blueprint for a progress testing program: A paired-comparisons approach

Abstract Context: Creating a new testing program requires the development of a test blueprint that will determine how the items on each test form are distributed across possible content areas and practice domains. To achieve validity, categories of a blueprint are typically based on the judgments of content experts. How experts judgments are elicited and combined is important to the quality of resulting test blueprints. Methods: Content experts in dentistry participated in a day-long faculty-wide workshop to discuss, refine, and confirm the categories and their relative weights. After reaching agreement on categories and their definitions, experts judged the relative importance between category pairs, registering their judgments anonymously using iClicker, an audience response system. Judgments were combined in two ways: a simple calculation that could be performed during the workshop and a multidimensional scaling of the judgments performed later. Results: Content experts were able to produce a set of relative weights using this approach. The multidimensional scaling yielded a three-dimensional model with the potential to provide deeper insights into the basis of the experts’ judgments. Conclusion: The approach developed and demonstrated in this study can be applied across academic disciplines to elicit and combine content experts judgments for the development of test blueprints.

When I say … validity argument

Imagine that you are part of a medical or dental programme that is developing a progress testing programme. The test will be administered to students at least once a year while they are in the programme to help them understand their progress toward mastery of the knowledge needed at graduation. The test is intended to support the students’ learning and to inform the programme’s curricular and instructional decisions. Developing the test requires you and your colleagues to make countless decisions that will affect the validity of the use of the test results.

Emerging Scholarship of Teaching at Canadian Dental Schools

Traditionally, dental schools have been devoted to the mission of preparing future oral health care professionals, with teaching as their main focus. However, as tuition and competition for students have increased, the demand for quality education has grown. Furthermore, when situated within a