Len Swanson

A Method for Severely Constrained Item Selection in Adaptive Testing

Previous attempts at incorporating expert test construction practices into computerized adaptive testing paradigms are described. A new method is presented for incorporating a large number of con straints on adaptive item selection. The meth odology emulates the test construction practices of expert test specialists, which is a necessity if com puterized adaptive testing is to compete with con ventional tests. Two examples—one for a verbal measure and the other for a quantitative measure— are provided of the successful use of the proposed method in designing adaptive tests.

A model and heuristic for solving very large item selection problems

A model for solving very large item selection problems is presented. The model builds on previous work in binary programming applied to test con struction. Expert test construction practices are applied to situations in which all specifications for item selection cannot necessarily be met. A heuristic for selecting items that satisfy the constraints in the model also is presented. The heuristic is particu larly useful for situations in which the size of the test construction problem exceeds the limits of current implementations of linear programming algorithms. A variety of test construction problems involving real test specifications and item data from actual test assemblies were investigated using the model and the heuristic.

Optimal Design of Item Banks for Computerized Adaptive Tests

Methods of optimal test assembly have served as the foundation for the development of methods for assembling adaptive tests. A model and a heuristic that facilitate the assembly of adaptive tests have been developed (Stocking & Swanson, 1993; Swanson & Stocking, 1993). Similar methods can be used to assemble item banks for adaptive testing in which optimal design seeks to simultaneously reduce item exposure to enhance item security and to increase exposure to enhance item efficiency. In this study, optimal design methods were applied to the item bank design of adaptive testing.

Application of an automated item selection method to real data

A method of automatically selecting items for inclusion in a test that has constraints on item content and statistical properties was applied to real data. Two tests were assembled by test specialists who assemble such tests on a routine basis. Using the same pool of items and the same constraints, the two tests were reassembled auto matically. Test specialists not involved in the original manual assembly compared the tests constructed manually to the tests constructed automatically. The results indicated that the progress of automated test assembly methods lies in improving item banking systems, classification schemes, and quality control measures, rather than in the development of different algorithms or in the improvement of computer time and cost.

Computerized Adaptive Testing: Principles and Directions.

Abstract Computerized adaptive testing (CAT) is emerging as a useful alternative to conventional paper-and-peneil group-administered testing. In appropriate environments. CAT offers a number of advantages over conventional testing, and should be considered a viable assessment technique. This paper describes the rationale underlying computerized adaptive testing, discusses the psychometric and technical developments which make it practical, and reviews some previous work which has shaped the current state of the art. The potential advantages, constraints and operational requirements, and possible areas of application, are examined.

A cognitive model for structuring an introductory programming curriculum

From its inception the electronic digital computer has been involved in education, although its role has been the subject of some debate. Academically, the computer has been used as a device for conducting or augmenting instruction, as a calculating device adjunctive to courses in engineering and the sciences, and most recently as an object for study in its own right. Increasing attention is now being focused on undergraduate training in computer use, partly as a result of the recommendation of the Presidents Science Advisory Committee that computing education be provided to all college undergraduates. We can expect that the number of students who undertake incidental study of programming as a part of their undergraduate curricula will continue to increase rapidly.

OPTIMAL DESIGN OF ITEM POOLS FOR COMPUTERIZED ADAPTIVE TESTS

Methods of optimal test assembly have served in the past as the foundation for the development of methods of assembling adaptive tests on-the-fly, A particular model and heuristic that facilitate the assembly of adaptive tests of the same complex structure as linear tests have already been developed (Swanson & Stocking, 1993; Stocking & Swanson, 1993). Similar methods can be used in the assembly of item pools designed to support adaptive testing. In the continuous testing environment of adaptive testing, optimal item pool design must simultaneously take into account the competing goals of reducing item exposure to enhance item security and maximizing this exposure to make the greatest possible use of every item. An extension of previous methods of optimal design to the issues of item pool design in the context of adaptive testing is developed in this paper, with examples.

A METHOD FOR SEVERELY CONSTRAINED ITEM SELECTION IN ADAPTIVE TESTING

This paper examines the current state of attempts to incorporate expert test construction practices into a computerized adaptive testing paradigm. A new methodology for incorporating large numbers of constraints on adaptive test item selection is described. This methodology builds on the conventional, that is, linear, test construction practices employed by expert test specialists. The incorporation of such practices is necessary if computerized adaptive testing is to become a serious competitor to conventional paper-and-pencil testing in the future. The successful application of this new methodology to the design of an adaptive test in a real-life setting is demonstrated by an example.