William R. Koch

Computerized adaptive testing with polytomous items

Polytomous item response theory models and the research that has been conducted to investigate a variety of possible operational procedures for polytomous model-based computerized adaptive testing (CAT) are reviewed. Studies that compared polytomous CAT systems based on competing item response theory models that are appropriate for the same measurement objective, as well as applications of polytomous CAT in marketing and educational psychology, also are reviewed. Directions for future research using polytomous model-based CAT are suggested.

Operational Characteristics of Adaptive Testing Procedures Using the Graded Response Model

The purpose of the present research was to develop general guidelines to assist practitioners in setting up operational computerized adaptive testing (CAT) sys tems based on the graded response model. Simulated data were used to investigate the effects of systematic manipulation of various aspects of the CAT procedures for the model. The effects of three major variables were examined: item pool size, the stepsize used along the trait continuum until maximum likelihood estima tion could be calculated, and the stopping rule em ployed. The findings suggest three guidelines for graded response CAT procedures: (1) item pools with as few as 30 items may be adequate for CAT; (2) the variable-stepsize method is more useful than the fixed- stepsize methods; and (3) the minimum-standard-error stopping rule will yield fewer cases of nonconverg ence, administer fewer items, and produce higher cor relations of CAT θ estimates with full-scale estimates and the known θs than the minimum-information stop ping rule. The implications of these findings for psy chological assessment are discussed. Index terms: computerized adaptive testing, graded response model, item response theory, polychotomous scoring.

Effects of Variations in Item Step Values on Item and Test Information in the Partial Credit Model

Simulated data were used to investigate systemati cally the impact of various orderings of step difficul ties on the distribution of item information for the par tial credit model. It was found that the distribution of information for an item was a function of (1) the range of the step difficulty values, (2) the number of step difficulties that were out of sequential order, and (3) the distance between the step values that were out of order. Also, by using relative efficiency compari sons, the relationship between the step estimates and the distribution of item information was used to dem onstrate the effects of various test revisions (through the addition and/or deletion of items with specific step characteristics) on the resulting tests precision of measurement. The usefulness of item and test infor mation functions for specific measurement applications of the partial credit model is also discussed.

Computerized Adaptive Testing Using the Partial Credit Model: Effects of Item Pool Characteristics and Different Stopping Rules

Simulated datasets were used to research the effects of the systematic variation of three major variables on the performance of computerized adaptive testing (CAT) procedures for the partial credit model. The three variables studied were the stopping rule for terminating the CATs, item pool size, and the distribution of the difficulty of the items in the pool. Results indicated that the standard error stopping rule performed better across the variety of CAT conditions than the minimum information stopping rule. In addition it was found that item pools that consisted of as few as 30 items were adequate for CAT provided that the item pool was of medium difficulty. The implications of these findings for implementing CAT systems based on the partial credit model are discussed.

Discerning the Epidemiology of Drug Use among a Sample of College Students.

Although estimates concerning the prevalence of various forms of substance abuse vary considerably, it is critical to note that even the more conservative estimates indicate a major public health problem. Furthermore, increased drug experimentation has been frequently associated with the college experience. Utilizing a slightly modified version of the National Institute on Drug Abuse (NIDA) Monitoring the Future Survey, the present study examined the epidemiology of self-reported drug taking behaviors (e.g., alcohol, marijuana, tranquilizers, stimulants, psychedelics, etc.) among a sample (n = 683) of students attending a major public university. Data analyses focused on the specific settings and social groupings in which drug use occurred as well as on the frequency and type(s) of drug taking behaviors. Based on the results yielded from a disjoint cluster analysis, four distinct groups of drug users were identified. These four groups then served as categories of the dependent variable in a discriminant function analysis. A general linear models procedure was employed to identify the more efficacious discriminating (i.e., independent) variables.

Personality/behavioral characteristics in children: Differential effects of putative anterior versus posterior cerebral asymmetry

The current study was carried out to examine the possible relationships among personality/behavioral characteristics and anterior/posterior (A/P) functional cerebral asymmetry in children referred for learning problems. Two hundred nineteen children between 7 and 12 years of age were administered a battery of neuropsychological measures, and their parents completed the Child Behavior Checklist (CBC) and the Personality Inventory for Children-Revised (PIC). Anterior and posterior composite scores were obtained for each subject using scores on the neuropsychological measures. Out of this subject pool, 33 children had sufficient anterior/posterior (A/P) score differences (i.e., greater than one standard deviation difference) to permit their categorization into either an A (n = 19) or P (n = 14) group. The MANOVA results showed that the A group scored significantly higher than the P group on the CBC scales of social withdrawal, aggressiveness, hyperactivity and externalizing, while the P group scored higher, though not significantly, on the (PIC) scale of anxiety. Thus, it appeared that, by using neuropsychological measures along the A/P dimension to classify children with learning problems, significant differences could also be identified on personality/ behavioral variables for some children. In addition, in the current study, children with A type functional cerebral asymmetry exhibited a relatively greater number of behavioral problems.

Item and Scale Information Functions for the Successive Intervals Rasch Model.

Simulated data were used to investigate systematically the impact of various characteristics of the threshold values (number, symmetry, and distance between adjacent threshold values) and of the delta values on the distribution of item information in the successive intervals Rasch model. The results revealed that the shift in the peak of the information function away from the scale value of an item depended on the degree of asymmetry of the threshold values and the magnitude of the delta value of the item. The implications of the findings for computerized adaptive attitude measurement are discussed.

An Investigation of Procedures for Computerized Adaptive Testing Using the Successive Intervals Rasch Model

The major objective of the present study was to evaluate basic procedures for performing computerized adaptive testing (CAT) based on the successive intervals (SI) Rasch model. The SI model is designed for measuring attitudes based on Thurstones successive intervals concept. The SI model was applied to both simulated and real attitude data sets for which the item response data were polytomous. Using simulated data, such variables as item pool size, method of item selection, and sizes of item dispersion parameters were studied. The results indicated that the CAT procedures performed very well using item pools of either 30 or 61 items; there was excellent recovery of the known attitude trait levels of the simulees, with only about 17 items administered on average during the CATs. For the real data set, the CAT procedures also performed very well in terms of number of items administered and accuracy of trait estimation.

Practical Strategies for Developing Stable Models to Predict Cocaine Use among College Students

The purpose of the study was to identify a stable model of psychosocial variables related to cocaine use among college students. Development and validation of this model could contribute to a useful cocaine abuse risk profile for college students. Of the 683 students who participated in the study, 179 reported using cocaine at least once in their life on an anonymous survey. The relationship between an index of cocaine use (coef. alpha = .79) and eight psychosocial variables was examined using a multiple regression procedure. The results indicated that two variables, perceived academic ability and interest in school, were significantly associated with scores on the cocaine index. Cross-validation procedures, including Tukeys Jackknife method, were conducted to ascertain the stability of the multiple regression model based on changes in the R-Squared value. The implications for future research and prevention programming efforts are discussed.

Book Review : The Basics of Item Response Theory: Frank B. Baker Portsmouth NH: Heinemann Educational Books, 1985, 131 pp., approx.

psychological measurement to the basic ideas and features of the theory. The book does not presume that the reader possesses a sophisticated background in mathematics or probability, but neither does it neglect the quantitative aspects of IRT. Persons new to the area of IRT would find it relatively easy to learn the basic concepts of the theory by working their way through the book and its accompanying exercises. An important and valuable innovation, if the reader has access to an Apple II + or IIe microcomputer, is the available floppy diskette. The diskette offers several advantages. First, it overcomes a serious problem faced by teachers and textbook authors in the area of IRT, namely that substantial computational effort may be necessary to demonstrate even the simplest principles of the theory. Second, the diskette provides a series of effective exercises that complement and illustrate the material in the text. Third, the use of the diskette is menu-driven with a few easy commands or keystrokes. Finally, the user has the option to input desired values of interest in the exercises or to have them supplied by the computer program. The obvious limitation of the optional diskette is that a version is not available for persons with IBM-PC systems. Another limitation is that there is no simple capability to print interesting graphics or tables that appear on the video display. The advantages of the diskette far outweigh the disadvantages, however. The diskette provides the user with an invaluable tool to explore and apply the concepts of IRT in a practical manner. For example, item characteristic curves can be plotted for each of the three popular dichotomous IRT models, as can test characteristic curves and information functions. Also, item parameters and person parameters may be estimated in small-scale fashion, and the principles of test calibration, test equating, and test construction can be simulated in the IRT context. The diskette is such an integral part of the book and such a valuable learning device that its purchase should be considered essential rather than optional. Each of the eight chapters in the book begins with a brief treatment of a single topic, followed by