Hanneke Geerlings

Analysis of longitudinal randomized clinical trials using item response models

Patient-relevant outcomes, such as impairments, disability and health-related quality of life, are becoming increasingly popular as outcome measures in clinical research. These outcomes are generally assessed using questionnaires. In a longitudinal randomized clinical trial where the outcome is measured by a questionnaire or some other instrument consisting of a set of discretely scored items, treatment effects can be analyzed using item response theory. The problem addressed is how to take the estimation error in the estimates of the latent outcome variables into account in the estimation of the treatment effects. Three approaches are compared: plausible value imputation (PVI), concurrent marginal maximum likelihood (MML) estimation and a limited information two-step marginal maximum likelihood method. The results show that the power of the former two methods to detect small and moderate effect sizes is considerably larger than the power of the latter approach. An additional advantage of the PVI method as compared to MML is that the treatment effects can be estimated with standard software. An example using data from a longitudinal randomized clinical trial illustrates the use of the methods in a practical setting. It is shown that even when responses on different sets of items for different groups of patients are used for the data analysis, the power to detect the experimental effects is comparable to the power obtained when responses to all items for all patients are used in the analysis. This creates considerable flexibility in the design and use of measures in experiments.

Estimation of the Parameters in an Item-Cloning Model for Adaptive Testing

Item response theory (IRT) models with random person parameters have become a common choice among practitioners in the field of educational and psychological measurement. Though initially the choice for such models was motivated by an attempt to get rid of the statistical problems inherent in the incidental nature of the person parameters (Bock & Lieberman, 1970), the insight soon emerged that such models more adequately represent cases where the focus is not on the measurement of individual persons but on the estimation of characteristics of populations. Early examples of models with random person parameters in the literature are those proposed by Andersen and Madsen (1977) and Sanathanan and Blumenthal (1978), who were interested in estimates of the mean and variance in a population of persons, and by Mislevy (1991), who provided tools for inference from a response model with a regression structure on the person parameters introduced to account for sampling persons differing background variables.

Optimal test design with rule-based item generation

Optimal test-design methods are applied to rule-based item generation. Three different cases of automated test design are presented: (a) test assembly from a pool of pregenerated, calibrated items; (b) test generation on the fly from a pool of calibrated item families; and (c) test generation on the fly directly from calibrated features defining the item families. The last two cases do not assume any item calibration under a regular response theory model; instead, entire item families or critical features of them are assumed to be calibrated using a hierarchical response model developed for rule-based item generation. The test-design models maximize an expected version of the Fisher information in the test and control critical attributes of the test forms through explicit constraints. Results from a study with simulated response data highlight both the effects of within-family item-parameter variability and the severity of the constraint sets in the test-design models on their optimal solutions.

Testing the difficulty theory of the SON-R 512 –17, a non-verbal test of intelligence

Fischers (1973) linear logistic test model can be used to test hypotheses regarding the effect of covariates on item difficulty and to predict the difficulty of newly constructed test items. However, its assumptions of equal discriminatory power across items and a perfect prediction of item difficulty are never absolutely met. The amount of misfit in an application of a Bayesian version of the model to two subtests of the SON-R 5(1/2)-17 is investigated by means of item fit statistics in the framework of posterior predictive checks and by means of a comparison with a model that allows for residual (co)variance in the item parameters. The effect of the degree of residual (co)variance on the robustness of inferences is investigated in a simulation study.