Heinz Holling

Advances in optimum experimental design for conjoint analysis and discrete choice models

The authors review current developments in experimental design for conjoint analysis and discrete choice models emphasizing the issue of design efficiency. Drawing on recently developed optimal paired comparison designs, theoretical as well as empirical evidence is provided that established design strategies can be improved with respect to design efficiency.

Optimal paired comparison designs for first-order interactions

In many fields of applications paired comparisons are used in which either full or partial profiles of the alternatives are presented. For this situation we introduce an appropriate model and derive optimal designs in the presence of interactions when all attributes have the same number of levels.

Attitudes Towards Severity of Punishment: A Conjoint Analytic Approach

Past research suggests that attitudes towards severity of punishment are affected by crime‐specific factors. The impact of such factors has usually been investigated by between‐subjects designs. The studies reported in this paper, however, are based on within‐subjects designs, using conjoint analysis for data collection and analysis. Study 1 employs a rape scenario for investigating the impact of the victim–offender relationship and of two victim characteristics – provocative behavior and intoxication. Study 2 uses a theft and an assault scenario for analyzing the influence of several offender and crime characteristics on sanctioning: offenders age, readiness to confess, previous convictions, and severity of the offense. Results from both studies are reported and discussed in terms of utility values. These values represent the importance placed on the case characteristics focused upon. In addition to the general evaluation of case characteristics, inter‐individual differences are analyzed by means of hierarchical cluster analysis. Advantages of the conjoint analytic approach over conventional research methods on sanctioning behavior are discussed.

Efficient Designs for Paired Comparisons with a Polynomial Factor

In psychological research paired comparisons, which demand judges to evaluate the trade-off between two alternatives, have been shown to yield valid estimates of the judges’ preferences. For this situation we present optimal and efficient designs in a response surface setting where the alternatives are modelled by a polynomial.

The Application of Statistical Methods of Meta-Analysis for Heterogeneity Modelling in Medicine and Pharmacy, Psychology, Quality Control and Assurance

In the past few years meta-analysis has become increasingly popular in many areas of science such as medicine and pharmacy, psychology and other social sciences. In these areas of application meta-analyses have been performed in order to obtain a pooled estimate of various single studies. Obtaining a single summary measure implicitly assumes homogeneity of these studies, i.e. the results of individual studies differ only by chance. In this case a combined estimate of the individual studies provides a powerful and important result. However this pooled estimate may be seriously misleading if study conditions are heterogenous.

Optimal Designs for Linear Logistic Test Models

An important class of models within item response theory are Linear Logistic Test Models (LLTM). These models provide a means for rule-based item generation in educational and psychological testing based upon cognitive theories. After a short introduction into the LLTM, optimal designs for the LLTM will be developed with respect to the item calibration step assuming that persons’ abilities are known. Therefore, the LLTM is embedded in a particular generalized linear model. Finally, future developments are outlined.

Transitioning from Fixed-Length Questionnaires to Computer-Adaptive Versions

To investigate how an existing questionnaire can be transformed into a computer-adaptive version, we developed an adaptive version of the Interpersonal Competence Questionnaire (ICQ). This adaptive version was based on a representative sample (N = 1934) of respondents who answered 30 items from a German translation of the ICQ. A random half of the sample was used to evaluate test dimensionality, calibrate the items, and model the relation between person parameters and raw total scores. The other random half of the sample was employed to assess the comparability of person parameters and raw scores. After these tests and item calibration, 28 items remained in the item pool. A high correlation was found between raw scores and estimated scores using all items. Raw scores could be predicted accurately from estimated person parameters. These results indicate that our approach is an effective technique for transforming an existing questionnaire into a computer-adaptive version.

Meta-Analysis of Binary Data Based upon Dichotomized Criteria

Abstract. This paper considers meta-analysis of binary data that use a dichotomized continuous score. Classification into two categories, e.g., qualified or not qualified, is often based upon a threshold or cut-off value. This threshold might vary between studies since intentionally different values are used. However, conventional meta-analysis methodology analyzing sensitivity and specificity separately might then be confounded by a potentially unknown variation of the cut-off value. In order to cope with varying thresholds, an overall estimate of the misclassification error is suggested instead, which is equivalent to the well-known Youden index. It is argued that this index is less prone to between-study variation of cut-off values. To adjust for potential study effects a Mantel-Haenszel estimator of the overall misclassification error is suggested. Arguments are illustrated using, as an example, the diagnosis of alcoholism using the Alcohol Use Disorders Identification Test (AUDIT).

A Latent Ability Model for Count Data and Application to Processing Speed

A new family of item response theory models for count data, based on item characteristic curves (ICCs) of binary models, is presented. These models assume a Poisson distribution for the observed scores where the mean is given by the product of a speed parameter and an ICC, for example, the curve of the one- or two-parameter logistic model. Joint and marginal maximum likelihood parameter estimations are discussed and the proposed procedures are evaluated by computer simulation. As an application, item level data from a test measuring processing speed are analyzed and item fit and test information are explored.

Optimum design of experiments for statistical inference: discussion

Summary.  One attractive feature of optimum design criteria, such as D- and A-optimality, is that they are directly related to statistically interpretable properties of the designs that are obtained, such as minimizing the volume of a joint confidence region for the parameters. However, the assumed relationships with inferential procedures are valid only if the variance of experimental units is assumed to be known. If the variance is estimated, then the properties of the inferences depend also on the number of degrees of freedom that are available for estimating the error variance. Modified optimality criteria are defined, which correctly reflect the utility of designs with respect to some common types of inference. For fractional factorial and response surface experiments, the designs that are obtained are quite different from those which are optimal under the standard criteria, with many more replicate points required to estimate error. The optimality of these designs assumes that inference is the only purpose of running the experiment, but in practice interpretation of the point estimates of parameters and checking for lack of fit of the treatment model assumed are also usually important. Thus, a compromise between the new criteria and others is likely to be more relevant to many practical situations. Compound criteria are developed, which take account of multiple objectives, and are applied to fractional factorial and response surface experiments. The resulting designs are more similar to standard designs but still have sufficient residual degrees of freedom to allow effective inferences to be carried out. The new procedures developed are applied to three experiments from the food industry to see how the designs used could have been improved and to several illustrative examples. The design optimization is implemented through a simple exchange algorithm.