Joris Mulder

Multidimensional Adaptive Testing with Optimal Design Criteria for Item Selection

Several criteria from the optimal design literature are examined for use with item selection in multidimensional adaptive testing. In particular, it is examined what criteria are appropriate for adaptive testing in which all abilities are intentional, some should be considered as a nuisance, or the interest is in the testing of a composite of the abilities. Both the theoretical analyses and the studies of simulated data in this paper suggest that the criteria of A-optimality and D-optimality lead to the most accurate estimates when all abilities are intentional, with the former slightly outperforming the latter. The criterion of E-optimality showed occasional erratic behavior for this case of adaptive testing, and its use is not recommended. If some of the abilities are nuisances, application of the criterion of As-optimality (or Ds-optimality), which focuses on the subset of intentional abilities is recommended. For the measurement of a linear combination of abilities, the criterion of c-optimality yielded the best results. The preferences of each of these criteria for items with specific patterns of parameter values was also assessed. It was found that the criteria differed mainly in their preferences of items with different patterns of values for their discrimination parameters.

Bayes factors for testing inequality constrained hypotheses: Issues with prior specification

Several issues are discussed when testing inequality constrained hypotheses using a Bayesian approach. First, the complexity (or size) of the inequality constrained parameter spaces can be ignored. This is the case when using the posterior probability that the inequality constraints of a hypothesis hold, Bayes factors based on non-informative improper priors, and partial Bayes factors based on posterior priors. Second, the Bayes factor may not be invariant for linear one-to-one transformations of the data. This can be observed when using balanced priors which are centred on the boundary of the constrained parameter space with a diagonal covariance structure. Third, the information paradox can be observed. When testing inequality constrained hypotheses, the information paradox occurs when the Bayes factor of an inequality constrained hypothesis against its complement converges to a constant as the evidence for the first hypothesis accumulates while keeping the sample size fixed. This paradox occurs when using Zellners g prior as a result of too much prior shrinkage. Therefore, two new methods are proposed that avoid these issues. First, partial Bayes factors are proposed based on transformed minimal training samples. These training samples result in posterior priors that are centred on the boundary of the constrained parameter space with the same covariance structure as in the sample. Second, a g prior approach is proposed by letting g go to infinity. This is possible because the Jeffreys-Lindley paradox is not an issue when testing inequality constrained hypotheses. A simulation study indicated that the Bayes factor based on this g prior approach converges fastest to the true inequality constrained hypothesis.

Bayesian evaluation of inequality constrained hypotheses.

Bayesian evaluation of inequality constrained hypotheses enables researchers to investigate their expectations with respect to the structure among model parameters. This article proposes an approximate Bayes procedure that can be used for the selection of the best of a set of inequality constrained hypotheses based on the Bayes factor in a very general class of statistical models. The software package BIG is provided such that psychologists can use the approach proposed for the analysis of their own data. To illustrate the approximate Bayes procedure and the use of BIG, we evaluate inequality constrained hypotheses in a path model and a logistic regression model. Two simulation studies on the performance of our approximate Bayes procedure show that it results in accurate Bayes factors.

Bayesian evaluation of constrained hypotheses on variances of multiple independent groups.

Research has shown that independent groups often differ not only in their means, but also in their variances. Comparing and testing variances is therefore of crucial importance to understand the effect of a grouping variable on an outcome variable. Researchers may have specific expectations concerning the relations between the variances of multiple groups. Such expectations can be translated into hypotheses with inequality and/or equality constraints on the group variances. Currently, however, no methods are available for testing (in)equality constrained hypotheses on variances. This article proposes a novel Bayesian approach to this challenging testing problem. Our approach has the following useful properties: First, it can be used to simultaneously test multiple (non)nested hypotheses with equality as well as inequality constraints on the variances. Second, our approach is fully automatic in the sense that no subjective prior specification is needed. Only the hypotheses need to be provided. Third, a user-friendly software application is included that can be used to perform this Bayesian test in an easy manner.

Relative Effects at Work Bayes Factors for Order Hypotheses

Assessing the relative importance of predictors has been of historical importance in a variety of disciplines including management, medicine, economics, and psychology. When approaching hypotheses on the relative ordering of the magnitude of predicted effects (e.g., the effects of discrimination from managers and coworkers are larger than that from clients), one quickly runs into problems within a traditional frequentist framework. Null hypothesis significance testing does not allow researchers to directly map research hypotheses on to results and suffers from a multiple testing problem that leads to low statistical power. Furthermore, all traditional structural equation modeling fit indices lose much of their suitability for model comparison, because order hypotheses are not countable in terms of degrees of freedom. To adequately tackle order hypotheses, we advocate a Bayesian method that provides a single internally consistent solution for estimation and inference. The key element in the proposed model comparison approach is the use of the Bayes factor and the incorporation of order constraints by means of a smart formulation of prior distributions. An easy-to-use software package BIEMS (Bayesian inequality and equality constrained model selection) is introduced and two empirical examples in the organizational behavior area are provided to showcase the method, both offering new findings that have implications for theory: the first on the differential impact of discrimination in the workplace from insiders and outsiders to the organization on employees’ well-being, and the second on Karasek’s stressor–strain theory about how the relative order of magnitude of the effects of job control and demands depends on the specific well-being outcome dimension.

Bayesian tests on components of the compound symmetry covariance matrix

Complex dependency structures are often conditionally modeled, where random effects parameters are used to specify the natural heterogeneity in the population. When interest is focused on the dependency structure, inferences can be made from a complex covariance matrix using a marginal modeling approach. In this marginal modeling framework, testing covariance parameters is not a boundary problem. Bayesian tests on covariance parameter(s) of the compound symmetry structure are proposed assuming multivariate normally distributed observations. Innovative proper prior distributions are introduced for the covariance components such that the positive definiteness of the (compound symmetry) covariance matrix is ensured. Furthermore, it is shown that the proposed priors on the covariance parameters lead to a balanced Bayes factor, in case of testing an inequality constrained hypothesis. As an illustration, the proposed Bayes factor is used for testing (non-)invariant intra-class correlations across different group types (public and Catholic schools), using the 1982 High School and Beyond survey data.

Prior sensitivity analysis in default Bayesian structural equation modeling

Abstract Bayesian structural equation modeling (BSEM) has recently gained popularity because it enables researchers to fit complex models and solve some of the issues often encountered in classical maximum likelihood estimation, such as nonconvergence and inadmissible solutions. An important component of any Bayesian analysis is the prior distribution of the unknown model parameters. Often, researchers rely on default priors, which are constructed in an automatic fashion without requiring substantive prior information. However, the prior can have a serious influence on the estimation of the model parameters, which affects the mean squared error, bias, coverage rates, and quantiles of the estimates. In this article, we investigate the performance of three different default priors: noninformative improper priors, vague proper priors, and empirical Bayes priors—with the latter being novel in the BSEM literature. Based on a simulation study, we find that these three default BSEM methods may perform very differently, especially with small samples. A careful prior sensitivity analysis is therefore needed when performing a default BSEM analysis. For this purpose, we provide a practical step-by-step guide for practitioners to conducting a prior sensitivity analysis in default BSEM. Our recommendations are illustrated using a well-known case study from the structural equation modeling literature, and all code for conducting the prior sensitivity analysis is available in the online supplemental materials.

One after the other: Effects of sequence patterns of breached and overfulfilled obligations

ABSTRACT To date, the study of psychological contracts has primarily centred on the question how retrospective evaluations of the psychological contract impact employee attitudes and behaviours, and/or focus on individual coping processes in explaining responses to breached or overfulfilled obligations. In this study, we aim to assess the extent to which sequences of breached and overfulfilled obligations impact job satisfaction and citizenship behaviour intentions. By integrating psychological contract research and theories on cognitive information processing, we formulate competing hypotheses on how sequences of breached and/or overfulfilled obligations lead to patterns of job satisfaction and citizenship behaviour intentions. These competing hypotheses were tested using a vignette study and an experiment. A Bayesian approach was used to test these pattern hypotheses directly against each other. The results show that breached obligations have an immediate negative impact on our outcome variables. Moreover, sequentially breached obligations lead to a continuous decline of job satisfaction and citizenship behaviour intentions. Overfulfilled obligations do little to compensate this negative impact. Implications for theory and practice are discussed.

Bayesian estimation of the network autocorrelation model

The network autocorrelation model has been extensively used by researchers interested modeling social influence effects in social networks. The most common inferential method in the model is classical maximum likelihood estimation. This approach, however, has known problems such as negative bias of the network autocorrelation parameter and poor coverage of confidence intervals. In this paper, we develop new Bayesian techniques for the network autocorrelation model that address the issues inherent to maximum likelihood estimation. A key ingredient of the Bayesian approach is the choice of the prior distribution. We derive two versions of Jeffreys prior, the Jeffreys rule prior and the Independence Jeffreys prior, which have not yet been developed for the network autocorrelation model. These priors can be used for Bayesian analyses of the model when prior information is completely unavailable. Moreover, we propose an informative as well as a weakly informative prior for the network autocorrelation parameter that are both based on an extensive literature review of empirical applications of the network autocorrelation model across many fields. Finally, we provide new and efficient Markov Chain Monte Carlo algorithms to sample from the resulting posterior distributions. Simulation results suggest that the considered Bayesian estimators outperform the maximum likelihood estimator with respect to bias and frequentist coverage of credible and confidence intervals.

Bayes factor covariance testing in item response models

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.