George Karabatsos

Comparing the Aberrant Response Detection Performance of Thirty-Six Person-Fit Statistics.

The accurate measurement of examinee test performance is critical to educational decision-making, and inaccurate measurement can lead to negative consequences for examinees. Person-fit statistics are important in a psychometric analysis for detecting examinees with aberrant response patterns that lead to inaccurate measurement. Unfortunately, although a large number of person-fit statistics is available, there is little consensus as to which ones are most useful. The purpose of this study was to compare 36 person-fit indices, under different testing conditions, to obtain a better consensus as to their relative merits. The results of these comparisons, and their implications, are discussed.

Markov chain estimation for test theory without an answer key

This study develops Markov Chain Monte Carlo (MCMC) estimation theory for the General Condorcet Model (GCM), an item response model for dichotomous response data which does not presume the analyst knows the correct answers to the test a priori (answer key). In addition to the answer key, respondent ability, guessing bias, and difficulty parameters are estimated. With respect to data-fit, the study compares between the possible GCM formulations, using MCMC-based methods for model assessment and model selection. Real data applications and a simulation study show that the GCM can accurately reconstruct the answer key from a small number of respondents.

Order-Constrained Bayes Inference for Dichotomous Models of Unidimensional Nonparametric IRT

This study introduces an order-constrained Bayes inference framework useful for analyzing data containing dichotomous-scored item responses, under the assumptions of either the monotone homogeneity model or the double monotonicity model of nonparametric item response theory (NIRT). The framework involves the implementation of Gibbs sampling to estimate order-constrained parameters, followed by inference with the posterior-predictive distribution to test the monotonicity, invariant item ordering, and local independence assumptions of NIRT. The Bayes framework is demonstrated through the analysis of real test data, and possible extensions of it are discussed.

Adaptive-modal Bayesian nonparametric regression

© 2012 by Institute of Mathematical Statistics, Electronic Journal of Statistics. The original publication is available at http://www.imstat.org/ejs/ DOI: 10.1214/12-EJS738

Mixed-effects Poisson regression analysis of adverse event reports: The relationship between antidepressants and suicide

A new statistical methodology is developed for the analysis of spontaneous adverse event (AE) reports from post-marketing drug surveillance data. The method involves both empirical Bayes (EB) and fully Bayes estimation of rate multipliers for each drug within a class of drugs, for a particular AE, based on a mixed-effects Poisson regression model. Both parametric and semiparametric models for the random-effect distribution are examined. The method is applied to data from Food and Drug Administration (FDA)s Adverse Event Reporting System (AERS) on the relationship between antidepressants and suicide. We obtain point estimates and 95 per cent confidence (posterior) intervals for the rate multiplier for each drug (e.g. antidepressants), which can be used to determine whether a particular drug has an increased risk of association with a particular AE (e.g. suicide). Confidence (posterior) intervals that do not include 1.0 provide evidence for either significant protective or harmful associations of the drug and the adverse effect. We also examine EB, parametric Bayes, and semiparametric Bayes estimators of the rate multipliers and associated confidence (posterior) intervals. Results of our analysis of the FDA AERS data revealed that newer antidepressants are associated with lower rates of suicide adverse event reports compared with older antidepressants. We recommend improvements to the existing AERS system, which are likely to improve its public health value as an early warning system.

Enumerating and testing conjoint measurement models

Abstract A Monte Carlo procedure was used to generate three types of 3×3×2 conjoint measurement structures, each type having an additive (respectively, distributive, dual-distributive) simple polynomial representation. The Monte Carlo results illustrate the restrictiveness of the axioms of joint independence (respectively, distributive cancellation, dual-distributive cancellation), and show how close each comes to characterizing the relevant simple polynomial representation. An empirical example illustrates how the generated measurement structures aid in the selection and testing of axiomatic models using order-restricted statistical inference.

A Bayesian nonparametric meta-analysis model

In a meta-analysis, it is important to specify a model that adequately describes the effect-size distribution of the underlying population of studies. The conventional normal fixed-effect and normal random-effects models assume a normal effect-size population distribution, conditionally on parameters and covariates. For estimating the mean overall effect size, such models may be adequate, but for prediction, they surely are not if the effect-size distribution exhibits non-normal behavior. To address this issue, we propose a Bayesian nonparametric meta-analysis model, which can describe a wider range of effect-size distributions, including unimodal symmetric distributions, as well as skewed and more multimodal distributions. We demonstrate our model through the analysis of real meta-analytic data arising from behavioral-genetic research. We compare the predictive performance of the Bayesian nonparametric model against various conventional and more modern normal fixed-effects and random-effects models.

A Statistician’s View on Bayesian Evaluation of Informative Hypotheses

Theory testing lies at the heart of the scientific process. This is especially true in psychology, where, typically, multiple theories are advanced to explain a given psychological phenomenon, such as a mental disorder or a perceptual process. It is therefore important to have a rigorous methodology available for the psychologist to evaluate the validity and viability of such theories, or models for that matter. However, it may be argued that the current practice of theory testing is not entirely satisfactory.

A menu-driven software package of Bayesian nonparametric (and parametric) mixed models for regression analysis and density estimation

Most of applied statistics involves regression analysis of data. In practice, it is important to specify a regression model that has minimal assumptions which are not violated by data, to ensure that statistical inferences from the model are informative and not misleading. This paper presents a stand-alone and menu-driven software package, Bayesian Regression: Nonparametric and Parametric Models, constructed from MATLAB Compiler. Currently, this package gives the user a choice from 83 Bayesian models for data analysis. They include 47 Bayesian nonparametric (BNP) infinite-mixture regression models; 5 BNP infinite-mixture models for density estimation; and 31 normal random effects models (HLMs), including normal linear models. Each of the 78 regression models handles either a continuous, binary, or ordinal dependent variable, and can handle multi-level (grouped) data. All 83 Bayesian models can handle the analysis of weighted observations (e.g., for meta-analysis), and the analysis of left-censored, right-censored, and/or interval-censored data. Each BNP infinite-mixture model has a mixture distribution assigned one of various BNP prior distributions, including priors defined by either the Dirichlet process, Pitman-Yor process (including the normalized stable process), beta (two-parameter) process, normalized inverse-Gaussian process, geometric weights prior, dependent Dirichlet process, or the dependent infinite-probits prior. The software user can mouse-click to select a Bayesian model and perform data analysis via Markov chain Monte Carlo (MCMC) sampling. After the sampling completes, the software automatically opens text output that reports MCMC-based estimates of the model’s posterior distribution and model predictive fit to the data. Additional text and/or graphical output can be generated by mouse-clicking other menu options. This includes output of MCMC convergence analyses, and estimates of the model’s posterior predictive distribution, for selected functionals and values of covariates. The software is illustrated through the BNP regression analysis of real data.

A Bayesian Nonparametric Causal Model for Regression Discontinuity Designs

For non-randomized studies, the regression discontinuity design (RDD) can be used to identify and estimate causal effects from a “locally randomized” subgroup of subjects, under relatively mild conditions. However, current models focus causal inferences on the impact of the treatment (versus non-treatment) variable on the mean of the dependent variable, via linear regression. For RDDs, we propose a flexible Bayesian nonparametric regression model that can provide accurate estimates of causal effects, in terms of the predictive mean, variance, quantile, probability density, distribution function, or any other chosen function of the outcome variable. We illustrate the model through the analysis of two real educational data sets, involving (resp.) a sharp RDD and a fuzzy RDD.