Lawrence T. DeCarlo

On the meaning and use of kurtosis

For symmetric unimodal distributions, positive kurtosis indicates heavy tails and peakedness relative to the normal distribution, whereas negative kurtosis indicates light tails and flatness. Many textbooks, however, describe or illustrate kurtosis incompletely or incorrectly. In this article, kurtosis is illustrated with well-known distributions, and aspects of its interpretation and misinterpretation are discussed. The role of kurtosis in testing univariate and multivariate normality; as a measure of departures from normality; in issues of robustness, outliers, and bimodality; in generalized tests and estimators, as well as limitations of and alternatives to the kurtosis measure [32, are discussed.

Signal detection theory with finite mixture distributions: theoretical developments with applications to recognition memory.

An extension of signal detection theory (SDT) that incorporates mixtures of the underlying distributions is presented. The mixtures can be motivated by the idea that a presentation of a signal shifts the location of an underlying distribution only if the observer is attending to the signal; otherwise, the distribution is not shifted or is only partially shifted. Thus, trials with a signal presentation consist of a mixture of 2 (or more) latent classes of trials. Mixture SDT provides a general theoretical framework that offers a new perspective on a number of findings. For example, mixture SDT offers an alternative to the unequal variance signal detection model; it can also account for nonlinear normal receiver operating characteristic curves, as found in recent research.

On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix

Cognitive diagnostic models (CDMs) attempt to uncover latent skills or attributes that examinees must possess in order to answer test items correctly. The DINA (deterministic input, noisy ‘‘and’’) model is a popular CDM that has been widely used. It is shown here that a logistic version of the model can easily be fit with standard software for latent class analysis. A partly Bayesian approach to estimation, posterior mode estimation, is used as a simple alternative to a fully Bayesian approach via Markov chain Monte Carlo methods. A latent-class analysis of a widely analyzed data set, the fraction subtraction data of K. K. Tatsuoka, reveals some neglected problems with respect to the classification of examinees; for example, examinees who get all of the items incorrect are classified as having most of the skills. It is also noted that obtaining large estimates of the latent class sizes can indicate misspecification of the Q-matrix, such as the inclusion of an irrelevant skill. It is shown, analytically and via simulations, that the problems are largely associated with the structure of the Q-matrix.

Regularities of source recognition: ROC analysis

Source memory has become the focus of a growing number of investigations in a variety of fields. An appropriate model for source memory is, therefore, of increasing importance. A simple 2-dimensional signal-detection model of source recognition is presented. The receiver operating characteristics (ROCs) obtained from 3 experiments are then used to test the model. The data demonstrate 3 regularities: convex ROCs, z-ROCs with linear slopes of 1.00, and slightly concave z-ROCs. Two of these regularities support the model. The 3rd requires a revision of the model. This revised model is fitted to the data. The implications of these regularities for other theories are also discussed.

Work Interest as a Predictor of Competitive Employment: Policy Implications for Psychiatric Rehabilitation

Consumers with serious mental illness (N=166) enrolling in two community-based mental health programs, a vocational Program of Assertive Community Treatment and a clubhouse certified by the International Center for Clubhouse Development (ICCD), were asked about their interest in work. About one third of the new enrollees expressed no interest in working. Equivalent supported employment services were then offered to all participants in each program. Stated interest in work and receipt of vocational services were statistically significant predictors of whether a person would work and how long it would take to get a job. Two thirds of those interested in work and half of those with no initial interest obtained a competitive job if they received at least one hour of vocational service. Once employed, these two groups held comparable jobs for the same length of time. These findings demonstrate the importance of making vocational services continuously available to all people with serious mental illness, and the viability of integrating these services into routine mental health care.

Recognizing Uncertainty in the Q-Matrix via a Bayesian Extension of the DINA Model

In the typical application of a cognitive diagnosis model, the Q-matrix, which reflects the theory with respect to the skills indicated by the items, is assumed to be known. However, the Q-matrix is usually determined by expert judgment, and so there can be uncertainty about some of its elements. Here it is shown that this uncertainty can be recognized and explored via a Bayesian extension of the DINA (deterministic input noisy and) model. The approach used is to specify some elements of the Q-matrix as being random rather than as fixed; posterior distributions can then be used to obtain information about elements whose inclusion in the Q-matrix is questionable. Simulations show that this approach helps to recover the true Q-matrix when there is uncertainty about some elements. An application to the fraction-subtraction data of K. K. Tatsuoka suggests a modified Q-matrix that gives improved relative fit.

Source monitoring and multivariate signal detection theory, with a model for selection ☆

Participants in source monitoring studies,in addition to determining whether an item is old or new,also discriminate the source of the item,such as whether the item was presented in a male or female voice. This article shows how to apply multivariate signal detection theory (SDT) to source monitoring. An interesting aspect of one version of the source monitoring procedure,from the perspective of multivariate SDT,is that it involves a type of selection,in that a discrimination response is observed only if the detection decision is that an item is old. If the selection is ignored,then the estimate of the discrimination parameter can be biased; the nature and magnitude of the bias are illustrated. A bivariate signal detection model that recognizes selection is presented and its application is illustrated. The approach to source monitoring via multivariate SDT provides new results that are informative about underlying psychological processes. r 2003 Elsevier Science (USA). All rights reserved.

An application of signal detection theory with finite mixture distributions to source discrimination.

A mixture extension of signal detection theory is applied to source discrimination. The basic idea of the approach is that only a portion of the sources (say A or B) of items to be discriminated is encoded or attended to during the study period. As a result, in addition to 2 underlying probability distributions associated with the 2 sources, there is a 3rd distribution that represents items for which sources were not attended to. Thus, over trials, the observed response results from a mixture of an attended (A or B) distribution and a nonattended distribution. The situation differs in an interesting way from detection in that, for detection, there is mixing only on signal trials and not on noise trials, whereas for discrimination, there is mixing on both A and B trials. Predictions of the mixture model are examined for data from several recent studies and in a new experiment.

The Mirror Effect and Mixture Signal Detection Theory

The mirror effect for word frequency refers to the finding that low-frequency words have higher hit rates and lower false alarm rates than high-frequency words. This result is typically interpreted in terms of conventional signal detection theory (SDT), in which case it indicates that the order of the underlying old item distributions mirrors the order of the new item distributions. However, when viewed in terms of a mixture version of SDT, the order of hits and false alarms does not necessarily imply the same order in the underlying distributions because of possible effects of mixing. A reversal in underlying distributions did not appear for fits of mixture SDT models to data from 4 experiments.

Using the PLUM procedure of SPSS to fit unequal variance and generalized signal detection models

The recent addition of a procedure in SPSS for the analysis of ordinal regression models offers a simple means for researchers to fit the unequal variance normal signal detection model and other extended signal detection models. The present article shows how to implement the analysis and how to interpret the SPSS output. Examples of fitting the unequal variance normal model and other generalized signal detection models are given. The approach offers a convenient means for applying signal detection theory to a variety of research.