Pasquale Anselmi

On the Link between Cognitive Diagnostic Models and Knowledge Space Theory

The present work explores the connections between cognitive diagnostic models (CDM) and knowledge space theory (KST) and shows that these two quite distinct approaches overlap. It is proved that in fact the Multiple Strategy DINA (Deterministic Input Noisy AND-gate) model and the CBLIM, a competence-based extension of the basic local independence model (BLIM), are equivalent. To demonstrate the benefits that arise from integrating the two theoretical perspectives, it is shown that a fairly complete picture on the identifiability of these models emerges by combining results from both camps. The impact of the results is illustrated by an empirical example, and topics for further research are pointed out.

Assessing the local identifiability of probabilistic knowledge structures

Given a collection Q of problems, in knowledge space theory Doignon & Falmagne, (International Journal of Man–Machine Studies 23:175–196, 1985) the knowledge state of a student is the collection K ⊆ Q of all problems that this student is capable of solving. A knowledge structure is a pair (Q, ), where is a collection of knowledge states that contains at least the empty set and Q. A probabilistic knowledge structure (PKS) is a knowledge structure (Q, , π), where π is a probability distribution on the knowledge states. The PKS that has received the most attention is the basic local independence model BLIM; Falmagne & Doignon, (British Journal of Mathematical and Statistical Psychology 41:1–23, 1988a, Journal of Mathematical Psychology 32:232–258, 1988b). To the best of our knowledge, systematic investigations in the literature concerning the identifiability of the BLIM are totally missing. Based on the theoretical work of Bamber and van Santen (Journal of Mathematical Psychology 29:443–473, 1985), the present article is aimed to present a method and a corresponding computerized procedure for assessing the local identifiability of the BLIM, which is applicable to any finite knowledge structure of moderate size.

Measurement of change in health status with Rasch models

BackgroundThe traditional approach to the measurement of change presents important drawbacks (no information at individual level, ordinal scores, variance of the measurement instrument across time points), which Rasch models overcome. The article aims to illustrate the features of the measurement of change with Rasch models.MethodsTo illustrate the measurement of change using Rasch models, the quantitative data of a longitudinal study of heart-surgery patients (N = 98) were used. The scale “Perception of Positive Change” was used as an example of measurement instrument. All patients underwent cardiac rehabilitation, individual psychological intervention, and educational intervention. Nineteen patients also attended progressive muscle relaxation group trainings. The scale was administered before and after the interventions. Three Rasch approaches were used. Two separate analyses were run on the data from the two time points to test the invariance of the instrument. An analysis was run on the stacked data from both time points to measure change in a common frame of reference. Results of the latter analysis were compared with those of an analysis that removed the influence of local dependency on patient measures. Statistics t, χ2 and F were used for comparing the patient and item measures estimated in the Rasch analyses (a-priori α = .05). Infit, Outfit, R and item Strata were used for investigating Rasch model fit, reliability, and validity of the instrument.ResultsData of all 98 patients were included in the analyses. The instrument was reliable, valid, and substantively unidimensional (Infit, Outfit < 2 for all items, R = .84, item Strata range = 3.93-6.07). Changes in the functioning of the instrument occurred across the two time, which prevented the use of the two separate analyses to unambiguously measure change. Local dependency had a negligible effect on patient measures (p ≥ .8674). Thirteen patients improved, whereas 3 worsened. The patients who attended the relaxation group trainings did not report greater improvement than those who did not (p = .1007).ConclusionsRasch models represent a valid framework for the measurement of change and a useful complement to traditional approaches.

Preferring thin people does not imply derogating fat people. A rasch analysis of the implicit weight attitude

The meaning of the implicit weight attitude in individuals of different weight by distinguishing the contribution of positive and negative associations to the overall measure was investigated.

Implicit Sexual Attitude of Heterosexual, Gay and Bisexual Individuals: Disentangling the Contribution of Specific Associations to the Overall Measure

The article aims to measure implicit sexual attitude in heterosexual, gay and bisexual individuals. A Many-Facet Rasch Measurement analysis was used to disentangle the contribution of specific associations to the overall IAT measure. A preference for heterosexuals relative to homosexuals is observed in heterosexual respondents, driven most by associating positive attributes with heterosexuals rather than negative attributes with homosexuals. Differently, neither the negative nor the positive evaluation of any of the target groups play a prominent role in driving the preference for homosexuals observed in gay respondents. A preference for heterosexuals relative to homosexuals is observed in bisexual respondents, that results most from ascribing negative attributes to homosexuals rather than positive attributes to heterosexuals. The results are consistent with the expression of the need for achieving a positive self-image and with the influence of shared social norms concerning sexuality.

Positive Associations Primacy in the IAT

Two studies investigated the different contribution of positive and negative associations to the size of the Implicit Association Test (IAT) effect. A Many-Facet Rasch Measurement analysis was applied for the purpose. Across different IATs (Race and Weight) and different groups of respondents (White, Normal weight, and Obese people) we observed that positive words increase the IAT effect whereas negative words tend to decrease it. Results suggest that the IAT is influenced by a positive associations primacy effect. As a consequence, we argue that researchers should be careful when interpreting IAT effects as a measure of implicit prejudice.

Assessing learning processes with the gain-loss model.

Within the framework of knowledge space theory, a probabilistic skill multimap model for assessing learning processes is proposed. The learning process of a student is modeled as a function of the interaction between his competence state and the effect of a learning object on specific skills. Model parameters are initial probabilities of the skills, effects of learning objects on gaining and losing the skills, careless error, and lucky guess probabilities of the problems. A simulation study assessed model identifiability and goodness-of-recovery under several conditions. Practical implications of using the model are discussed, and the MATLAB code for simulating, estimating and testing it is available in the Psychonomic Society supplemental archive.

Assessing Parameter Invariance in the BLIM: Bipartition Models

In knowledge space theory, the knowledge state of a student is the set of all problems he is capable of solving in a specific knowledge domain and a knowledge structure is the collection of knowledge states. The basic local independence model (BLIM) is a probabilistic model for knowledge structures. The BLIM assumes a probability distribution on the knowledge states and a lucky guess and a careless error probability for each problem. A key assumption of the BLIM is that the lucky guess and careless error probabilities do not depend on knowledge states (invariance assumption). This article proposes a method for testing the violations of this specific assumption. The proposed method was assessed in a simulation study and in an empirical application. The results show that (1) the invariance assumption might be violated by the empirical data even when the model’s fit is very good, and (2) the proposed method may prove to be a promising tool to detect invariance violations of the BLIM.

Modeling missing data in knowledge space theory.

Missing data are a well known issue in statistical inference, because some responses may be missing, even when data are collected carefully. The problem that arises in these cases is how to deal with missing data. In this article, the missingness is analyzed in knowledge space theory, and in particular when the basic local independence model (BLIM) is applied to the data. Two extensions of the BLIM to missing data are proposed: The former, called ignorable missing BLIM (IMBLIM), assumes that missing data are missing completely at random; the latter, called missing BLIM (MissBLIM), introduces specific dependencies of the missing data on the knowledge states, thus assuming that the missing data are missing not at random. The IMBLIM and the MissBLIM modeled the missingness in a satisfactory way, in both a simulation study and an empirical application, depending on the process that generates the missingness: If the missing data-generating process is of type missing completely at random, then either IMBLIM or MissBLIM provide adequate fit to the data. However, if the pattern of missingness is functionally dependent upon unobservable features of the data (e.g., missing answers are more likely to be wrong), then only a correctly specified model of the missingness distribution provides an adequate fit to the data.

A short questionnaire to assess pediatric resident’s competencies: the validation process

BackgroundIn order to help assess resident performance during training, the Residency Affair Committee of the Pediatric Residency Program of the University of Padua (Italy) administered a Resident Assessment Questionnaire (ReAQ), which both residents and faculty were asked to complete. The aim of this article is to present the ReAQ and its validation.MethodsThe ReAQ consists of 20 items that assess the six core competencies identified by the Accreditation Council of Graduate Medical Education (ACGME). A many-facet Rasch measurement analysis was used for validating the ReAQ.ResultsBetween July 2011 and June 2012, 211 evaluations were collected from residents and faculty. Two items were removed because their functioning changed with the gender of respondents. The step calibrations were ordered. The self evaluations (residents rating themselves) positively correlated with the hetero evaluations (faculty rating residents; Spearman’s ρ = 0.75, p < 0.001). Unfortunately, the observed agreement among faculty was smaller than expected (Exp = 47.1%; Obs = 41%), which indicates that no enough training to faculty for using the tool was provided.ConclusionsIn its final form, the ReAQ provides a valid unidimensional measure of core competences in pediatric residents. It produces reliable measures, distinguishes among groups of residents according to different levels of performance, and provides a resident evaluation that holds an analogous meaning for residents and faculty.