Guido Makransky

Improving biotech education through gamified laboratory simulations

Gamified laboratory simulations motivate students and improve learning outcomes compared with traditional teaching methods.

Critical Values for Yen’s Q3: Identification of Local Dependence in the Rasch model using Residual Correlations

The assumption of local independence is central to all item response theory (IRT) models. Violations can lead to inflated estimates of reliability and problems with construct validity. For the most widely used fit statistic Q3, there are currently no well-documented suggestions of the critical values which should be used to indicate local dependence (LD), and for this reason, a variety of arbitrary rules of thumb are used. In this study, an empirical data example and Monte Carlo simulation were used to investigate the different factors that can influence the null distribution of residual correlations, with the objective of proposing guidelines that researchers and practitioners can follow when making decisions about LD during scale development and validation. A parametric bootstrapping procedure should be implemented in each separate situation to obtain the critical value of LD applicable to the data set, and provide example critical values for a number of data structure situations. The results show that for the Q3 fit statistic, no single critical value is appropriate for all situations, as the percentiles in the empirical null distribution are influenced by the number of items, the sample size, and the number of response categories. Furthermore, the results show that LD should be considered relative to the average observed residual correlation, rather than to a uniform value, as this results in more stable percentiles for the null distribution of an adjusted fit statistic.

Psychometric Properties of the Parent and Teacher ADHD Rating Scale (ADHD-RS) Measurement Invariance Across Gender, Age, and Informant

Attention deficit/hyperactivity disorder (ADHD) is one of the most common psychiatric disorders in childhood and adolescence. Rating the severity of psychopathology and symptom load is essential in daily clinical practice and in research. The parent and teacher ADHD-Rating Scale (ADHD-RS) includes inattention and hyperactivity/impulsivity subscales and is one of the most frequently used scales in treatment evaluation of children with ADHD. An extended version, mADHD-RS, also includes an oppositional defiant disorder subscale. The partial credit Rasch model, which is based on item response theory, was used to test the psychometric properties of this scale in a sample of 566 Danish school children between 6 and 16 years of age. The results indicated that parents and teachers had different frames of reference when rating symptoms in the mADHD-RS. There was support for the unidimensionality of the three subscales when parent and teacher ratings were analyzed independently. Nonetheless, evidence for differential item functioning was found across gender and age for specific items within each of the subscales. The findings expand existing psychometric information about the mADHD-RS and support its use as a valid and reliable measure of symptom severity when used in age- and gender-stratified materials.

Improving Personality Facet Scores With Multidimensional Computer Adaptive Testing An Illustration With the Neo Pi-R

Narrowly defined personality facet scores are commonly reported and used for making decisions in clinical and organizational settings. Although these facets are typically related, scoring is usually carried out for a single facet at a time. This method can be ineffective and time consuming when personality tests contain many highly correlated facets. This article investigates the possibility of increasing the precision of the NEO PI-R facet scores by scoring items with multidimensional item response theory and by efficiently administering and scoring items with multidimensional computer adaptive testing (MCAT). The increase in the precision of personality facet scores is obtained from exploiting the correlations between the facets. Results indicate that the NEO PI-R could be substantially shorter without attenuating precision when the MCAT methodology is used. Furthermore, the study shows that the MCAT methodology is particularly appropriate for constructs that have many highly correlated facets.

Unproctored Internet Test Verification: Using Adaptive Confirmation Testing

Unproctored Internet testing (UIT) is commonly used in employment test administration. When the test is high stakes, the International Guidelines on Computer-Based and Internet-Delivered Testing recommend to follow up the results with a confirmation test in a controlled setting. This article proposes and compares two methods for detecting whether a test taker’s original UIT responses are consistent with the responses from a follow-up confirmation test. The first method is a fixed length adaptive confirmation test using the likelihood ratio (LR) test to evaluate cheating and the second method is a variable length adaptive confirmation test using an extension of the stochastic curtailed truncated sequential probability ratio test (SCTSPRT) to evaluate cheating. Simulation studies indicated that the adaptive confirmation test using the SCTSPRT was almost four times shorter while maintaining the same detection power. The study also demonstrated that cheating can have a detrimental effect on the validity of a selection procedure and illustrated that the use of a confirmation test can remedy the negative effect of cheating on validity.

Equating the HBSC Family Affluence Scale across survey years: a method to account for item parameter drift using the Rasch model

AbstractPurpose To investigate the measurement invariance (MI) of the Family Affluence Scale (FAS) measured in the Health Behavior in School-aged Children (HBSC) survey, and to describe a method for equating the scale when MI is violated across survey years.Methods This study used a sample of 14,076 Norwegian and 17,365 Scottish adolescents from the 2002, 2006 and 2010 HBSC surveys to investigate the MI of the FAS across survey years. Violations of MI in the form of differential item functioning (DIF) due to item parameter drift (IPD) were modeled within the Rasch framework to ensure that the FAS scores from different survey years remain comparable.ResultsThe results indicate that the FAS is upwardly biased due to IPD in the computer item across survey years in the Norwegian and Scottish samples. Ignoring IPD across survey years resulted in the conclusion that family affluence is increasing quite consistently in Norway and Scotland. However, the results show that a large part of the increase in the FAS scores can be attributed to bias in the FAS because of IPD across time. The increase in the FAS was more modest in Scotland and slightly negative in Norway once the DIF in the computer item was accounted for in this study.ConclusionsWhen the comparison of family affluence is necessary over different HBSC survey years or when the longitudinal implications of family affluence are of interest, it is necessary to account for IPD in interpretation of changes in family affluence across time.

Early productive vocabulary predicts academic achievement 10 years later

We use a longitudinal design to examine associations for a diverse sample of 2,120 Danish 16- to 30-month-old children between early expressive vocabulary and later reading and math outcomes in the sixth grade. Educational outcomes, in particular decoding and reading comprehension, can be predicted from an early vocabulary measure as early as 16 months with effect sizes (in proportion of variance accounted for) comparable to 1 years mean growth in reading scores. The findings confirm in a relatively large population-based study that late talkers are at risk for low educational attainment because the majority of children experiencing early language delay obtain scores below average in measures of reading in the sixth grade. Low scores have the greatest predictive power, indicating that children with early delays have elevated risk for later reading problems.

As time goes by: reasons and characteristics of prolonged episodes of mechanical restraint in forensic psychiatry

Abstract Evidence suggests the prevalence and duration of mechanical restraint are particularly high among forensic psychiatric inpatients. However, only sparse knowledge exists regarding the reasons for, and characteristics of, prolonged use of mechanical restraint in forensic psychiatry. This study therefore aimed to investigate prolonged episodes of mechanical restraint on forensic psychiatric inpatients. Documentary data from medical records were thematically analyzed. Results show that the reasons for prolonged episodes of mechanical restraint on forensic psychiatric inpatients can be characterized by multiple factors: “confounding” (behaviors associated with psychiatric conditions, substance abuse, medical noncompliance, etc.), “risk” (behaviors posing a risk for violence), and “alliance parameters” (qualities of the staff–patient alliance and the patients’ openness to alliance with staff), altogether woven into a mechanical restraint spiral that in itself becomes a reason for prolonged mechanical restraint. The study also shows lack of consistent clinical assessment during periods of restraint. Further investigation is indicated to develop an assessment tool with the capability to reduce time spent in mechanical restraint.

Development and validation of the Multimodal Presence Scale for virtual reality environments

Presence is one of the most important psychological constructs for understanding human-computer interaction. However, different terminology and operationalizations of presence across fields have plagued the comparability and generalizability of results across studies. Lees (2004) unified understanding of presence as a multidimensional construct made up of physical, social, and self-presence, has created a unified theory of presence; nevertheless, there are still no psychometrically valid measurement instruments based on the theory. Two studies were conducted that describe the development of a standardized multidimensional measure of presence (the MPS) for a VR learning context based on this theory, and its validation using confirmatory factor analysis and item response theory. The results from Study 1 which included 161 medical students from Denmark indicated that the items used in the MPS measure a three dimensional theoretical model of presence: physical, social, and self-presence. Furthermore, IRT analyses indicated that it was possible to limit the number of items in the MPS to 15 (five items per sub-dimension) while maintaining the construct validity and reliability of the measure. The results of Study 2, which included 118 biology students from Scotland, supported the validity and generalizability of the MPS in a new context. A new scale measuring physical, social, and self-presence was developed.Confirmatory factor analysis supported the three-factor solution.Item response theory supported the validity and reliability of the 15 item measure.A cross-validation study provided evidence for the generalizability of the results.

Analysis of the Construct Validity and Measurement Invariance of the Career Decision Self-Efficacy Scale: A Rasch Model Approach

The Career Decision Self-Efficacy Scale–Short Form (CDSE-SF) is one of the most frequently used instruments to assess individual levels of career-related self-efficacy. This study used the partial credit model within the framework of item response theory to examine the content, structural, substantive, and generalizability aspects of validity for the CDSE-SF in a sample of 534 Australian high school students aged between 14 and 19 years. The results showed clear evidence of multidimensionality for the CDSE-SF. Furthermore, there was strong support for the content, structural, and substantive aspects of validity when using the five subscales individually. The evidence of measurement invariance was found across grade levels; however, there were individual items that exhibited differential item functioning across gender, achievement level, and age-groups. The implications for career counseling and research are discussed.