g-Te Chen

Using SAS PROC NLMIXED to fit item response theory models

Researchers routinely construct tests or questionnaires containing a set of items that measure personality traits, cognitive abilities, political attitudes, and so forth. Typically, responses to these items are scored in discrete categories, such as points on a Likert scale or a choice out of several mutually exclusive alternatives. Item response theory (IRT) explains observed responses to items on a test (questionnaire) by a person’s unobserved trait, ability, or attitude. Although applications of IRT modeling have increased considerably because of its utility in developing and assessing measuring instruments, IRT modeling has not been fully integrated into the curriculum of colleges and universities, mainly because existing general purpose statistical packages do not provide built-in routines with which to perform IRT modeling. Recent advances in statistical theory and the incorporation of those advances into general purpose statistical software such as the Statistical Analysis System (SAS) allow researchers to analyze measurement data by using a class of models known as generalized linear mixed effects models (McCulloch & Searle, 2001), which include IRT models as special cases. The purpose of this article is to demonstrate the generality and flexibility of using SAS to estimate IRT model parameters. With real data examples, we illustrate the implementations of a variety of IRT models for dichotomous, polytomous, and nominal responses. Since SAS is widely available in educational institutions, it is hoped that this article will contribute to the spread of IRT modeling in quantitative courses.

Effects of Ignoring Item Interaction on Item Parameter Estimation and Detection of Interacting Items

This study explores the effects of ignoring item interaction on item parameter estimation and the efficiency of using the local dependence index Q3 and the SAS NLMIXED procedure to detect item interaction under the three-parameter logistic model and the generalized partial credit model. Through simulations, it was found that ignoring positive item interaction led to overestimation for the discrimination parameters, underestimation for the difficulty parameters, and a Q3 much smaller than zero. As the guessing parameters approached zero, the overestimation for the discrimination parameters became more serious. In contrast, ignoring negative item interaction led to underestimation for the discrimination parameters, overestimation for the difficulty parameters, and a Q3 much larger than zero. As the guessing parameters approached zero, the underestimation for the discrimination parameters became less serious. A modification of posterior predictive p value for Q3 was proposed to detect item interaction and was found to work very well. Direct modeling of item interaction using NLMIXED was demonstrated.

A prospective study of the responsiveness of the original and the short form Berg Balance Scale in people with stroke

Objective: The study aim was to examine the responsiveness of the Berg Balance Scale (BBS) and that of its short form (SFBBS) at both the individual person level and the group level. Design: A repeated-measurements design. Setting: Hospital and home setting. Participants: Patients with stroke. Results: Totals of 226, 202, and 168 patients with stroke were assessed with the BBS at 14, 30, and 90 days after stroke, respectively. The SFBBS data were extracted from the patients’ responses on the BBS. At the group level, the BBS and the SFBBS had sufficient and similar responsiveness. For the Rasch scores, the effect sizes of the three change scores for the BBS and the SFBBS, respectively, had similar ranges between 0.38 and 0.88 and between 0.39 and 0.85, respectively. The standardized response means of the three change scores for the BBS and the SFBBS ranged from 0.74 to 1.33 and from 0.72 to 1.13, respectively. At the individual person level, the BBS detected significant balance improvement in about twice as many patients as the SFBBS detected. Conclusion: The responsiveness of the BBS at the individual person level was better than that of the SFBBS in patients with stroke. The BBS is recommended as an outcome measure to better detect changes in individual patients.

Improving the Control of Type I Error Rate in Assessing Differential Item Functioning for Hierarchical Generalized Linear Model When Impact Is Presented

Hierarchical generalized linear models (HGLMs) have been used to assess differential item functioning (DIF). For model identification, some literature assumed that the reference (majority) and focal (minority) groups have an equal mean ability so that all items in a test can be assessed for DIF. In reality, it is very unlikely that the two groups have an identical mean. If so, other model identification procedures should be adopted. A feasible procedure for model identification is to set an item that is the most likely to be DIF-free as a reference, so that the two groups can have different means and the other items can be assessed for DIF. In Simulation Study 1, several methods based on HGLMs in selecting DIF-free items were compared. In Simulation Study 2, those items assessed as DIF-free were anchored, and the other items were assessed for DIF. This new method was compared with the traditional method based on HGLMs in which the two groups are assumed to have an equal mean in terms of the Type I error rate and the power rate. The results showed that the new method outperformed the traditional method when the two groups did not have an equal mean.

Evaluating the European Health Literacy Survey Questionnaire in Patients with Stroke: A Latent Trait Analysis Using Rasch Modeling

BackgroundThe effective self-management and treatment of long-term disability after stroke depends greatly on the health literacy of patients. The European Health Literacy Survey Questionnaire (HLS-EU-Q) is a comprehensive and theory-based measure that captures multiple self-perceived competencies of health literacy and covers a diverse range of health contexts. However, there is no psychometric evidence on the HLS-EU-Q in the stroke population.ObjectiveThe aim of this study was to examine the validity of the HLS-EU-Q in patients with stroke using Rasch analysis.MethodsWe compared the model deviance among the one-domain, three-domain, four-domain, and 12-domain structures using likelihood ratio tests to determine the dimensionality of the HLS-EU-Q. Thereafter, we examined the unidimensionality of each domain, local independence, item fit, response categories, and differential item functioning (DIF) for the best fitting structure.ResultsA total of 311 patients with stroke participated in this study. Rasch analysis revealed that the 12-domain HLS-EU-Q demonstrated the best data–model fit. The original 4-point scales showed disordering, which can be corrected by rescaling them as 3-point scales with the two middle categories collapsed. All 47 items in the rescaled HLS-EU-Q fit the 12-domain Rasch model, demonstrated local independence, assessed the 12 unidimensional domains respectively, and had invariant difficulties between different age or education groups of the patients with stroke.ConclusionWe recommend using the 12-domain scores of the rescaled HLS-EU-Q to comprehensively and accurately capture the competencies to access, understand, appraise, and apply health information within the three health contexts of healthcare, disease prevention, and health promotion for patients with stroke.

Validation of the Integrated Model of Health Literacy in Patients With Breast Cancer

Background Health literacy (HL) enables patients with breast cancer to actively participate in health decisions and promote positive health outcomes. The Integrated Model of Health Literacy (IMHL), defined as the personal, situational, and societal/environmental factors that predict the level of HL that can influence health outcomes, incorporates the concepts, determinants, and consequences of HL. Objective The aim of this study was to examine the mechanisms and completeness of the IMHL in patients with breast cancer. Methods Five hundred eleven Taiwanese patients were prospectively recruited. We conducted structural equation modeling to confirm and modify the predictive pathways linking the HL-related factors in the IMHL. Results Results on a total of 511 breast cancer patients showed good model-data fit. An alternative model revealed better fit with 2 pathways added from cancer stage to self-rated health and from cancer duration to shared decision making. Both the original model and alternative model modification revealed that only personal determinants (age, education, cancer stage, and duration) and not situational determinants (marital status) or social/environmental determinants (residence and occupation) could significantly predict the 3 domains of HL. Theorized consequences of HL were significantly influenced by HL in both models. Conclusions Our results partially support the relationships proposed in the IMHL for patients with breast cancer as only personal determinants significantly predicted HL. Implications for Practice Understanding the predictive pathways of the integrated HL model could help clinicians to tailor HL interventions using a patient’s personal determinants to facilitate participation in decision making and promote health for breast cancer patients.

Validation of the short-form Health Literacy Scale in patients with stroke

OBJECTIVE We aimed to validate a Mandarin version of the short-form Health Literacy Scale (SHEAL) in patients with stroke. METHODS Each patient with stroke was interviewed with the SHEAL. The Public Stroke Knowledge Quiz (PSKQ) was administered as a criterion for examining the convergent validity of the SHEAL. The discriminative validity of the SHEAL was determined with age and education level as independent grouping variables. RESULTS A total of 87 patients with stroke volunteered to participate in this prospective study. The SHEAL demonstrated sufficient internal consistency reliability (alpha=0.82) and high correlation with the PSKQ (r=0.62). The SHEAL scores between different age groups and education level groups were significantly different. The SHEAL, however, showed a notable ceiling effect (24.1% of the participants), indicating that the SHEAL cannot differentiate level of health literacy between individuals with high health literacy. CONCLUSION The internal consistency reliability, convergent validity, and discriminative validity of the SHEAL were adequate. However, the internal consistency reliability and ceiling effect of the SHEAL need to be improved. PRACTICE IMPLICATIONS The SHEAL has shown its potential for assessing the health literacy of patients with stroke for research purposes. For clinical usage, however, the SHEAL should be used with caution.

Item-saving assessment of self-care performance in children with developmental disabilities: A prospective caregiver-report computerized adaptive test

Objective The purpose of this study was to construct a computerized adaptive test (CAT) for measuring self-care performance (the CAT-SC) in children with developmental disabilities (DD) aged from 6 months to 12 years in a content-inclusive, precise, and efficient fashion. Methods The study was divided into 3 phases: (1) item bank development, (2) item testing, and (3) a simulation study to determine the stopping rules for the administration of the CAT-SC. A total of 215 caregivers of children with DD were interviewed with the 73-item CAT-SC item bank. An item response theory model was adopted for examining the construct validity to estimate item parameters after investigation of the unidimensionality, equality of slope parameters, item fitness, and differential item functioning (DIF). In the last phase, the reliability and concurrent validity of the CAT-SC were evaluated. Results The final CAT-SC item bank contained 56 items. The stopping rules suggested were (a) reliability coefficient greater than 0.9 or (b) 14 items administered. The results of simulation also showed that 85% of the estimated self-care performance scores would reach a reliability higher than 0.9 with a mean test length of 8.5 items, and the mean reliability for the rest was 0.86. Administering the CAT-SC could reduce the number of items administered by 75% to 84%. In addition, self-care performances estimated by the CAT-SC and the full item bank were very similar to each other (Pearson r = 0.98). Conclusion The newly developed CAT-SC can efficiently measure self-care performance in children with DD whose performances are comparable to those of TD children aged from 6 months to 12 years as precisely as the whole item bank. The item bank of the CAT-SC has good reliability and a unidimensional self-care construct, and the CAT can estimate self-care performance with less than 25% of the items in the item bank. Therefore, the CAT-SC could be useful for measuring self-care performance in children with DD in clinical and research settings.

Improving the Assessment of Differential Item Functioning in Large-Scale Programs With Dual-Scale Purification of Rasch Models: The PISA Example

By design, large-scale educational testing programs often have a large proportion of missing data. Since the effect of missing data on differential item functioning (DIF) assessment has been investigated in recent years and it has been found that Type I error rates tend to be inflated, it is of great importance to adapt existing DIF assessment methods to the inflation. The DIF-free-then-DIF (DFTD) strategy, which originally involved one single-scale purification procedure to identify DIF-free items, has been extended to involve another scale purification procedure for the DIF assessment in this study, and this new method is called the dual-scale purification (DSP) procedure. The performance of the DSP procedure in assessing DIF in large-scale programs, such as Program for International Student Assessment (PISA), was compared with the DFTD strategy through a series of simulation studies. Results showed the superiority of the DSP procedure over the DFTD strategy when tests consisted of many DIF items and when data were missing by design as in large-scale programs. Moreover, an empirical study of the PISA 2009 Taiwan sample was provided to show the implications of the DSP procedure. The applications as well as further studies of DSP procedure are also discussed.

Theory of Mind Deficit Is Associated with Pretend Play Performance, but Not Playfulness, in Children with Autism Spectrum Disorder

Objective/Background This study aimed to examine the relationships of theory of mind (ToM) to both pretend play and playfulness in children with autism spectrum disorder (ASD). Methods Twenty children with ASD aged between 3 years and 7.11 years were assessed with the ToM test, and then placed in a free play condition and a pretend play condition to assess pretend play and playfulness with the Child-Initiated Pretend Play Assessment and Test of Playfulness, respectively. In addition, the childrens symptom severities of ASD and verbal abilities were also assessed with the Childhood Autism Rating Scale (CARS) and Receptive and Expressive Vocabulary Test—second edition, respectively. Results The results of the regression analysis confirmed that ToM significantly predicted pretend play variables, namely, Number of Object Substitutions (R2 = .158, p = .002) and Number of Imitated Actions (R2 = .175, p = .001), but not playfulness. The CARS score was a significant predictor of the Percentage of Elaborate Pretend Play Actions of pretend play (R2 = .075, p = .034), as well as the internal control (R2 = .125, p = .006) and framing (R2 = .071, p = .039) variables of playfulness. Conclusion The findings support the idea that children with ASD who have better ToM might be able to develop better pretend play, but not better playfulness, which might be more strongly related to their autistic severity.