Maria Orlando Edelen

Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement

BackgroundHealth outcomes researchers are increasingly applying Item Response Theory (IRT) methods to questionnaire development, evaluation, and refinement efforts.ObjectiveTo provide a brief overview of IRT, to review some of the critical issues associated with IRT applications, and to demonstrate the basic features of IRT with an example.MethodsExample data come from 6,504 adolescent respondents in the National Longitudinal Study of Adolescent Health public use data set who completed to the 19-item Feelings Scale for depression. The sample was split into a development and validation sample. Scale items were calibrated in the development sample with the Graded Response Model and the results were used to construct a 10-item short form. The short form was evaluated in the validation sample by examining the correspondence between IRT scores from the short form and the original, and by comparing the proportion of respondents identified as depressed according to the original and short form observed cut scores.ResultsThe 19 items varied in their discrimination (slope parameter range: .86–2.66), and item location parameters reflected a considerable range of depression (−.72–3.39). However, the item set is most discriminating at higher levels of depression. In the validation sample IRT scores generated from the short and long forms were correlated at .96 and the average difference in these scores was −.01. In addition, nearly 90% of the sample was classified identically as at risk or not at risk for depression using observed score cut points from the short and long forms.ConclusionsWhen used appropriately, IRT can be a powerful tool for questionnaire development, evaluation, and refinement, resulting in precise, valid, and relatively brief instruments that minimize response burden.

Identification of differential item functioning using item response theory and the likelihood-based model comparison approach. Application to the Mini-Mental State Examination.

Background:An important part of examining the adequacy of measures for use in ethnically diverse populations is the evaluation of differential item functioning (DIF) among subpopulations such as those administered the measure in different languages. A number of methods exist for this purpose. Objective:The objective of this study was to introduce and demonstrate the identification of DIF using item response theory (IRT) and the likelihood-based model comparison approach. Methods:Data come from a sample of community-residing elderly who were part of a dementia case registry. A total of 1578 participants were administered either an English (n = 913) or Spanish (n = 665) version of the 21-item Mini-Mental State Examination. IRT was used to identify language DIF in these items with the likelihood-based model comparison approach. Results:Fourteen of the 21 items exhibited significant DIF according to language of administration. However, because the direction of the identified DIF was not consistent for one language version over the other, the impact at the scale level was negligible. Conclusions:IRT and the likelihood-based model comparison approach comprise a powerful tool for DIF detection that can aid in the development, refinement, and evaluation of measures for use in ethnically diverse populations.

Psychosocial Mediators of Antiretroviral Nonadherence in HIV-positive Adults with Substance Use and Mental Health Problems

This study used data from 1,889 HIV-positive patients on antiretroviral (ARV) medications who participated in the HIV Cost and Services Utilization Study to investigate whether nonadherence to ARV medications among patients with mental health and substance use problems could be explained by difficulty getting and negative attitudes toward ARV medications, poor fit of the regimen with lifestyle, lack of instruction and cues for remembering the regimen from a health care provider, and poor support from others for taking ARV medications. Difficulty getting ARV medications and poor fit with lifestyle were significant mediators of nonadherence for patients with a probable psychiatric disorder. Difficulty getting medication was a mediator for heavy drinkers, and poor fit with lifestyle was a mediator for drug users who drank heavily. Further research is needed to identify and address the barriers to adherence in these populations.

A Comparison of Interview and Self-Report Methods for the Assessment of Borderline Personality Disorder Criteria

Interview methods are widely regarded as the standard for the diagnosis of borderline personality disorder (BPD), whereas self-report methods are considered a time-efficient alternative. However, the relative validity of these methods has not been sufficiently tested. The current study used data from the Collaborative Longitudinal Personality disorder Study to compare diagnostic base rates and the relative validity of interview and self-report methods for assessing functional outcome in BPD. Although self-report yielded higher base rates of criteria endorsement, results did not support the common assumption that diagnostic interviews are more valid than self-reports, but instead indicated the combined use of these methods optimally identifies BPD criteria.

Personality traits as prospective predictors of suicide attempts

Objective:  To examine higher order personality factors of negative affectivity (NA) and disinhibition (DIS), as well as lower order facets of impulsivity, as prospective predictors of suicide attempts in a predominantly personality disordered sample.

Nursing Home Assessment of Cognitive Impairment: Development and Testing of a Brief Instrument of Mental Status

OBJECTIVES: To test the accuracy of a brief cognitive assessment of nursing home (NH) residents and to determine whether facility nurses can reliably perform this assessment.

The association of personality disorders with the prospective 7-year course of anxiety disorders

BACKGROUND This study prospectively examined the natural clinical course of six anxiety disorders over 7 years of follow-up in individuals with personality disorders (PDs) and/or major depressive disorder. Rates of remission, relapse, new episode onset and chronicity of anxiety disorders were examined for specific associations with PDs. METHOD Participants were 499 patients with anxiety disorders in the Collaborative Longitudinal Personality Disorders Study, who were assessed with structured interviews for psychiatric disorders at yearly intervals throughout 7 years of follow-up. These data were used to determine probabilities of changes in disorder status for social phobia (SP), generalized anxiety disorder (GAD), obsessive-compulsive disorder (OCD), post-traumatic stress disorder (PTSD), panic disorder and panic disorder with agoraphobia. RESULTS Estimated remission rates for anxiety disorders in this study group ranged from 73% to 94%. For those patients who remitted from an anxiety disorder, relapse rates ranged from 34% to 67%. Rates for new episode onsets of anxiety disorders ranged from 3% to 17%. Specific PDs demonstrated associations with remission, relapse, new episode onsets and chronicity of anxiety disorders. Associations were identified between schizotypal PD with course of SP, PTSD and GAD; avoidant PD with course of SP and OCD; obsessive-compulsive PD with course of GAD, OCD, and agoraphobia; and borderline PD with course of OCD, GAD and panic with agoraphobia. CONCLUSIONS Findings suggest that specific PD diagnoses have negative prognostic significance for the course of anxiety disorders underscoring the importance of assessing and considering PD diagnoses in patients with anxiety disorders.

Running Away From Home: A Longitudinal Study of Adolescent Risk Factors and Young Adult Outcomes

Little is known about the adolescent risk factors and young adult health-related outcomes associated with running away from home. We examined these correlates of running away using longitudinal data from 4,329 youth (48% female, 85% white) who were followed from Grade 9 to age 21. Nearly 14% of the sample reported running away in the past year at Grade 10 and/or Grade 11. Controlling for demographics and general delinquency, running away from home was predicted by lack of parental support, school disengagement, greater depressive affect, and heavier substance use at Grade 9. In turn, runaways had higher drug dependence scores and more depressive symptoms at age 21 than non-runaways, even after taking these antecedent risk factors into account. Runaway status did not predict alcohol dependence risk at age 21. Results highlight the importance of substance use and depression, both as factors propelling adolescents to run away and as important long-term consequences of running away.

The relationship between sleep disturbance and the course of anxiety disorders in primary care patients

This study examined the relationship between sleep disturbance and the course of anxiety disorders in primary care patients. Participants were part of the Primary Care Anxiety Project (PCAP), a naturalistic, longitudinal study of anxiety disorders in primary care. Participants completed an intake evaluation and follow-up assessments at 6 months, 12 months, and annually thereafter. Only participants with sleep data at intake were included in the current study (n=533). The majority (74%) reported experiencing sleep disturbance at intake. Those with a diagnosis of generalized anxiety disorder (GAD) or post-traumatic stress disorder (PTSD) were over 2 times more likely to have sleep problems. Sleep disturbance at intake did not relate to the longitudinal course of GAD, social phobia, panic disorder, or obsessive-compulsive disorder. However, it did predict the course of PTSD, controlling for comorbid major depressive disorder (MDD) and having more than one anxiety disorder diagnosis intake, those with sleep disturbance at intake being less likely to remit from PTSD in the 5 years of follow-up. By year 5, only 34% of those with sleep problems at intake remitted from PTSD whereas 56% of those without sleep disturbance remitted from the disorder. The findings suggest that sleep disturbance in PTSD may have prognostic significance and may be important to address in clinical interventions.

Temporal associations of cigarette smoking with social influences, academic performance, and delinquency: a four-wave longitudinal study from ages 13-23.

This study examined the temporal associations of cigarette smoking with prosmoking social influences, academic performance, and delinquency in a cohort of 6,527 adolescents surveyed at ages 13, 16, 18, and 23 years. Prosmoking peer and family influences were risk factors for future smoking throughout adolescence, with family influences perhaps also operating indirectly through the adolescents exposure to prosmoking peers. There were reciprocal associations of youth smoking with parental approval, peer smoking, and poor grades (but not delinquency), with youth smoking emerging as a stronger antecedent than consequence of these psychosocial factors. Few gender differences in these associations were observed. Implications of these findings for efforts to prevent youth smoking are discussed.