Nathan E. Dodds

Development of Short Forms From the PROMIS™ Sleep Disturbance and Sleep-Related Impairment Item Banks

This article reports on the development of short forms from the Patient-Reported Outcomes Measurement Information System (PROMIS™) Sleep Disturbance (SD) and Sleep-Related Impairment (SRI) item banks. Results from post-hoc computerized adaptive testing (CAT) simulations, item discrimination parameters, item means, and clinical judgments were used to select the best-performing 8 items for SD and SRI. The final 8-item short forms provided less test information than the corresponding full banks, but correlated strongly with the longer forms. The short forms had greater measurement precision than the Pittsburgh Sleep Quality Index (PSQI) and the Epworth Sleepiness Scale (ESS), as indicated by larger test information values across the continuum of severity, despite having fewer total items—a major advantage for both research and clinical settings.

Item banks for alcohol use from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Use, consequences, and expectancies

BACKGROUND We report on the development and calibration of item banks for alcohol use, negative and positive consequences of alcohol use, and negative and positive expectancies regarding drinking as part of the Patient-Reported Outcomes Measurement Information System (PROMIS). METHODS Comprehensive literature searches yielded an initial bank of more than 5000 items from over 200 instruments. After qualitative item analysis (including focus groups and cognitive interviewing), 141 items were included in field testing. Items for alcohol use and consequences were written in a first-person, past-tense format with a 30-day time frame and 5 response options reflecting frequency. Items for expectancies were written in a third-person, present-tense format with no time frame specified and 5 response options reflecting intensity. The calibration sample included 1407 respondents, 1000 from the general population (ascertained through an internet panel) and 407 from community treatment programs participating in the National Institute on Drug Abuse (NIDA) Clinical Trials Network (CTN). RESULTS Final banks of 37, 31, 20, 11, and 9 items (108 total items) were calibrated for alcohol use, negative consequences, positive consequences, negative expectancies, and positive expectancies, respectively, using item response theory (IRT). Seven-item static short forms were also developed from each item bank. CONCLUSIONS Test information curves showed that the PROMIS item banks provided substantial information in a broad range of severity, making them suitable for treatment, observational, and epidemiological research.

Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting

PurposeOur work as a primary research site of the Patient-Reported Outcomes Measurement Information System (PROMIS®), combined with support from the Patient-Centered Outcomes Research Institute, allowed us to evaluate the real-world applicability and acceptability of PROMIS measures in an addiction medicine setting.MethodsAs part of a 3-month prospective observational study, 225 outpatients at a substance abuse treatment clinic completed PROMIS item banks for alcohol use (as well as 15 additional item banks from 8 other PROMIS domains, including emotional distress, sleep, and pain), with assessments at intake, 1-month follow-up, and 3-month follow-up. A subsample of therapists and their patients completed health domain importance ratings and qualitative interviews to elicit feedback regarding the content and format of the patients’ assessment results.ResultsThe importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the non-alcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Based on their suggestions, changes were made to item response feedback to facilitate comprehension and communication.ConclusionsBoth therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The results of our pilot work demonstrated the value and practicality of incorporating a comprehensive health assessment within a substance abuse treatment setting.

Measuring nonspecific factors in treatment: item banks that assess the healthcare experience and attitudes from the patient’s perspective

PurposeNonspecific factors that accompany healthcare treatments, such as patients’ attitudes and expectations, are important parts of the experience of care and can influence outcomes. However, no precise, concise, and generalizable instruments to measure these factors exist. We report on the development and calibration of new item banks, titled the Healing Encounters and Attitudes Lists (HEAL), that assess nonspecific factors across a broad range of treatments and conditions.MethodsThe instrument development methodology of the Patient-Reported Outcomes Measurement Information System (PROMIS®) was used. Patient focus groups and clinician interviews informed our HEAL conceptual model. Literature searches of eight databases yielded over 500 instruments and resulted in an initial item pool of several thousand items. After qualitative item analysis, including cognitive interviewing, 296 items were included in field testing. The calibration sample included 1657 respondents, 1400 obtained through an Internet panel and 257 from conventional and integrative medicine clinics. Following exploratory and confirmatory factor analyses, the HEAL item banks were calibrated using item response theory.ResultsThe final HEAL item banks were Patient–Provider Connection (57 items), Healthcare Environment (25 items), Treatment Expectancy (27 items), Positive Outlook (27 items), and Spirituality (26 items). Short forms were also developed from each item bank. A six-item short form, Attitudes toward Complementary and Alternative Medicine (CAM), was also created.ConclusionsHEAL item banks provided substantial information across a broad range of each construct. HEAL item banks showed initial evidence of predictive and concurrent validity, suggesting that they are suitable for measuring nonspecific factors in treatment.

Item banks for substance use from the Patient-Reported Outcomes Measurement Information System (PROMIS®): Severity of use and positive appeal of use☆

BACKGROUND Two item banks for substance use were developed as part of the Patient-Reported Outcomes Measurement Information System (PROMIS(®)): severity of substance use and positive appeal of substance use. METHODS Qualitative item analysis (including focus groups, cognitive interviewing, expert review, and item revision) reduced an initial pool of more than 5300 items for substance use to 119 items included in field testing. Items were written in a first-person, past-tense format, with 5 response options reflecting frequency or severity. Both 30-day and 3-month time frames were tested. The calibration sample of 1336 respondents included 875 individuals from the general population (ascertained through an internet panel) and 461 patients from addiction treatment centers participating in the National Drug Abuse Treatment Clinical Trials Network. RESULTS Final banks of 37 and 18 items were calibrated for severity of substance use and positive appeal of substance use, respectively, using the two-parameter graded response model from item response theory (IRT). Initial calibrations were similar for the 30-day and 3-month time frames, and final calibrations used data combined across the time frames, making the items applicable with either interval. Seven-item static short forms were also developed from each item bank. CONCLUSIONS Test information curves showed that the PROMIS item banks provided substantial information in a broad range of severity, making them suitable for treatment, observational, and epidemiological research in both clinical and community settings.

Individuals with knee impairments identify items in need of clarification in the Patient Reported Outcomes Measurement Information System (PROMIS®) pain interference and physical function item banks – a qualitative study

BackgroundThe content and wording of the Patient Reported Outcome Measurement Information System (PROMIS) Physical Function and Pain Interference item banks have not been qualitatively assessed by individuals with knee joint impairments. The purpose of this investigation was to identify items in the PROMIS Physical Function and Pain Interference Item Banks that are irrelevant, unclear, or otherwise difficult to respond to for individuals with impairment of the knee and to suggest modifications based on cognitive interviews.MethodsTwenty-nine individuals with knee joint impairments qualitatively assessed items in the Pain Interference and Physical Function Item Banks in a mixed-methods cognitive interview. Field notes were analyzed to identify themes and frequency counts were calculated to identify items not relevant to individuals with knee joint impairments.ResultsIssues with clarity were identified in 23 items in the Physical Function Item Bank, resulting in the creation of 43 new or modified items, typically changing words within the item to be clearer. Interpretation issues included whether or not the knee joint played a significant role in overall health and age/gender differences in items. One quarter of the original items (31 of 124) in the Physical Function Item Bank were identified as irrelevant to the knee joint. All 41 items in the Pain Interference Item Bank were identified as clear, although individuals without significant pain substituted other symptoms which interfered with their life.ConclusionsThe Physical Function Item Bank would benefit from additional items that are relevant to individuals with knee joint impairments and, by extension, to other lower extremity impairments. Several issues in clarity were identified that are likely to be present in other patient cohorts as well.

An item bank for abuse of prescription pain medication from the Patient-Reported Outcomes Measurement Information System (PROMIS®)

Objective There is a need to monitor patients receiving prescription opioids to detect possible signs of abuse. To address this need, we developed and calibrated an item bank for severity of abuse of prescription pain medication as part of the Patient-Reported Outcomes Measurement Information System (PROMIS ® ). Methods Comprehensive literature searches yielded an initial bank of 5,310 items relevant to substance use and abuse, including abuse of prescription pain medication, from over 80 unique instruments. After qualitative item analysis (i.e., focus groups, cognitive interviewing, expert review, and item revision), 25 items for abuse of prescribed pain medication were included in field testing. Items were written in a first-person, past-tense format, with a three-month time frame and five response options reflecting frequency or severity. The calibration sample included 448 respondents, 367 from the general population (ascertained through an internet panel) and 81 from community treatment programs participating in the National Drug Abuse Treatment Clinical Trials Network. Results A final bank of 22 items was calibrated using the two-parameter graded response model from item response theory. A seven-item static short form was also developed. The test information curve showed that the PROMIS ® item bank for abuse of prescription pain medication provided substantial information in a broad range of severity. Conclusion The initial psychometric characteristics of the item bank support its use as a computerized adaptive test or short form, with either version providing a brief, precise, and efficient measure relevant to both clinical and community samples.

Validation of the alcohol use item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS®)

BACKGROUND The Patient-Reported Outcomes Measurement Information System (PROMIS) includes five item banks for alcohol use. There are limited data, however, regarding their validity (e.g., convergent validity, responsiveness to change). To provide such data, we conducted a prospective study with 225 outpatients being treated for substance abuse. METHODS Assessments were completed shortly after intake and at 1-month and 3-month follow-ups. The alcohol item banks were administered as computerized adaptive tests (CATs). Fourteen CATs and one six-item short form were also administered from eight other PROMIS domains to generate a comprehensive health status profile. After modeling treatment outcome for the sample as a whole, correlates of outcome from the PROMIS health status profile were examined. RESULTS For convergent validity, the largest correlation emerged between the PROMIS alcohol use score and the Alcohol Use Disorders Identification Test (r=.79 at intake). Regarding treatment outcome, there were modest changes across the target problem of alcohol use and other domains of the PROMIS health status profile. However, significant heterogeneity was found in initial severity of drinking and in rates of change for both abstinence and severity of drinking during follow-up. This heterogeneity was associated with demographic (e.g., gender) and health-profile (e.g., emotional support, social participation) variables. CONCLUSIONS The results demonstrated the validity of PROMIS CATs, which require only 4-6 items in each domain. This efficiency makes it feasible to use a comprehensive health status profile within the substance use treatment setting, providing important prognostic information regarding abstinence and severity of drinking.

Using peer review to improve research and promote collaboration.

ObjectiveThe declining success rate of National Institutes of Health (NIH) grant applications highlights the need for interdisciplinary work within a large, diverse department to improve chances of federal funding success. The authors demonstrate how systematic peer review promotes two goals: enhancing the quality of research proposals and cultivating a collaborative departmental culture.MethodsChanges to the Research Review Committee (RRC) in the Department of Psychiatry at the University of Pittsburgh were instituted to accommodate the increasingly interdisciplinary nature of grant applications, integrate revisions to NIH grant application processes, and incorporate advances in computer technology.ResultsThe internal peer review process is associated with success in obtaining research support and with significant levels of collaborative scientific work reflected in both grant applications and peer-reviewed publications.ConclusionsA rich collaborative environment promoted through a rigorous internal peer review system has many benefits for both the quality of scholarly work and the collegiality of the research environment.