Jin Shei Lai

The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008

OBJECTIVESnPatient-reported outcomes (PROs) are essential when evaluating many new treatments in health care; yet, current measures have been limited by a lack of precision, standardization, and comparability of scores across studies and diseases. The Patient-Reported Outcomes Measurement Information System (PROMIS) provides item banks that offer the potential for efficient (minimizes item number without compromising reliability), flexible (enables optional use of interchangeable items), and precise (has minimal error in estimate) measurement of commonly studied PROs. We report results from the first large-scale testing of PROMIS items.nnnSTUDY DESIGN AND SETTINGnFourteen item pools were tested in the U.S. general population and clinical groups using an online panel and clinic recruitment. A scale-setting subsample was created reflecting demographics proportional to the 2000 U.S. census.nnnRESULTSnUsing item-response theory (graded response model), 11 item banks were calibrated on a sample of 21,133, measuring components of self-reported physical, mental, and social health, along with a 10-item Global Health Scale. Short forms from each bank were developed and compared with the overall bank and with other well-validated and widely accepted (legacy) measures. All item banks demonstrated good reliability across most of the score distributions. Construct validity was supported by moderate to strong correlations with legacy measures.nnnCONCLUSIONnPROMIS item banks and their short forms provide evidence that they are reliable and precise measures of generic symptoms and functional reports comparable to legacy instruments. Further testing will continue to validate and test PROMIS items and banks in diverse clinical populations.

Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS)

Background:The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. Objectives:Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. Analyses:Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. Recommendations:Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.

Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales.

Magnitude differences in scores on a measure of quality of life that correspond to differences in function or clinical course are called clinically important differences (CIDs). Anchor-based and distribution-based methods were used to provide ranges of CIDs for five targeted scale scores of the Functional Assessment of Cancer Therapy-Anemia (FACT-An) questionnaire. Three samples of cancer patients were used: Sample 1 included 50 patients participating in a validation study of the FACT-An; Sample 2 included 131 patients participating in a longitudinal study of chemotherapy-induced fatigue; sample 3 included 2,402 patients enrolled in a community-based clinical trial evaluating the effectiveness and safety of a treatment for anemia. Three clinical indicators (hemoglobin level; performance status; response to treatment) were used to determine anchor-based differences. One-half of the standard deviation and 1 standard error of measurement were used as distribution-based criteria. Analyses supported the following whole number estimates of a minimal CID for these five targeted scores: Fatigue Scale = 3.0; FACT-G total score = 4.0; FACT-An total score = 7.0; Trial Outcome Index-Fatigue = 5.0; and Trial Outcome Index-Anemia = 6.0. These estimates provide a basis for sample size estimation when planning for a clinical trial or other longitudinal study, when the purpose is to ensure detection of meaningful change over time. They can also be used in conjunction with more traditional clinical markers to assist investigators in determining treatment efficacy.

Fatigue in cancer patients compared with fatigue in the general United States population

Although fatigue is a common symptom among cancer patients, it is also a common experience in the general, healthy population. Its universality has made it difficult to appreciate whether the fatigue experienced by patients with cancer is distinguishable from the fatigue experienced by the general population. Because the etiology of fatigue is multifactorial, it also has been difficult to appreciate fully the relative contribution of anemia to cancer‐related fatigue.

The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment

The use of item banks and computerized adaptive testing (CAT) begins with clear definitions of important outcomes, and references those definitions to specific questions gathered into large and well-studied pools, or “banks” of items. Items can be selected from the bank to form customized short scales, or can be administered in a sequence and length determined by a computer programmed for precision and clinical relevance. Although far from perfect, such item banks can form a common definition and understanding of human symptoms and functional problems such as fatigue, pain, depression, mobility, social function, sensory function, and many other health concepts that we can only measure by asking people directly. The support of the National Institutes of Health (NIH), as witnessed by its cooperative agreement with measurement experts through the NIH Roadmap Initiative known as PROMIS (www.nihpromis.org), is a big step in that direction. Our approach to item banking and CAT is practical; as focused on application as it is on science or theory. From a practical perspective, we frequently must decide whether to re-write and retest an item, add more items to fill gaps (often at the ceiling of the measure), re-test a bank after some modifications, or split up a bank into units that are more unidimensional, yet less clinically relevant or complete. These decisions are not easy, and yet they are rarely unforgiving. We encourage people to build practical tools that are capable of producing multiple short form measures and CAT administrations from common banks, and to further our understanding of these banks with various clinical populations and ages, so that with time the scores that emerge from these many activities begin to have not only a common metric and range, but a shared meaning and understanding across users. In this paper, we provide an overview of item banking and CAT, discuss our approach to item banking and its byproducts, describe testing options, discuss an example of CAT for fatigue, and discuss models for long term sustainability of an entity such as PROMIS. Some barriers to success include limitations in the methods themselves, controversies and disagreements across approaches, and end-user reluctance to move away from the familiar.

Development of a PROMIS item bank to measure pain interference

&NA; This paper describes the psychometric properties of the PROMIS‐pain interference (PROMIS‐PI) bank. An initial candidate item pool (n = 644) was developed and evaluated based on the review of existing instruments, interviews with patients, and consultation with pain experts. From this pool, a candidate item bank of 56 items was selected and responses to the items were collected from large community and clinical samples. A total of 14,848 participants responded to all or a subset of candidate items. The responses were calibrated using an item response theory (IRT) model. A final 41‐item bank was evaluated with respect to IRT assumptions, model fit, differential item function (DIF), precision, and construct and concurrent validity. Items of the revised bank had good fit to the IRT model (CFI and NNFI/TLI ranged from 0.974 to 0.997), and the data were strongly unidimensional (e.g., ratio of first and second eigenvalue = 35). Nine items exhibited statistically significant DIF. However, adjusting for DIF had little practical impact on score estimates and the items were retained without modifying scoring. Scores provided substantial information across levels of pain; for scores in the T‐score range 50–80, the reliability was equivalent to 0.96–0.99. Patterns of correlations with other health outcomes supported the construct validity of the item bank. The scores discriminated among persons with different numbers of chronic conditions, disabling conditions, levels of self‐reported health, and pain intensity (p < 0.0001). The results indicated that the PROMIS‐PI items constitute a psychometrically sound bank. Computerized adaptive testing and short forms are available.

Standardizing patient-reported outcomes assessment in cancer clinical trials: a patient-reported outcomes measurement information system initiative.

Patient-reported outcomes (PROs), such as symptom scales or more broad-based health-related quality-of-life measures, play an important role in oncology clinical trials. They frequently are used to help evaluate cancer treatments, as well as for supportive and palliative oncology care. To be most beneficial, these PROs must be relevant to patients and clinicians, valid, and easily understood and interpreted. The Patient-Reported Outcomes Measurement Information System (PROMIS) Network, part of the National Institutes of Health Roadmap Initiative, aims to improve appreciably how PROs are selected and assessed in clinical research, including clinical trials. PROMIS is establishing a publicly available resource of standardized, accurate, and efficient PRO measures of major self-reported health domains (eg, pain, fatigue, emotional distress, physical function, social function) that are relevant across chronic illnesses including cancer. PROMIS is also developing measures of self-reported health domains specifically targeted to cancer, such as sleep/wake function, sexual function, cognitive function, and the psychosocial impacts of the illness experience (ie, stress response and coping; shifts in self-concept, social interactions, and spirituality). We outline the qualitative and quantitative methods by which PROMIS measures are being developed and adapted for use in clinical oncology research. At the core of this activity is the formation and application of item banks using item response theory modeling. We also present our work in the fatigue domain, including a short-form measure, as a sample of PROMIS methodology and work to date. Plans for future validation and application of PROMIS measures are discussed.

Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale.

Fatigue is a common symptom among cancer patients and the general population. Due to its subjective nature, fatigue has been difficult to effectively and efficiently assess. Modern computerized adaptive testing (CAT) can enable precise assessment of fatigue using a small number of items from a fatigue item bank. CAT enables brief assessment by selecting questions from an item bank that provide the maximum amount of information given a persons previous responses. This article illustrates steps to prepare such an item bank, using 13 items from the Functional Assessment of Chronic Illness Therapy Fatigue Subscale (FACIT-F) as the basis. Samples included 1022 cancer patients and 1010 people from the general population. An Item Response Theory (IRT)-based rating scale model, a polytomous extension of the Rasch dichotomous model was utilized. Nine items demonstrating acceptable psychometric properties were selected and positioned on the fatigue continuum. The fatigue levels measured by these nine items along with their response categories covered 66.8% of the general population and 82.6% of the cancer patients. Although the operational CAT algorithms to handle polytomously scored items are still in progress, we illustrated how CAT may work by using nine core items to measure level of fatigue. Using this illustration, a fatigue measure comparable to its full-length 13-item scale administration was obtained using four items. The resulting item bank can serve as a core to which will be added a psychometrically sound and operational item bank covering the entire fatigue continuum.

PROMIS Pediatric Pain Interference Scale: An Item Response Theory Analysis of the Pediatric Pain Item Bank

UNLABELLEDnAn aim of the National Institutes of Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS) initiative is to develop item banks and computerized adaptive tests (CAT) that are applicable across a wide variety of chronic disorders. The PROMIS Pediatric Cooperative Group has concentrated on the development of pediatric self-report item banks for ages 8 through 17 years. The objective of the present study is to describe the Item Response Theory (IRT) analysis of the NIH PROMIS pediatric pain item bank and the measurement properties of the new unidimensional PROMIS Pediatric Pain Interference Scale. Test forms containing pediatric pain items were completed by a total of 3048 respondents. IRT analyses regarding scale dimensionality, item local dependence, and differential item functioning were conducted. A pain item pool was developed to yield scores on a T-score scale with a mean of 50 and standard deviation of 10. The recommended 8-item unidimensional short form for the PROMIS Pediatric Pain Interference Scale contains the item set which provides the maximum test information at the mean (50) on the T-score metric. A simulated CAT was computed that provides the most information at 5 possible score locations (30, 40, 50, 60, and 70 on the T-score metric).nnnPERSPECTIVEnThe present study provides initial calibrations of the NIH PROMIS pediatric pain item bank and the creation of the PROMIS Pediatric Pain Interference Scale. It is anticipated that this new scale will have application in pediatric chronic and recurrent pain.

Factor analysis techniques for assessing sufficient unidimensionality of cancer related fatigue

Background: Fatigue is the most common unrelieved symptom experienced by people with cancer. The purpose of this study was to examine whether cancer-related fatigue (CRF) can be summarized using a single score, that is, whether CRF is sufficiently unidimensional for measurement approaches that require or assume unidimensionality. We evaluated this question using factor analysis techniques including the theory-driven bi-factor model. Methods: Five hundred and fifty five cancer patients from the Chicago metropolitan area completed a 72-item fatigue item bank, covering a range of fatigue-related concerns including intensity, frequency and interference with physical, mental, and social activities. Dimensionality was assessed using exploratory and confirmatory factor analysis (CFA) techniques. Results: Exploratory factor analysis (EFA) techniques identified from 1 to 17 factors. The bi-factor model suggested that CRF was sufficiently unidimensional. Conclusions: CRF can be considered sufficiently unidimensional for applications that require unidimensionality. One such application, item response theory (IRT), will facilitate the development of short-form and computer-adaptive testing. This may further enable practical and accurate clinical assessment of CRF.