Bryce B. Reeve

The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years.

Background:The National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) Roadmap initiative (www.nihpromis.org) is a 5-year cooperative group program of research designed to develop, validate, and standardize item banks to measure patient-reported outcomes (PROs) relevant across common medical conditions. In this article, we will summarize the organization and scientific activity of the PROMIS network during its first 2 years. Design:The network consists of 6 primary research sites (PRSs), a statistical coordinating center (SCC), and NIH research scientists. Governed by a steering committee, the network is organized into functional subcommittees and working groups. In the first year, we created an item library and activated 3 interacting protocols: Domain Mapping, Archival Data Analysis, and Qualitative Item Review (QIR). In the second year, we developed and initiated testing of item banks covering 5 broad domains of self-reported health. Results:The domain mapping process is built on the World Health Organization (WHO) framework of physical, mental, and social health. From this framework, pain, fatigue, emotional distress, physical functioning, social role participation, and global health perceptions were selected for the first wave of testing. Item response theory (IRT)-based analysis of 11 large datasets supplemented and informed item-level qualitative review of nearly 7000 items from available PRO measures in the item library. Items were selected for rewriting or creation with further detailed review before the first round of testing in the general population and target patient populations. Conclusions:The NIH PROMIS network derived a consensus-based framework for self-reported health, systematically reviewed available instruments and datasets that address the initial PROMIS domains. Qualitative item research led to the first wave of network testing which began in the second year.

Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS)

Background:The construction and evaluation of item banks to measure unidimensional constructs of health-related quality of life (HRQOL) is a fundamental objective of the Patient-Reported Outcomes Measurement Information System (PROMIS) project. Objectives:Item banks will be used as the foundation for developing short-form instruments and enabling computerized adaptive testing. The PROMIS Steering Committee selected 5 HRQOL domains for initial focus: physical functioning, fatigue, pain, emotional distress, and social role participation. This report provides an overview of the methods used in the PROMIS item analyses and proposed calibration of item banks. Analyses:Analyses include evaluation of data quality (eg, logic and range checking, spread of response distribution within an item), descriptive statistics (eg, frequencies, means), item response theory model assumptions (unidimensionality, local independence, monotonicity), model fit, differential item functioning, and item calibration for banking. Recommendations:Summarized are key analytic issues; recommendations are provided for future evaluations of item banks in HRQOL assessment.

Intensity-Modulated Radiation Therapy, Proton Therapy, or Conformal Radiation Therapy and Morbidity and Disease Control in Localized Prostate Cancer

CONTEXT There has been rapid adoption of newer radiation treatments such as intensity-modulated radiation therapy (IMRT) and proton therapy despite greater cost and limited demonstrated benefit compared with previous technologies. OBJECTIVE To determine the comparative morbidity and disease control of IMRT, proton therapy, and conformal radiation therapy for primary prostate cancer treatment. DESIGN, SETTING, AND PATIENTS Population-based study using Surveillance, Epidemiology, and End Results-Medicare-linked data from 2000 through 2009 for patients with nonmetastatic prostate cancer. MAIN OUTCOME MEASURES Rates of gastrointestinal and urinary morbidity, erectile dysfunction, hip fractures, and additional cancer therapy. RESULTS Use of IMRT vs conformal radiation therapy increased from 0.15% in 2000 to 95.9% in 2008. In propensity score-adjusted analyses (N = 12,976), men who received IMRT vs conformal radiation therapy were less likely to receive a diagnosis of gastrointestinal morbidities (absolute risk, 13.4 vs 14.7 per 100 person-years; relative risk [RR], 0.91; 95% CI, 0.86-0.96) and hip fractures (absolute risk, 0.8 vs 1.0 per 100 person-years; RR, 0.78; 95% CI, 0.65-0.93) but more likely to receive a diagnosis of erectile dysfunction (absolute risk, 5.9 vs 5.3 per 100 person-years; RR, 1.12; 95% CI, 1.03-1.20). Intensity-modulated radiation therapy patients were less likely to receive additional cancer therapy (absolute risk, 2.5 vs 3.1 per 100 person-years; RR, 0.81; 95% CI, 0.73-0.89). In a propensity score-matched comparison between IMRT and proton therapy (n = 1368), IMRT patients had a lower rate of gastrointestinal morbidity (absolute risk, 12.2 vs 17.8 per 100 person-years; RR, 0.66; 95% CI, 0.55-0.79). There were no significant differences in rates of other morbidities or additional therapies between IMRT and proton therapy. CONCLUSIONS Among patients with nonmetastatic prostate cancer, the use of IMRT compared with conformal radiation therapy was associated with less gastrointestinal morbidity and fewer hip fractures but more erectile dysfunction; IMRT compared with proton therapy was associated with less gastrointestinal morbidity.

Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement

BackgroundHealth outcomes researchers are increasingly applying Item Response Theory (IRT) methods to questionnaire development, evaluation, and refinement efforts.ObjectiveTo provide a brief overview of IRT, to review some of the critical issues associated with IRT applications, and to demonstrate the basic features of IRT with an example.MethodsExample data come from 6,504 adolescent respondents in the National Longitudinal Study of Adolescent Health public use data set who completed to the 19-item Feelings Scale for depression. The sample was split into a development and validation sample. Scale items were calibrated in the development sample with the Graded Response Model and the results were used to construct a 10-item short form. The short form was evaluated in the validation sample by examining the correspondence between IRT scores from the short form and the original, and by comparing the proportion of respondents identified as depressed according to the original and short form observed cut scores.ResultsThe 19 items varied in their discrimination (slope parameter range: .86–2.66), and item location parameters reflected a considerable range of depression (−.72–3.39). However, the item set is most discriminating at higher levels of depression. In the validation sample IRT scores generated from the short and long forms were correlated at .96 and the average difference in these scores was −.01. In addition, nearly 90% of the sample was classified identically as at risk or not at risk for depression using observed score cut points from the short and long forms.ConclusionsWhen used appropriately, IRT can be a powerful tool for questionnaire development, evaluation, and refinement, resulting in precise, valid, and relatively brief instruments that minimize response burden.

Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations

PurposeWhile clinical care is frequently directed at making patients “feel better,” patients’ reports on their functioning and well-being (patient-reported outcomes [PROs]) are rarely collected in routine clinical practice. The International Society for Quality of Life Research (ISOQOL) has developed a User’s Guide for Implementing Patient-Reported Outcomes Assessment in Clinical Practice. This paper summarizes the key issues from the User’s Guide.MethodsUsing the literature, an ISOQOL team outlined considerations for using PROs in clinical practice; options for designing the intervention; and strengths, weaknesses, and resource requirements associated with each option.ResultsImplementing routine PRO assessment involves a number of methodological and practical decisions, including (1) identifying the goals for collecting PROs in clinical practice, (2) selecting the patients, setting, and timing of assessments, (3) determining which questionnaire(s) to use, (4) choosing a mode for administering and scoring the questionnaire, (5) designing processes for reporting results, (6) identifying aids to facilitate score interpretation, (7) developing strategies for responding to issues identified by the questionnaires, and (8) evaluating the impact of the PRO intervention on the practice.ConclusionsIntegrating PROs in clinical practice has the potential to enhance patient-centered care. The online version of the User’s Guide will be updated periodically.

Standardizing patient-reported outcomes assessment in cancer clinical trials: a patient-reported outcomes measurement information system initiative.

Patient-reported outcomes (PROs), such as symptom scales or more broad-based health-related quality-of-life measures, play an important role in oncology clinical trials. They frequently are used to help evaluate cancer treatments, as well as for supportive and palliative oncology care. To be most beneficial, these PROs must be relevant to patients and clinicians, valid, and easily understood and interpreted. The Patient-Reported Outcomes Measurement Information System (PROMIS) Network, part of the National Institutes of Health Roadmap Initiative, aims to improve appreciably how PROs are selected and assessed in clinical research, including clinical trials. PROMIS is establishing a publicly available resource of standardized, accurate, and efficient PRO measures of major self-reported health domains (eg, pain, fatigue, emotional distress, physical function, social function) that are relevant across chronic illnesses including cancer. PROMIS is also developing measures of self-reported health domains specifically targeted to cancer, such as sleep/wake function, sexual function, cognitive function, and the psychosocial impacts of the illness experience (ie, stress response and coping; shifts in self-concept, social interactions, and spirituality). We outline the qualitative and quantitative methods by which PROMIS measures are being developed and adapted for use in clinical oncology research. At the core of this activity is the formation and application of item banks using item response theory modeling. We also present our work in the fatigue domain, including a short-form measure, as a sample of PROMIS methodology and work to date. Plans for future validation and application of PROMIS measures are discussed.

Recommendations for Incorporating Patient-Reported Outcomes Into Clinical Comparative Effectiveness Research in Adult Oncology

Examining the patients subjective experience in prospective clinical comparative effectiveness research (CER) of oncology treatments or process interventions is essential for informing decision making. Patient-reported outcome (PRO) measures are the standard tools for directly eliciting the patient experience. There are currently no widely accepted standards for developing or implementing PRO measures in CER. Recommendations for the design and implementation of PRO measures in CER were developed via a standardized process including multistakeholder interviews, a technical working group, and public comments. Key recommendations are to include assessment of patient-reported symptoms as well as health-related quality of life in all prospective clinical CER studies in adult oncology; to identify symptoms relevant to a particular study population and context based on literature review and/or qualitative and quantitative methods; to assure that PRO measures used are valid, reliable, and sensitive in a comparable population (measures particularly recommended include EORTC QLQ-C30, FACT, MDASI, PRO-CTCAE, and PROMIS); to collect PRO data electronically whenever possible; to employ methods that minimize missing patient reports and include a plan for analyzing and reporting missing PRO data; to report the proportion of responders and cumulative distribution of responses in addition to mean changes in scores; and to publish results of PRO analyses simultaneously with other clinical outcomes. Twelve core symptoms are recommended for consideration in studies in advanced or metastatic cancers. Adherence to methodologic standards for the selection, implementation, and analysis/reporting of PRO measures will lead to an understanding of the patient experience that informs better decisions by patients, providers, regulators, and payers.

Evaluation of the Healthy Eating Index-2005

BACKGROUND The Healthy Eating Index (HEI), a measure of diet quality as specified by federal dietary guidance, was revised to conform to the Dietary Guidelines for Americans 2005. The HEI has several components, the scores of which are totaled. OBJECTIVE The validity and reliability of the HEI-2005 were evaluated. DESIGN Validity was assessed by answering four questions: Does the HEI-2005 1) give maximum scores to menus developed by experts; 2) distinguish between groups with known differences in diet quality-smokers and nonsmokers; 3) measure diet quality independently of energy intake, a proxy for diet quantity; and 4) have more than one underlying dimension? The relevant type of reliability, internal consistency, was also assessed. SUBJECTS Twenty-four-hour recalls from 8,650 participants, aged 2 years and older, in the National Health and Nutrition Examination Survey, 2001-2002 were analyzed to answer questions 2 to 4. Results were weighted to consider sample design and nonresponse. STATISTICAL ANALYSES T tests determined differences in scores between smokers and nonsmokers. Pearson correlation coefficients determined the relationship between energy intake and scores. Principal components analysis determined the number of factors that comprise the HEI-2005. Cronbachs coefficient alpha tested internal consistency. RESULTS HEI-2005 scores are at or very near the maximum levels for all sets of exemplary menus with one exception; the Harvard menus scored low on the milk component because these menus intentionally include only small amounts of milk products. Nine of 12 component scores were lower for smokers than nonsmokers. The correlations of component scores were virtually independent of energy intake (< I.22I). Multiple factors underlie the HEI-2005. Coefficient alpha was .43. The alpha value for all tests was .01. CONCLUSIONS The HEI-2005 is a valid measure of diet quality. Potential uses include population monitoring, evaluation of interventions, and research. The individual component scores provide essential information in addition to that provided by the total score.

Research design considerations for confirmatory chronic pain clinical trials: IMMPACT recommendations.

&NA; There has been an increase in the number of chronic pain clinical trials in which the treatments being evaluated did not differ significantly from placebo in the primary efficacy analyses despite previous research suggesting that efficacy could be expected. These findings could reflect a true lack of efficacy or methodological and other aspects of these trials that compromise the demonstration of efficacy. There is substantial variability among chronic pain clinical trials with respect to important research design considerations, and identifying and addressing any methodological weaknesses would enhance the likelihood of demonstrating the analgesic effects of new interventions. An IMMPACT consensus meeting was therefore convened to identify the critical research design considerations for confirmatory chronic pain trials and to make recommendations for their conduct. We present recommendations for the major components of confirmatory chronic pain clinical trials, including participant selection, trial phases and duration, treatment groups and dosing regimens, and types of trials. Increased attention to and research on the methodological aspects of confirmatory chronic pain clinical trials has the potential to enhance their assay sensitivity and ultimately provide more meaningful evaluations of treatments for chronic pain.

Development of the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)

The standard approach for documenting symptomatic adverse events (AEs) in cancer clinical trials involves investigator reporting using the National Cancer Institutes (NCIs) Common Terminology Criteria for Adverse Events (CTCAE). Because this approach underdetects symptomatic AEs, the NCI issued two contracts to create a patient-reported outcome (PRO) measurement system as a companion to the CTCAE, called the PRO-CTCAE. This Commentary describes development of the PRO-CTCAE by a group of multidisciplinary investigators and patient representatives and provides an overview of qualitative and quantitative studies of its measurement properties. A systematic evaluation of all 790 AEs listed in the CTCAE identified 78 appropriate for patient self-reporting. For each of these, a PRO-CTCAE plain language term in English and one to three items characterizing the frequency, severity, and/or activity interference of the AE were created, rendering a library of 124 PRO-CTCAE items. These items were refined in a cognitive interviewing study among patients on active cancer treatment with diverse educational, racial, and geographic backgrounds. Favorable measurement properties of the items, including construct validity, reliability, responsiveness, and between-mode equivalence, were determined prospectively in a demographically diverse population of patients receiving treatments for many different tumor types. A software platform was built to administer PRO-CTCAE items to clinical trial participants via the internet or telephone interactive voice response and was refined through usability testing. Work is ongoing to translate the PRO-CTCAE into multiple languages and to determine the optimal approach for integrating the PRO-CTCAE into clinical trial workflow and AE analyses. It is envisioned that the PRO-CTCAE will enhance the precision and patient-centeredness of adverse event reporting in cancer clinical research.