Stephen Joel Coons

Recommendations for Incorporating Patient-Reported Outcomes Into Clinical Comparative Effectiveness Research in Adult Oncology

Examining the patients subjective experience in prospective clinical comparative effectiveness research (CER) of oncology treatments or process interventions is essential for informing decision making. Patient-reported outcome (PRO) measures are the standard tools for directly eliciting the patient experience. There are currently no widely accepted standards for developing or implementing PRO measures in CER. Recommendations for the design and implementation of PRO measures in CER were developed via a standardized process including multistakeholder interviews, a technical working group, and public comments. Key recommendations are to include assessment of patient-reported symptoms as well as health-related quality of life in all prospective clinical CER studies in adult oncology; to identify symptoms relevant to a particular study population and context based on literature review and/or qualitative and quantitative methods; to assure that PRO measures used are valid, reliable, and sensitive in a comparable population (measures particularly recommended include EORTC QLQ-C30, FACT, MDASI, PRO-CTCAE, and PROMIS); to collect PRO data electronically whenever possible; to employ methods that minimize missing patient reports and include a plan for analyzing and reporting missing PRO data; to report the proportion of responders and cumulative distribution of responses in addition to mean changes in scores; and to publish results of PRO analyses simultaneously with other clinical outcomes. Twelve core symptoms are recommended for consideration in studies in advanced or metastatic cancers. Adherence to methodologic standards for the selection, implementation, and analysis/reporting of PRO measures will lead to an understanding of the patient experience that informs better decisions by patients, providers, regulators, and payers.

Capturing Patient-Reported Outcome (PRO) Data Electronically: The Past, Present, and Promise of ePRO Measurement in Clinical Trials

Patient-reported outcomes (PROs) are an important means of evaluating the treatment benefit of new medical products. It is recognized that PRO measures should be used when assessing concepts best known by the patient or best measured from the patient’s perspective. As a result, there is growing emphasis on well defined and reliable PRO measures. In addition, advances in technology have significantly increased electronic PRO (ePRO) data collection capabilities and options in clinical trials. The movement from paper-based to ePRO data capture has enhanced the integrity and accuracy of clinical trial data and is encouraged by regulators. A primary distinction in the types of ePRO platforms is between telephone-based interactive voice response systems and screen-based systems. Handheld touchscreen-based devices have become the mainstay for remote (i.e., off-site, unsupervised) PRO data collection in clinical trials. The conventional approach is to provide study subjects with a handheld device with a device-based proprietary software program. However, an emerging alternative for clinical trials is called bring your own device (BYOD). Leveraging study subjects’ own Internet-enabled mobile devices for remote PRO data collection (via a downloadable app or a Web-based data collection portal) has become possible due to the widespread use of personal smartphones and tablets. However, there are a number of scientific and operational issues that must be addressed before BYOD can be routinely considered as a practical alternative to conventional ePRO data collection methods. Nevertheless, the future for ePRO data collection is bright and the promise of BYOD opens a new chapter in its evolution.

Patient-reported outcomes in cancer drug development and US regulatory review perspectives from industry, the Food and Drug Administration, and the patient:

Data reported directly by patients about how they feel and function are rarely included in oncology drug labeling in the United States, in contrast to Europe and to nononcology labeling in the United States, where this practice is more common. Multiple barriers exist, including challenges unique to oncology trials, and industrys concerns regarding cost, logistical complexities, and the Food and Drug Administrations (FDAs) rigorous application of its 2009 guidance on the use of patient-reported outcome (PRO) measures. A panel consisting of representatives of industry, FDA, the PRO Consortium, clinicians, and patients was assembled at a 2014 workshop cosponsored by FDA to identify practical recommendations for overcoming these barriers. Key recommendations included increasing proactive encouragement by FDA to clinical trial sponsors for including PROs in drug development programs; provision of comprehensive PRO plans by sponsors to FDA early in drug development; promotion of an oncology-specific PRO research agenda; development of an approach to existing (legacy) PRO measures, when appropriate (focused initially on symptoms and functional status); and increased FDA and industry training in PRO methodology. FDA has begun implementing several of these recommendations.

A Median Model for Predicting United States Population-Based EQ-5D Health State Preferences

OBJECTIVEnThe D1 model that was developed to predict US societal preferences for EQ-5D health states addressed several important conceptual and statistical issues. However, it has been criticized for being too complex, failing to account for the nonnormal distribution of health state values, and the transformation of preferences for worse-than-death health states before estimation. This research was conducted to develop an improved model for predicting median preferences for EQ-5D health states for the US population.nnnMETHODSnProbability-weighted least absolute deviations regression was used to fit models to the time trade-off data collected in the US Valuation of the EQ-5D Health States study. No transformation was applied to the values for states considered worse than death. Several model specifications that differed with respect to explanatory variables were evaluated using two-sample cross-validation.nnnRESULTSnThe best-fitting model included only fixed effects for moderate or severe problems in each of the 5 EQ-5D dimensions and excluded a constant. This specification yielded rank correlations between observed and predicted values and median observed and predicted values of 0.635 and 0.991, respectively, as well as a median absolute error of 0.026. The predicted median preferences ranged from 1.00 for full health, to -0.81 for the worst possible health state.nnnCONCLUSIONSnDue to its simplicity and robustness, a median model is superior to other models for predicting US population preferences for EQ-5D health states. The predictions of this model are suggested for use in applications that require US societal health state values.

Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format

BackgroundPatients receiving complementary and alternative medicine (CAM) therapies often report shifts in well-being that go beyond resolution of the original presenting symptoms. We undertook a research program to develop and evaluate a patient-centered outcome measure to assess the multidimensional impacts of CAM therapies, utilizing a novel mixed methods approach that relied upon techniques from the fields of anthropology and psychometrics. This tool would have broad applicability, both for CAM practitioners to measure shifts in patients states following treatments, and conventional clinical trial researchers needing validated outcome measures. The US Food and Drug Administration has highlighted the importance of valid and reliable measurement of patient-reported outcomes in the evaluation of conventional medical products. Here we describe Phase I of our research program, the iterative process of content identification, item development and refinement, and response format selection. Cognitive interviews and psychometric evaluation are reported separately.MethodsFrom a database of patient interviews (n = 177) from six diverse CAM studies, 150 interviews were identified for secondary analysis in which individuals spontaneously discussed unexpected changes associated with CAM. Using ATLAS.ti, we identified common themes and language to inform questionnaire item content and wording. Respondents language was often richly textured, but item development required a stripping down of language to extract essential meaning and minimize potential comprehension barriers across populations. Through an evocative card sort interview process, we identified those items most widely applicable and covering standard psychometric domains. We developed, pilot-tested, and refined the format, yielding a questionnaire for cognitive interviews and psychometric evaluation.ResultsThe resulting questionnaire contained 18 items, in visual analog scale format, in which each line was anchored by the positive and negative extremes relevant to the experiential domain. Because of frequent informant allusions to response set shifts from before to after CAM therapies, we chose a retrospective pretest format. Items cover physical, emotional, cognitive, social, spiritual, and whole person domains.ConclusionsThis paper reports the success of a novel approach to the development of outcome instruments, in which items are extracted from patients words instead of being distilled from pre-existing theory. The resulting instrument, focused on measuring shifts in patients perceptions of health and well-being along pre-specified axes, is undergoing continued testing, and is available for use by cooperating investigators.

Patient-Reported Outcomes in Clinical Trials of CKD-Related Therapies: Report of a Symposium Sponsored by the National Kidney Foundation and the US Food and Drug Administration

The National Kidney Foundation and the U.S. Food and Drug Administration (FDA) convened a symposium in September 2010, bringing together more than 70 experts, including representatives from the FDA, the National Institutes of Health, the Critical Path Institute, nephrologists, patients, and the pharmaceutical industry to discuss the feasibility and process of developing patient-reported outcome (PRO) measures to access how patients feel or function to be used in clinical trials for regulatory review of treatment benefit. Three disease areas were evaluated for development of end point models in which PRO measures may be useful: anemia secondary to chronic kidney disease, autosomal dominant polycystic kidney disease (ADPKD), and nephrotic syndrome. The participants thought it valuable to use observational data to generate hypotheses regarding patient baseline characteristics that are likely to predict clinically important changes in PROs in response to anemia treatment and to design adequately powered blinded randomized controlled trials of anemia treatment using PROs as primary rather than secondary end points. Validated PRO instruments that reflect the patient experience in ADPKD and nephrotic syndrome are essential to incorporate into clinical trials of new therapeutic interventions because glomerular filtration rate decline may occur late in the disease course, at which point therapeutic benefit is less likely. Conference attendees addressed how PRO measures could be used to evaluate, monitor, provide care, and facilitate the introduction of treatments for patients with these challenging conditions.

PRO Data Collection in Clinical Trials Using Mixed Modes: Report of the ISPOR PRO Mixed Modes Good Research Practices Task Force

The objective of this report was to address the use and mixing of data collection modes within and between trials in which patient-reported outcome (PRO) end points are intended to be used to support medical product labeling. The report first addresses the factors that should be considered when selecting a mode or modes of PRO data collection in a clinical trial, which is often when mixing is first considered. Next, a summary of how to faithfully migrate instruments is presented followed by a section on qualitative and quantitative study designs used to evaluate measurement equivalence of the new and original modes of data collection. Finally, the report discusses a number of issues that must be taken into account when mixing modes is deemed necessary or unavoidable within or between trials, including considerations of the risk of mixing at different levels within a clinical trial program and mixing between different types of platforms. In the absence of documented evidence of measurement equivalence, it is strongly recommended that a quantitative equivalence study be conducted before mixing modes in a trial to ensure that sufficient equivalence can be demonstrated to have confidence in pooling PRO data collected by the different modes. However, we also strongly discourage the mixing of paper and electronic field-based instruments and suggest that mixing of electronic modes be considered for clinical trials and only after equivalence has been established. If proceeding with mixing modes, it is important to implement data collection carefully in the trial itself in a planned manner at the country level or higher and minimize ad hoc mixing by sites or individual subjects. Finally, when mixing occurs, it must be addressed in the statistical analysis plan for the trial and the ability to pool the data must be evaluated to then evaluate treatment effects with mixed modes data. A successful mixed modes trial requires a faithful migration, measurement equivalence established between modes, and carefully planned implementation to minimize the risk of increased measurement error impacting the power of the trial to detect a treatment effect.

Developing a patient-centered outcome measure for complementary and alternative medicine therapies II: Refining content validity through cognitive interviews

BackgroundAvailable measures of patient-reported outcomes for complementary and alternative medicine (CAM) inadequately capture the range of patient-reported treatment effects. The Self-Assessment of Change questionnaire was developed to measure multi-dimensional shifts in well-being for CAM users. With content derived from patient narratives, items were subsequently focused through interviews on a new cohort of participants. Here we present the development of the final version in which the content and format is refined through cognitive interviews.MethodsWe conducted cognitive interviews across five iterations of questionnaire refinement with a culturally diverse sample of 28 CAM users. In each iteration, participant critiques were used to revise the questionnaire, which was then re-tested in subsequent rounds of cognitive interviews. Following all five iterations, transcripts of cognitive interviews were systematically coded and analyzed to examine participants understanding of the format and content of the final questionnaire. Based on this data, we established summary descriptions and selected exemplar quotations for each word pair on the final questionnaire.ResultsThe final version of the Self-Assessment of Change questionnaire (SAC) includes 16 word pairs, nine of which remained unchanged from the original draft. Participants consistently said that these stable word pairs represented opposite ends of the same domain of experience and the meanings of these terms were stable across the participant pool. Five pairs underwent revision and two word pairs were added. Four word pairs were eliminated for redundancy or because participants did not agree on the meaning of the terms. Cognitive interviews indicate that participants understood the format of the questionnaire and considered each word pair to represent opposite poles of a shared domain of experience.ConclusionsWe have placed lay language and direct experience at the center of questionnaire revision and refinement. In so doing, we provide an innovative model for the development of truly patient-centered outcome measures. Although this instrument was designed and tested in a CAM-specific population, it may be useful in assessing multi-dimensional shifts in well-being across a broader patient population.

“Bring Your Own Device” (BYOD) The Future of Field-Based Patient-Reported Outcome Data Collection in Clinical Trials?

Field-based patient-reported outcome (PRO) assessments, including measures of signs, symptoms, and events that are administered outside of the research clinic, can be critical in evaluating the efficacy and safety of new medical treatments. Collection of this type of data commonly involves providing subjects with stand-alone electronic devices, such as smartphones, that they can use to respond to assessments in their home or work environment. Although this approach has proven useful, it is also limited in several ways: For example, provisioning stand-alone devices can be costly for sponsors, and requiring subjects to carry a device that is exclusively dedicated to the study can be burdensome. The “Bring Your Own Device” (BYOD) approach, in which subjects use their own smartphone or Internet-enabled device to complete field-based PRO assessments, addresses many of these concerns. However, the BYOD model has its own limitations that should be considered. In this article, representatives of the ePRO Consortium review operational, privacy/security, and scientific/regulatory considerations regarding BYOD. We hope that this review will allow researchers to make informed decisions when choosing methods to collect field-based PRO data in future clinical trials. Additionally, we hope that the discussion in this article will establish a research agenda for further examination of BYOD approaches.

Testing the measurement equivalence of paper and interactive voice response system versions of the EORTC QLQ-C30

PurposeThe objective of this study was to evaluate the measurement equivalence of an interactive voice response system (IVRS) version and the original paper-based version of the EORTC QLQ-C30.MethodsThe QLQ-C30 is a cancer-specific, health-related quality of life questionnaire consisting of nine multi-item scales (physical, role, emotional, cognitive and social functioning, fatigue, nausea/vomiting, pain, and quality of life) and six single item measures (dyspnea, insomnia, appetite loss, constipation, diarrhea, and financial problems). This study utilized a crossover design with subjects randomly assigned to one of two assessment orders: (1) paper then IVRS or (2) IVRS then paper. Equivalence between the two administration modes was established by comparing the 95xa0% lower confidence interval (CI) of the intraclass correlation coefficients (ICCs) for each scale, with a critical value of 0.70.ResultsThe ICCs for the nine multi-item scales were all above 0.79, ranging from 0.791 to 0.899 (ICC 95xa0% lower CI range 0.726–0.865) and significantly different from our threshold reliability of 0.70. The ICCs for the six single items ranged from 0.689 to 0.896 (ICC 95xa0% lower CI range 0.611–0.888). Two of the items, insomnia and appetite loss, were not statistically different from 0.70. When considered together, the per-protocol analysis results support the equivalence of the paper and IVRS versions of the QLQ-C30 for 13 of the 15 scores.ConclusionThis analysis provides evidence that the scores obtained from the IVRS version of the QLQ-C30 are equivalent to those obtained with the original paper version except for the insomnia and appetite loss items.