Laurie B. Burke

Content Validity—Establishing and Reporting the Evidence in Newly Developed Patient-Reported Outcomes (PRO) Instruments for Medical Product Evaluation: ISPOR PRO Good Research Practices Task Force Report: Part 2—Assessing Respondent Understanding

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts developers or users purport it to assess. A PRO instrument measures the concepts most relevant and important to a patients condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Part 1 of this task force report covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Building on qualitative interviews and focus groups used to elicit concepts, cognitive interviews help developers craft items that can be understood by respondents in the target population and can ultimately confirm that the final instrument is appropriate, comprehensive, and understandable in the target population. Part 2 details: 1) the methods for conducting cognitive interviews that address patient understanding of items, instructions, and response options; and 2) the methods for tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. The task force reports two parts are meant to be read together. They are intended to offer suggestions for good practice in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Interpreting the clinical importance of group differences in chronic pain clinical trials: IMMPACT recommendations.

ABSTRACT An essential component of the interpretation of results of randomized clinical trials of treatments for chronic pain involves the determination of their clinical importance or meaningfulness. This involves two distinct processes—interpreting the clinical importance of individual patient improvements and the clinical importance of group differences—which are frequently misunderstood. In this article, we first describe the essential differences between the interpretation of the clinical importance of patient improvements and of group differences. We then discuss the factors to consider when evaluating the clinical importance of group differences, which include the results of responder analyses of the primary outcome measure, the treatment effect size compared to available therapies, analyses of secondary efficacy endpoints, the safety and tolerability of treatment, the rapidity of onset and durability of the treatment benefit, convenience, cost, limitations of existing treatments, and other factors. The clinical importance of individual patient improvements can be determined by assessing what patients themselves consider meaningful improvement using well‐described methods. In contrast, the clinical meaningfulness of group differences must be determined by a multi‐factorial evaluation of the benefits and risks of the treatment and of other available treatments for the condition in light of the primary goals of therapy. Such determinations must be conducted on a case‐by‐case basis, and are ideally informed by patients and their significant others, clinicians, researchers, statisticians, and representatives of society at large.

Content Validity—Establishing and Reporting the Evidence in Newly Developed Patient-Reported Outcomes (PRO) Instruments for Medical Product Evaluation: ISPOR PRO Good Research Practices Task Force Report: Part 1—Eliciting Concepts for a New PRO Instrument

The importance of content validity in developing patient reported outcomes (PRO) instruments is stressed by both the US Food and Drug Administration and the European Medicines Agency. Content validity is the extent to which an instrument measures the important aspects of concepts that developers or users purport it to assess. A PRO instrument measures the concepts most significant and relevant to a patients condition and its treatment. For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population and comprehensive with respect to patient concerns. Documentation of target population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity. Developing content for, and assessing respondent understanding of, newly developed PRO instruments for medical product evaluation will be discussed in this two-part ISPOR PRO Good Research Practices Task Force Report. Topics include the methods for generating items, documenting item development, coding of qualitative data from item generation, cognitive interviewing, and tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies. Part 1 covers elicitation of key concepts using qualitative focus groups and/or interviews to inform content and structure of a new PRO instrument. Part 2 covers the instrument development process, the assessment of patient understanding of the draft instrument using cognitive interviews and steps for instrument revision. The two parts are meant to be read together. They are intended to offer suggestions for good practices in planning, executing, and documenting qualitative studies that are used to support the content validity of PRO instruments to be used in medical product evaluation.

Identifying important outcome domains for chronic pain clinical trials: An IMMPACT survey of people with pain

&NA; This two‐phase study was conducted to identify relevant domains of patient‐reported outcomes from the perspective of people who experience chronic pain. In Phase 1, focus groups were conducted to generate a pool of patient outcome‐related domains and their components. The results of the focus groups identified 19 aspects of their lives that were significantly impacted by the presence of their symptoms and for which improvements were important criteria they would use in evaluating the effectiveness of any treatment. Phase 2 was conducted to examine the importance and relevance of domains identified from a much larger and diverse sample of people with chronic pain. A survey was developed and posted on the American Chronic Pain Association website. Participants were asked to rate the importance of each item or domain identified by the focus groups on a scale of 0 to10 (i.e., 0 = “not at all important” and 10 = “extremely important”). The survey was completed by 959 individuals. The results indicate that all 19 aspects of daily life derived from the focus groups were considered important with a majority of respondents indicating a score of 8 or greater. In addition to pain reduction, the most important aspects were enjoyment of life, emotional well‐being, fatigue, weakness, and sleep‐related problems. Chronic pain clearly impacts health‐related quality of life. The results of the two phases of the study indicate that people with chronic pain consider functioning and well‐being as important areas affected by the presence of symptoms and as appropriate targets of treatment. These multiple outcomes should be considered when evaluating the efficacy and effectiveness of chronic pain treatments.

Development and initial validation of an expanded and revised version of the Short-form McGill Pain Questionnaire (SF-MPQ-2).

ABSTRACT The objective of the present research was to develop a single measure of the major symptoms of both neuropathic and non‐neuropathic pain that can be used in studies of epidemiology, natural history, pathophysiologic mechanisms, and treatment response. We expanded and revised the Short‐form McGill Pain Questionnaire1 (SF‐MPQ) pain descriptors by adding symptoms relevant to neuropathic pain and by modifying the response format to a 0–10 numerical rating scale to provide increased responsiveness in longitudinal studies and clinical trials. The reliability, validity, and subscale structure of the revised SF‐MPQ (SF‐MPQ‐21) were examined in responses from 882 individuals with diverse chronic pain syndromes and in 226 patients with painful diabetic peripheral neuropathy who participated in a randomized clinical trial. The data suggest that the SF‐MPQ‐2 has excellent reliability and validity, and the results of both exploratory and confirmatory factor analyses provided support for four readily interpretable subscales—continuous pain, intermittent pain, predominantly neuropathic pain, and affective descriptors. These results provide a basis for use of the SF‐MPQ‐2 in future clinical research, including clinical trials of treatments for neuropathic and non‐neuropathic pain conditions.

Research design considerations for confirmatory chronic pain clinical trials: IMMPACT recommendations.

&NA; There has been an increase in the number of chronic pain clinical trials in which the treatments being evaluated did not differ significantly from placebo in the primary efficacy analyses despite previous research suggesting that efficacy could be expected. These findings could reflect a true lack of efficacy or methodological and other aspects of these trials that compromise the demonstration of efficacy. There is substantial variability among chronic pain clinical trials with respect to important research design considerations, and identifying and addressing any methodological weaknesses would enhance the likelihood of demonstrating the analgesic effects of new interventions. An IMMPACT consensus meeting was therefore convened to identify the critical research design considerations for confirmatory chronic pain trials and to make recommendations for their conduct. We present recommendations for the major components of confirmatory chronic pain clinical trials, including participant selection, trial phases and duration, treatment groups and dosing regimens, and types of trials. Increased attention to and research on the methodological aspects of confirmatory chronic pain clinical trials has the potential to enhance their assay sensitivity and ultimately provide more meaningful evaluations of treatments for chronic pain.

Developing patient-reported outcome measures for pain clinical trials : IMMPACT recommendations

a University of Washington, Seattle, WA 98195, USA b University of Rochester School of Medicine and Dentistry, Rochester, NY, USA c United States Food and Drug Administration, Rockville, MD, USA d Northwestern University, Chicago, IL, USA e Johnson and Johnson, Raritan, NY, USA f AstraZeneca, Wilmington, DE, USA g University of California San Diego, La Jolla, CA, USA h Merck and Company, Blue Bell, PA, USA i University of Texas, M.D. Anderson Cancer Center, USA j American Chronic Pain Association, Rocklin, CA, USA k Allergan, Inc, Irvine, CA, USA l National Institute of Dental and Craniofacial Research, Bethesda, MD, USA m University of Pennsylvania, Philadelphia, PA, USA n Johns Hopkins University, Baltimore, MD, USA o University Health Network and University of Toronto, Toronto, Canada p Novartis Pharmaceuticals, East Hanover, NJ, USA q VA Connecticut Healthcare System, West Haven, CT, USA r Yale University, New Haven, CT, USA s Celgene Corporation, Warren, NJ, USA t Pfizer Global Research and Development, Ann Arbor, MI, USA u Dalhousie University, Halifax, Nova Scotia, Canada v London Regional Cancer Centre, London, Ont., Canada

Use of Existing Patient-Reported Outcome (PRO) Instruments and Their Modification: The ISPOR Good Research Practices for Evaluating and Documenting Content Validity for the Use of Existing Instruments and Their Modification PRO Task Force Report

BACKGROUND Patient-reported outcome (PRO) instruments are used to evaluate the effect of medical products on how patients feel or function. This article presents the results of an ISPOR task force convened to address good clinical research practices for the use of existing or modified PRO instruments to support medical product labeling claims. The focus of the article is on content validity, with specific reference to existing or modified PRO instruments, because of the importance of content validity in selecting or modifying an existing PRO instrument and the lack of consensus in the research community regarding best practices for establishing and documenting this measurement property. METHODS Topics addressed in the article include: definition and general description of content validity; PRO concept identification as the important first step in establishing content validity; instrument identification and the initial review process; key issues in qualitative methodology; and potential threats to content validity, with three case examples used to illustrate types of threats and how they might be resolved. A table of steps used to identify and evaluate an existing PRO instrument is provided, and figures are used to illustrate the meaning of content validity in relationship to instrument development and evaluation. RESULTS & RECOMMENDATIONS: Four important threats to content validity are identified: unclear conceptual match between the PRO instrument and the intended claim, lack of direct patient input into PRO item content from the target population in which the claim is desired, no evidence that the most relevant and important item content is contained in the instrument, and lack of documentation to support modifications to the PRO instrument. In some cases, careful review of the threats to content validity in a specific application may be reduced through additional well documented qualitative studies that specifically address the issue of concern. CONCLUSION Published evidence of the content validity of a PRO instrument for an intended application is often limited. Such evidence is, however, important to evaluating the adequacy of a PRO instrument for the intended application. This article provides an overview of key issues involved in assessing and documenting content validity as it relates to using existing instruments in the drug approval process.

Use of menopausal estrogens and medroxyprogesterone in the United States, 1982-1992

To describe trends in the prescription of menopausal estrogens and medroxyprogesterone in the United States. Methods Annual estimates of the number of prescriptions for menopausal estrogens and medroxyprogesterone and descriptive information on patients and providers were obtained from two pharmaceutical marketing research data bases, the National Prescription Audit and the National Disease and Therapeutic Index of IMS America. Results An estimated 13.6 million prescriptions were dispensed for oral menopausal estrogens in 1982, and 31.7 million in 1992, a 2.3-fold increase (P = .0001). In 1992, Premarin, the only oral conjugated estrogen currently approved for use, was the most frequently dispensed brand-name pharmaceutical in the United States. Dispensed prescriptions for Estraderm, a transdermal estradiol first marketed in 1986, increased from 1.5 million in 1987 to 4.7 million in 1992. Dispensed prescriptions for oral medroxy-progesterone also increased from 2.3 million prescriptions in 1982 to 11.3 million in 1992, a 4.9-fold increase (P = .0001). An estimated one in six to one in four postmenopausal women were taking menopausal hormones in 1992. These drugs were prescribed mainly by obstetrician-gynecologists. Conclusion The use of menopausal estrogens and medroxyprogesterone has increased substantially over the past decade. These trends indicate that American women are widely exposed to menopausal hormone replacement.

Review and recommendationsResearch design considerations for confirmatory chronic pain clinical trials: IMMPACT recommendations

&NA; There has been an increase in the number of chronic pain clinical trials in which the treatments being evaluated did not differ significantly from placebo in the primary efficacy analyses despite previous research suggesting that efficacy could be expected. These findings could reflect a true lack of efficacy or methodological and other aspects of these trials that compromise the demonstration of efficacy. There is substantial variability among chronic pain clinical trials with respect to important research design considerations, and identifying and addressing any methodological weaknesses would enhance the likelihood of demonstrating the analgesic effects of new interventions. An IMMPACT consensus meeting was therefore convened to identify the critical research design considerations for confirmatory chronic pain trials and to make recommendations for their conduct. We present recommendations for the major components of confirmatory chronic pain clinical trials, including participant selection, trial phases and duration, treatment groups and dosing regimens, and types of trials. Increased attention to and research on the methodological aspects of confirmatory chronic pain clinical trials has the potential to enhance their assay sensitivity and ultimately provide more meaningful evaluations of treatments for chronic pain.