William R. Lenderking

The use of patient-reported outcomes (PRO) within comparative effectiveness research: implications for clinical practice and health care policy.

Background:The goal of comparative effectiveness research (CER) is to explain the differential benefits and harms of alternate methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. To inform decision making, information from the patient’s perspective that reflects outcomes that patients care about are needed and can be collected rigorously using appropriate patient-reported outcomes (PRO). It can be challenging to select the most appropriate PRO measure given the proliferation of such questionnaires over the past 20 years. Objective:In this paper, we discuss the value of PROs within CER, types of measures that are likely to be useful in the CER context, PRO instrument selection, and key challenges associated with using PROs in CER. Methods:We delineate important considerations for defining the CER context, selecting the appropriate measures, and for the analysis and interpretation of PRO data. Emerging changes that may facilitate CER using PROs as an outcome are also reviewed including implementation of electronic and personal health records, hospital and population-based registries, and the use of PROs in national monitoring initiatives. The potential benefits of linking the information derived from PRO endpoints in CER to decision making is also reviewed. Conclusions:The recommendations presented for incorporating PROs in CER are intended to provide a guide to researchers, clinicians, and policy makers to ensure that information derived from PROs is applicable and interpretable for a given CER context. In turn, CER will provide information that is necessary for clinicians, patients, and families to make informed care decisions.

Evaluation of the Quality of Life Associated with Zidovudine Treatment in Asymptomatic Human Immunodeficiency Virus Infection

BACKGROUND Zidovudine therapy is recommended for asymptomatic patients infected with the human immunodeficiency virus (HIV) who have fewer than 500 CD4+ cells per cubic millimeter. An analysis of the quality of life associated with therapy that integrated both the effects of adverse events and the benefits of delayed disease progression might influence this recommendation. METHODS We applied a survival analysis adjusted for the quality of life to data from a randomized trial conducted by the AIDS Clinical Trials Group. The trial compared treatment with 500 mg of zidovudine per day, 1500 mg of zidovudine per day, and placebo (Protocol 019) in 1338 asymptomatic HIV-infected patients. RESULTS The average time with neither a progression of disease nor an adverse event (symptom or laboratory finding) was 15.7, 15.6, and 14.8 months for patients receiving placebo, 500 mg of zidovudine, and 1500 mg of zidovudine, respectively. The incidence of severe symptoms was 13.8 percent in the placebo group, 15.2 percent in the 500-mg group, and 19.9 percent in the 1500-mg group (P = 0.038). After 18 months, the 500-mg group gained an average of 0.5 months without disease progression, as compared with the placebo group, but had severe adverse events an average of 0.6 months sooner. The 500-mg group had more quality-of-life--adjusted time than the placebo group only if the time lived after the progression of disease was considered by a patient to have less value than the time after the occurrence of a severe symptom. CONCLUSIONS For asymptomatic patients treated with 500 mg of zidovudine, a reduction in the quality of life due to severe side effects of therapy approximately equals the increase in the quality of life associated with a delay in the progression of HIV disease.

Methods for interpreting change over time in patient-reported outcome measures

PurposeInterpretation guidelines are needed for patient-reported outcome (PRO) measures’ change scores to evaluate efficacy of an intervention and to communicate PRO results to regulators, patients, physicians, and providers. The 2009 Food and Drug Administration (FDA) Guidance for Industry Patient-Reported Outcomes (PRO) Measures: Use in Medical Product Development to Support Labeling Claims (hereafter referred to as the final FDA PRO Guidance) provides some recommendations for the interpretation of change in PRO scores as evidence of treatment efficacy.MethodsThis article reviews the evolution of the methods and the terminology used to describe and aid in the communication of meaningful PRO change score thresholds.ResultsAnchor- and distribution-based methods have played important roles, and the FDA has recently stressed the importance of cross-sectional patient global assessments of concept as anchor-based methods for estimation of the responder definition, which describes an individual-level treatment benefit. The final FDA PRO Guidance proposes the cumulative distribution function (CDF) of responses as a useful method to depict the effect of treatments across the study population.ConclusionsWhile CDFs serve an important role, they should not be a replacement for the careful investigation of a PRO’s relevant responder definition using anchor-based methods and providing stakeholders with a relevant threshold for the interpretation of change over time.

Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting

Content validity of patient-reported outcome measures (PROs) has been a focus of debate since the 2006 publication of the U.S. FDA Draft Guidance for Industry in Patient Reported Outcome Measurement. Under the auspices of the Patient Reported Outcomes Measurement Information System (PROMIS) initiative, a working meeting on content validity was convened with leading PRO measurement experts. Platform presentations and participant discussion highlighted key issues in the content validity debate, including inconsistency in the definition and evaluation of content validity, the need for empirical research to support methodological approaches to the evaluation of content validity, and concerns that continual re-evaluation of content validity slows the pace of science and leads to the proliferation of study-specific PROs. We advocate an approach to the evaluation of content validity, which includes meticulously documented qualitative and advanced quantitative methods. To advance the science of content validity in PROs, we recommend (1) development of a consensus definition of content validity; (2) development of content validity guidelines that delineate the role of qualitative and quantitative methods and the integration of multiple perspectives; (3) empirical evaluation of generalizability of content validity across applications; and (4) use of generic measures as the foundation for PROs assessment.

Quality-of-Life Evaluation in a Clinical Trial of Zidovudine Therapy in Patients with Mildly Symptomatic HIV Infection

OBJECTIVE To evaluate the effects of zidovudine therapy in patients with mildly symptomatic HIV infection using Q-TWiST (quality-adjusted: Time Without Symptoms and Toxicity). DESIGN Analysis of a previously reported multicenter, randomized, placebo-controlled clinical trial. SETTING Thirty-two AIDS Clinical Trial units. PATIENTS A total of 351 patients with mildly symptomatic HIV infection were assigned to placebo, and 360 patients were assigned to zidovudine, 1200 mg/d. MEASUREMENTS A modified Q-TWiST method for comparing treatments based on time spent without severe symptomatic adverse events and without disease progression. Zidovudine and placebo were compared in a threshold utility analysis considering reduction in quality of life associated with adverse events and disease progression. Adverse events defined by laboratory findings were distinguished from findings representing symptomatic events. RESULTS The incidence of severe symptomatic adverse events was 22.8% for the zidovudine group and 15.1% for the placebo group (P = 0.01), but, as previously reported, zidovudine improved progression-free survival relative to placebo (at 18 months, 91% compared with 81%; P = 0.001). In an 18-month period, patients receiving zidovudine went an average of 14.5 months without disease progression or a severe symptomatic adverse event compared with 14.7 months for placebo. The zidovudine group gained 0.9 months without disease progression but lost 1.1 months due to adverse events. Within the 18-month observation period, treatment provided more Q-TWiST than placebo if the quality of life after HIV disease progression was assumed to be 10% to 20% worse than the quality of life after a severe symptomatic adverse event. CONCLUSIONS The Q-TWiST analysis projects that quality-of-life reductions due to severe symptomatic adverse events might be balanced by the quality-of-life benefits of delayed HIV disease progression for patients who received zidovudine for mildly symptomatic HIV infection. At currently recommended doses (500 to 600 mg/d, half the dose used in this study) zidovudine therapy is likely to yield a more favorable result.

Measuring quality of life in early HIV disease: the modular approach.

Rationale: to examine the reliability and validity of the General Health Self-assessment, a modular questionnaire for self-assessment of quality of life (QoL) in human immunodeficiency virus (HIV) clinical trials and to describe the baseline QoL of participants in a large HIV clinical trial. Design: the domains assessed include health perceptions, physical, psychological and role/social functioning, health care utilization and symptom distress. Method: 1,694 subjects with early HIV infection enrolled in the AIDS Clinical Trials Group Protocol 175 completed the scale at baseline. Results: the domains demonstrated reliability, construct and discriminant validity. A worse QoL was associated with recent hospitalization and symptomatic status. Prior antiretroviral therapy was associated with higher health perceptions and well-being. The presence of symptom distress was related to lower QoL on the other scales. There was no relationship between QoL scales and the baseline CD4 count. Women showed a lower QoL than men on all scales, while ethnicity was related to differences in health perceptions and physical and psychological functioning. Conclusions: the General Health Self-assessment shows excellent potential as a measure of QoL for HIV-infected patients in clinical trials. Further research is necessary to determine the responsiveness of the scale to clinical and immunological changes in HIV-infected individuals.

The social context of drinking, drug use, and unsafe sex in the Boston Young Men Study.

The objective of this study was to evaluate the relation between drinking, drug use, and unprotected anal intercourse in young men who have sex with men. A cross-sectional analysis of first-visit data from a prospective cohort of 508 young gay men recruited from 1993 through 1994 from bars, college campuses, and the Fenway Community Health Center in Boston was performed. The major outcome measures were any unprotected anal intercourse, after drinking and when sober, stratified by type of sexual partner (steady or nonsteady) during the previous 6 months and during the most recent sexual encounter. The average age of the cohort was 23.3 years; 77.6% were white, and 76.4% were in college. These young men had a median of 10.5 male sexual partners in their lifetimes, and 3 sexual partners in the previous 6 months before enrollment. One hundred and thirty-four (26%) reported unprotected anal intercourse during the previous 6 months. Individuals who had unprotected anal intercourse were more likely to have a drinking problem (odds ratio [OR] = 1.95; 95% confidence interval [CI] = 1.26-3.01) and drank more (20.4 ml/day versus 13.9 ml/day; p < or = 0.01), compared with individuals who did not engage in unprotected anal intercourse. Overall, men were significantly less likely to have unprotected anal intercourse after alcohol or drug use, based on a series of paired analysis (OR = 0.27; 95% CI = 0.15-0.48). However, when we stratified by type of sexual partner, men were significantly more likely to have unprotected anal intercourse with their nonsteady sexual partners after drinking than when sober (OR = 4.33; 95% CI = 1.37-13.7), but were significantly less likely to have unprotected anal intercourse with steady partners (OR = 0.27; 95% CI = 0.15-0.48). The patterns observed as already mentioned for drinking were also found for substance use in general. Men who were more likely to have unprotected anal intercourse after substance use were significantly more likely to have a drinking problem (OR = 7.65; 95% CI = 2.34-24.59). These results suggest that the role of alcohol and unsafe sex in young gay men is complex, with the role of situational factors of paramount importance. Alcohol and substance use interventions designed to reduce HIV risk need to specify the role of substance use in the sexual context to be successful.

Is Rasch model analysis applicable in small sample size pilot studies for assessing item characteristics? An example using PROMIS pain behavior item bank data

PurposeLarge samples are generally considered necessary for Rasch model to obtain robust item parameter estimates. Recently, small sample Rasch analysis was suggested as preliminary assessment of items’ psychometric properties. This study is to evaluate the Rasch analysis results using small sample sizes.MethodsTen PROMIS pain behavior items were used. Random samples of 30, 50, 100, and 250, and a targeted sample of 30 were drawn 10 times each from a total of 800 subjects. Rasch analysis was conducted for each of these samples and the full sample.ResultsIn the full sample, there were 104 cases of extreme scores, no null categories, two incorrectly ordered items, and four misfit items. For samples of 250, 100, 50, 30, and targeted 30, the average numbers of extreme scores were 42.2, 17.1, 9.6, 6.1, and 1.2; the average numbers of null categories were 1.0, 3.2, 8.7, 13.4, and 8.3; the average numbers of items with incorrectly ordered item parameters were 0.1, 0.8, 2.9, 4.7, and 3.7; and the average numbers of items with fit residuals exceeding ±2.5 were 0.8, 0.3, 0.1, 0.2, and 0.3, respectively.ConclusionsRasch analysis based on small samples (≤50) identified a greater number of items with incorrectly ordered parameters than larger samples (≥100). However, fewer items were identified as misfitting. Results from small samples led to opposite conclusions from those based on larger samples. Rasch analysis based on small samples should be used for exploratory purposes with extreme caution.

Unsafe sex in men who have sex with both men and women.

The sexual behaviors of bisexually active men, defined as men having sex with a man and a woman in previous 6 months, were compared with men who had sex with men only. Differential sexual practices associated with HIV risk between the two groups of men, as well as in the bisexual men with their male and female partners, were evaluated. Cross-sectional analyses were performed on baseline data from a prospective cohort of 508 young gay men recruited from bars, college campuses, and a health center in Boston from 1993 to 1994. Odds ratios (OR) and 95% confidence intervals (CI) were calculated on categorical variables, and McNemars chi2 was used to compare the behaviors of bisexual men with their male versus female sex partners. Six months before the interview, 47 (10%) men had male and female sex partners, and 383 men had only male sex partners during the past year or ever. Fifty-eight percent of the men in the study had a female sexual partner in their lifetime, and 18% during the past year. Bisexual men were more likely to have drinking problems as identified by the Michigan Alcoholism Screening Test (MAST; OR = 3.96, 95% CI = 1.54-10.20), and fewer male partners over their lifetime (mean +/- standard deviation [SD], 24+/-42; median, 7; versus mean +/- SD, 69+/-516; median, 12), although this difference was not statistically significant. The two groups had similar levels of unprotected anal intercourse (25.5% versus 29.5%); however, bisexual men were half as likely to have anal sex as homosexual men (OR = 0.50; 95% CI = 0.27-0.93). Bisexual men were three times as likely to have unprotected sex with their female partner as their male partner (OR = 3.0; 95% CI = 1.02-8.8). Stratified analysis revealed similar discordant behavior while sober (OR = 4.0), drinking (OR = 7.0), and while drinking with concurrent drug use (OR = 8.0). Among this cohort of men who have sex with men (MSM), a sizable proportion also had vaginal sex with female partners in the previous 6 months. Bisexually active men were more likely to have unprotected sex with their female partners compared with their male partners, potentially increasing the risk for HIV and other sexually transmitted diseases. Behavioral interventions directed toward MSM need to address bisexual behaviors.

Case-crossover study of partner and situational factors for unprotected sex

Objectives: To identify situational and partner characteristics associated with unprotected sex among women at risk for HIV infection. Methods: The most recent unprotected and protected sexual encounters were compared using a case‐crossover design among 1,647 women enrolled in a prospective cohort study conducted in six U.S. cities. Information collected via audio computerassisted self‐interview included detailed situational and partner characteristics for participants’ most recent sexual encounters, with and without condom use. Paired odds ratios (ORs) and 95% confidence intervals were calculated for situational or partner characteristics that may differ between unprotected and protected sexual encounters, using conditional logistic regression. Results: In the adjusted analysis, partner age of older than 40 years (OR = 2.42), partner type (OR = 2.45 for a “steady” partner), partner use of alcohol (OR = 1.67) and drugs within 2 hours (OR = 1.42) of the sexual encounter, time since the encounters (OR = 0.41, 0.33, and 0.19), and exchange of sex for money or drugs (OR = 0.68) remained significantly associated with women’s most recent unprotected sexual encounter. Conclusions: Considerations related to partner and relationship characteristics should provide information for the development of interventions for women.