Donald L. Patrick

Measuring Health-Related Quality of Life

What Is Health-Related Quality of Life? Health status, functional status, and quality of life are three concepts often used interchangeably to refer to the same domain of health [1]. The health domain ranges from negatively valued aspects of life, including death, to the more positively valued aspects such as role function or happiness. The boundaries of definition usually depend on why one is assessing health as well as the particular concerns of patients, clinicians, and researchers. We use the term health-related quality of life (HRQL) because widely valued aspects of life exist that are not generally considered as health, including income, freedom, and quality of the environment. Although low or unstable income, the lack of freedom, or a low-quality environment may adversely affect health, these problems are often distant from a health or medical concern. Clinicians focus on HRQL, although when a patient is ill or diseased, almost all aspects of life can become health related. Why Measure HRQL? HRQL is important for measuring the impact of chronic disease [2]. Physiologic measures provide information to clinicians but are of limited interest to patients; they often correlate poorly with functional capacity and well-being, the areas in which patients are most interested and familiar. In patients with chronic heart and lung disease, exercise capacity in the laboratory is only weakly related to exercise capacity in daily life [3]. Another reason to measure HRQL is the commonly observed phenomena that two patients with the same clinical criteria often have dramatically different responses. For example, two patients with the same range of motion and even similar ratings of back pain may have different role function and emotional well-being. Although some patients may continue to work without major depression, others may quit their jobs and have major depression. These considerations explain why patients, clinicians, and health care administrators are all keenly interested in the effects of medical interventions on HRQL [4]. Administrators are particularly interested in HRQL because the case mix of patients affects use and expenditure patterns, because increasing efforts exist to incorporate HRQLs as measures of the quality of care and of clinical effectiveness, and because payers are beginning to use HRQL information in reimbursement decisions. The Structure of HRQL Measures Some HRQL measures consist of a single question that essentially asks How is your quality of life? [5] This question may be asked in a simple or a sophisticated fashion, but either way it yields limited information. More commonly, HRQL instruments are questionnaires made up of a number of items or questions. These items are added up in a number of domains (also sometimes called dimensions). A domain or dimension refers to the area of behavior or experience that we are trying to measure. Domains might include mobility and self-care (which could be further aggregated into physical function), or depression, anxiety, and well-being (which could be aggregated to form an emotional-function domain). For some instruments, investigators do rigorous valuation exercises in which the importance of each item is rated in relation to the others. More often, items are equally weighted, which assumes that their value is equal. Modes of Administration The strengths and weaknesses of the different modes of HRQL administration are summarized in Table 1. Health-related quality-of-life questionnaires are either administered by trained interviewers or self-administered. The former method is resource intensive but ensures compliance, decreases errors, and decreases missing items. The latter approach is much less expensive but increases the number of missing subjects and increases missing responses. A compromise between the two approaches is to have instruments completed with supervision. Another compromise is the phone interview, which decreases errors and decreases missing data but dictates a relatively simple questionnaire structure. Investigators have done initial experiments with computer-administration of HRQL measures, but this is not yet a common method of questionnaire administration. Table 1. Modes of Administration of HRQL Measures Investigators sometimes use a surrogate respondent to predict results that would be obtained from the patient. For instance, McKusker and Stoddard [6] were interested in what patients might score on a general, comprehensive measure of HRQLthe Sickness Impact Profilewhen they were too ill to complete the questionnaire. The investigators used a surrogate to respond on behalf of the patient but wanted assurance that surrogate responses would correspond to what patients would have said had they been capable of answering. They administered the Sickness Impact Profile to terminally ill patients who were still capable of completing the questionnaire and to close relatives of the respondents. The correlation between the two sets of responses was 0.55, and the difference between the two pairs of responses was greater than 6 on a 100-point scale for 50% of the patients. The results provide only moderate support for the validity of surrogate responses to the Sickness Impact Profile. These results are consistent with other evaluations of ratings by patients and proxies. In general, the correspondence between respondent and proxy response to HRQL measures varies depending on the domain assessed and the choice of proxy. Proxy reports of more observable domains, such as physical functioning and cognition, are more highly correlated with reports from the patients themselves. For functional limitations, proxy respondents tend to consider patients more impaired (they overestimate patient dysfunction relative to the patients themselves). This is particularly characteristic of those proxies with the greatest contact with the respondent [7]. For other sorts of morbidity, patients tend to report the most problems, followed by close relatives, and clinicians report the least. These findings have important clinical implications because they suggest that clinicians should concentrate on careful ascertainment of the reported behaviors and perceptions of patients themselves, and they should limit the inferences they make on the basis of the perceptions of the caregivers. What Makes a Good HRQL Instrument? Measuring at a Point in Time versus Measuring Change The goals of HRQL measures include differentiating between people who have a better HRQL and those who have a worse HRQL (a discriminative instrument) as well as measuring how much the HRQL has changed (an evaluative instrument) [8]. The construction of instruments for these two purposes is different. If we want to discriminate between those with and without thyroid disease, we would be unlikely to include fatigue as an item because fatigue is too common among people who do not have thyroid disease. On the other hand, in measuring improvement in HRQL with treatment, fatigue, because of its importance in the daily lives of people with thyroid disease, would be a key item. In the next sections, we list key measurement properties separately for discriminative and evaluative instruments. The properties that make useful discriminative and evaluative instruments are presented in Table 2. Table 2. What Makes a Good HRQL Measure? Signal and Noise Investigators examining physiologic end points know that reproducibility and accuracy are the necessary attributes of a good test. For HRQL instruments, reproducibility means having a high signal-to-noise ratio, and accuracy translates into whether they are really measuring what they intended to measure. For discriminative instruments, the way of quantitating the signal-to-noise ratio is called reliability. If the variability in scores between patients (the signal) is much greater than the variability within patients (the noise), an instrument will be deemed reliable. Reliable instruments will generally show that stable patients have more or less the same results after repeated administration. For evaluative instruments, those designed to measure changes within patients during a period of time, the method of determining the signal-to-noise ratio is called responsiveness. Responsiveness refers to an instruments ability to detect change. If a treatment results in an important difference in HRQL, investigators want to be confident that they will detect that difference, even if it is small. Responsiveness will be directly related to the magnitude of the difference in score in patients who have improved or deteriorated (the signal) and the extent to which patients who have not changed provide more or less the same scores (the noise). Validity When a Gold Standard Exists Although no gold standard for HRQL exists, instances occur in which a specific target for an HRQL measure exists that can be treated as a criterion or gold standard. Under these circumstances, one determines whether an instrument is measuring what is intended using criterion validity (an instrument is valid if its results correspond to those of the criterion standard). Criterion validity is applicable when a shorter version of an instrument (the test) is used to predict the results of the full-length index (the gold standard). Another example is using an HRQL instrument to predict death. In this instance, the instrument will be valid if variability in survival between patients (the gold standard) is explained by the questionnaire results (the test). Self-ratings of health, like more comprehensive and lengthy measures of general health perceptions, include a patients evaluation of physiologic, physical, psychological, and social well-being. Perceived health, measured through self-ratings, is an important predictor of death [9]. Validity When No Gold Standard Exists Validity examines whether the instrument is measuring what it is intended to measure. When no gold or criterion standard exists, HRQL invest

Screening for Depression in Well Older Adults: Evaluation of a Short Form of the CES-D

We derived and tested a short form of the Center for Epidemiologic Studies Depression Scale (CES-D) for reliability and validity among a sample of well older adults in a large Health Maintenance Organization. The 10-item screening questionnaire, the CESD-10, showed good predictive accuracy when compared to the full-length 20-item version of the CES-D (kappa = .97, P < .001). Cutoff scores for depressive symptoms were > or = 16 for the full-length questionnaire and > or = 10 for the 10-item version. We discuss other potential cutoff values. The CESD-10 showed an expected positive correlation with poorer health status scores (r = .37) and a strong negative correlation with positive affect (r = -.63). Retest correlations for the CESD-10 were comparable to those in other studies (r = .71). We administered the CESD-10 again after 12 months, and scores were stable with strong correlation of r = .59.

Generic and disease-specific measures in assessing health status and quality of life.

Application of generic and specific measures of health status and quality of life to different diseases, conditions, states, and populations is increasing. Four strategies for using these measures are separate generic and specific measures, modified generic measures, disease-specific supplements, and batteries. The preferred strategy depends on project aims, methodological concerns, and practical constraints. Generic measures are necessary to compare outcomes across different populations and interventions, particularly for cost-effectiveness studies. Disease-specific measures assess the special states and concerns of diagnostic groups. Specific measures may be more sensitive for the detection and quantification of small changes that are important to clinicians or patients. Comparison studies are needed of the validity, reliability, and responsiveness of generic and disease-specific measures in the same population and in minority and age-specific groups.

The validity of self-reported smoking: a review and meta-analysis.

OBJECTIVES The purpose of this study was to identify circumstances in which biochemical assessments of smoking produce systematically higher or lower estimates of smoking than self-reports. A secondary aim was to evaluate different statistical approaches to analyzing variation in validity estimates. METHODS Literature searches and personal inquiries identified 26 published reports containing 51 comparisons between self-reported behavior and biochemical measures. The sensitivity and specificity of self-reports of smoking were calculated for each study as measures of accuracy. RESULTS Sensitivity ranged from 6% to 100% (mean = 87.5%), and specificity ranged from 33% to 100% (mean = 89.2%). Interviewer-administered questionnaires, observational studies, reports by adults, and biochemical validation with cotinine plasma were associated with higher estimates of sensitivity and specificity. CONCLUSIONS Self-reports of smoking are accurate in most studies. To improve accuracy, biochemical assessment, preferably with cotinine plasma, should be considered in intervention studies and student populations.

Reproducibility and responsiveness of health status measures. Statistics and strategies for evaluation

Before being introduced to wide use, health status instruments should be evaluated for reliability and validity. Increasingly, they are also tested for responsiveness to important clinical changes. Although standards exist for assessing these properties, confusion and inconsistency arise because multiple statistics are used for the same property; controversy exists over how to measure responsiveness; many statistics are unavailable on common software programs; strategies for measuring these properties vary; and it is often unclear how to define a clinically important change in patient status. Using data from a clinical trial of therapy for back pain, we demonstrate the calculation of several statistics for measuring reproducibility and responsiveness, and demonstrate relationships among them. Simple computational guides for several statistics are provided. We conclude that reproducibility should generally be quantified with the intraclass correlation coefficient rather than the more common Pearson r. Assessing reproducibility by retest at one-to-two week intervals (rather than a shorter interval) may result in more realistic estimates of the variability to be observed among control subjects in a longitudinal study. Instrument responsiveness should be quantified using indicators of effect size, a modified effect size statistic proposed by Guyatt, or the use of receiver operating characteristic (ROC) curves to describe how well various score changes can distinguish improved from unimproved patients.

The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study

BackgroundAim of the COSMIN study (COnsensus-based Standards for the selection of health status Measurement INstruments) was to develop a consensus-based checklist to evaluate the methodological quality of studies on measurement properties. We present the COSMIN checklist and the agreement of the panel on the items of the checklist.MethodsA four-round Delphi study was performed with international experts (psychologists, epidemiologists, statisticians and clinicians). Of the 91 invited experts, 57 agreed to participate (63%). Panel members were asked to rate their (dis)agreement with each proposal on a five-point scale. Consensus was considered to be reached when at least 67% of the panel members indicated ‘agree’ or ‘strongly agree’.ResultsConsensus was reached on the inclusion of the following measurement properties: internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing and cross-cultural validity), criterion validity, responsiveness, and interpretability. The latter was not considered a measurement property. The panel also reached consensus on how these properties should be assessed.ConclusionsThe resulting COSMIN checklist could be useful when selecting a measurement instrument, peer-reviewing a manuscript, designing or reporting a study on measurement properties, or for educational purposes.

Assessing health-related quality of life in patients with sciatica.

Study Design This study analyzed health-related quality-of-life measures and other clinical and questionnaire data obtained from the Maine Lumbar Spine Study, a prospective cohort study of persons with low back problems. Objective For persons with sciatica, back pain-specific and general measures of health-related quality-of-life were compared with regard to internal consistency, construct validity, reproducibility, and responsiveness in detecting small changes over a 3-month period. Summary of Background Data Data were collected from 427 participants with sciatica. Baseline in-person interviews were conducted with surgical and medical patients before treatment and by mail at 3 months. Methods Health-related quality-of-life measures included symptoms (frequency and bothersomeness of pain and sciatica) functional status and well-being (modified back pain-specific Roland scale and Medical Outcomes Study 36-item Short Form Health Survey (SF-36), and disability (bed rest, work loss, and restricted activity days). Results Internal consistency of measures was high. Reproducibility was moderate, as expected after a 3-month interval. The SF-36 bodily pain item and the modified Roland measure demonstrated the greatest amount of change and were the most highly associated with self-rated improvement. The specific and generic measures changed in the expected direction, except for general health perceptions, which declined slightly. A high correlation between clinical findings or symptoms and the modified Roland measure, SF-36, and disability days indicated a high degree of construct validity. Conclusions These measures performed well in measuring the health-related quality-of-life of patients with sciatica. The modified Roland and the physical dimension of the SF-36 were the measures most responsive to change over time, suggesting their use in prospective evaluation. Disability day measures, although valuable for assessing the societal impact of dysfunction, were less responsive to changes over this short-term follow-up of 3 months.

The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content.

BackgroundThe COSMIN checklist (COnsensus-based Standards for the selection of health status Measurement INstruments) was developed in an international Delphi study to evaluate the methodological quality of studies on measurement properties of health-related patient reported outcomes (HR-PROs). In this paper, we explain our choices for the design requirements and preferred statistical methods for which no evidence is available in the literature or on which the Delphi panel members had substantial discussion.MethodsThe issues described in this paper are a reflection of the Delphi process in which 43 panel members participated.ResultsThe topics discussed are internal consistency (relevance for reflective and formative models, and distinction with unidimensionality), content validity (judging relevance and comprehensiveness), hypotheses testing as an aspect of construct validity (specificity of hypotheses), criterion validity (relevance for PROs), and responsiveness (concept and relation to validity, and (in) appropriate measures).ConclusionsWe expect that this paper will contribute to a better understanding of the rationale behind the items, thereby enhancing the acceptance and use of the COSMIN checklist.

Evaluating quality-of-life and health status instruments: development of scientific review criteria

The Medical Outcomes Trust is a depository and distributor of high-quality, standardized, health outcomes measurement instruments to national and international health communities. Every instrument in the Trust library is reviewed by the Scientific Advisory Committee against a rigorous set of eight attributes. These attributes consist of the following: (1) conceptual and measurement model; (2) reliability; (3) validity; (4) responsiveness; (5) interpretability; (6) respondent and administrative burden; (7) alternative forms; and (8) cultural and language adaptations. In addition to a full description of each attribute, we discuss uses of these criteria beyond evaluation of existing instruments and lessons learned in the first few rounds of instrument review against these criteria.

A Prospective Study of Advance Directives for Life-Sustaining Care

BACKGROUND The use of advance directives is recommended so that people can determine the medical care they will receive when they are no longer competent, but the effectiveness of such directives is not clear. METHODS In a prospective study conducted over a two-year period, 126 competent residents of a nursing home and 49 family members of incompetent patients were interviewed to determine their preferences with respect to hospitalization, intensive care, cardiopulmonary resuscitation, artificial ventilation, surgery, and tube feeding in the event of critical illness, terminal illness, or permanent unconsciousness. Advance directives, consisting of signed statements of treatment preferences, were placed in the medical record to assist in care in the nursing home and to be forwarded to the hospital if necessary. RESULTS In an analysis of 96 outcome events (hospitalization or death in the nursing home), care was consistent with previously expressed wishes 75 percent of the time; however, the presence of the written advance directive in the medical record did not facilitate consistency. Among the 24 events in which inconsistencies occurred, care was provided more aggressively than had been requested in 6 cases, largely because of unanticipated surgery or artificial ventilation, and less aggressively than requested in 18, largely because hospitalization or cardiopulmonary resuscitation was withheld. Inconsistencies were more likely in the nursing home than in the hospital. CONCLUSIONS. The effectiveness of written advance directives is limited by inattention to them and by decisions to place priority on considerations other than the patients autonomy. Since our study was performed in only one nursing home and one hospital, other studies are necessary to determine the generalizability of our findings.