Carol Cosenza

Development of instruments to measure the quality of breast cancer treatment decisions

Background  Women with early‐stage breast cancer face a multitude of decisions. The quality of a decision can be measured by the extent to which the treatment reflects what is most important to an informed patient. Reliable and valid measures of patients’ knowledge and their goals and concerns related to breast cancer treatments are needed to assess the decision quality.

Decision quality instrument for treatment of hip and knee osteoarthritis: a psychometric evaluation

BackgroundA high quality decision requires that patients who meet clinical criteria for surgery are informed about the options (including non-surgical alternatives) and receive treatments that match their goals. The aim of this study was to evaluate the psychometric properties and clinical sensibility of a patient self report instrument, to measure the quality of decisions about total joint replacement for knee or hip osteoarthritis.MethodsThe performance of the Hip/Knee Osteoarthritis Decision Quality Instrument (HK-DQI) was evaluated in two samples: (1) a cross-sectional mail survey with 489 patients and 77 providers (study 1); and (2) a randomized controlled trial of a patient decision aid with 138 osteoarthritis patients considering total joint replacement (study 2). The HK-DQI results in two scores. Knowledge items are summed to create a total knowledge score, and a set of goals and concerns are used in a logistic regression model to develop a concordance score. The concordance score measures the proportion of patients whose treatment matched their goals. Hypotheses related to acceptability, feasibility, reliability and validity of the knowledge and concordance scores were examined.ResultsIn study 1, the HK-DQI was completed by 382 patients (79%) and 45 providers (58%), and in study 2 by 127 patients (92%), with low rates of missing data. The DQI-knowledge score was reproducible (ICC = 0.81) and demonstrated discriminant validity (68% decision aid vs. 54% control, and 78% providers vs. 61% patients) and content validity. The concordance score demonstrated predictive validity, as patients whose treatments were concordant with their goals had more confidence and less regret with their decision compared to those who did not.ConclusionsThe HK-DQI is feasible and acceptable to patients. It can be used to assess whether patients with osteoarthritis are making informed decisions about surgery that are concordant with their goals.

Measuring decision quality: psychometric evaluation of a new instrument for breast cancer surgery

BackgroundThe purpose of this paper is to examine the acceptability, feasibility, reliability and validity of a new decision quality instrument that assesses the extent to which patients are informed and receive treatments that match their goals.MethodsCross-sectional mail survey of recent breast cancer survivors, providers and healthy controls and a retest survey of survivors. The decision quality instrument includes knowledge questions and a set of goals, and results in two scores: a breast cancer surgery knowledge score and a concordance score, which reflects the percentage of patients who received treatments that match their goals. Hypotheses related to acceptability, feasibility, discriminant validity, content validity, predictive validity and retest reliability of the survey instrument were examined.ResultsWe had responses from 440 eligible patients, 88 providers and 35 healthy controls. The decision quality instrument was feasible to implement in this study, with low missing data. The knowledge score had good retest reliability (intraclass correlation coefficient = 0.70) and discriminated between providers and patients (mean difference 35%, p < 0.001). The majority of providers felt that the knowledge items covered content that was essential for the decision. Five of the 6 treatment goals met targets for content validity. The five goals had moderate to strong retest reliability (0.64 to 0.87). The concordance score was 89%, indicating that a majority had treatments concordant with that predicted by their goals. Patients who had concordant treatment had similar levels of confidence and regret as those who did not.ConclusionsThe decision quality instrument met the criteria of feasibility, reliability, discriminant and content validity in this sample. Additional research to examine performance of the instrument in prospective studies and more diverse populations is needed.

Giving voice to the vulnerable: the development of a CAHPS nursing home survey measuring family members' experiences.

Background:To complement a nursing home resident survey, the team developed a survey asking family members about their experiences with nursing homes. Although a family member does not receive care directly from a nursing home, their experiences at the nursing home and with staff can contribute to understanding nursing home quality. Objectives:To describe how the nursing home family member instrument was developed, refined, tested, and finalized. Research Design:The team developed a draft survey using information from a literature review, 12 focus groups with family members involved in choosing a nursing home for someone, review of nursing home surveys, and expert/stakeholder input. The survey went through 2 rounds of cognitive interviews (n=54) and revisions and was fielded in 15 nursing homes. Data from the pilot survey (n=885) were subjected to psychometric analyses to evaluate the measurement properties of items as well as the reliability and validity of the resulting composites. On the basis of these analyses and input from experts, the survey was finalized. Results:Focus groups and experts provided input into discerning important indicators of quality, although in some cases family members were not the best sources of information. Cognitive testing refined the survey and eliminated some of the proxy items. The field test analysis and input from experts eliminated 10 items. The final survey included 21 items organized into 4 composites. Conclusions:This survey measures family members’ experiences of nursing home care, and the results contribute to the understanding of quality of care in nursing homes.

Psychometric Evaluation of a Decision Quality Instrument for Treatment of Lumbar Herniated Disc

Study Design. Retrospective and prospective patient surveys and a physician survey using a sample from American Medical Association master file. Objective. To evaluate the performance of a new instrument designed to measure the quality of decisions about treatment of herniated disc. Summary of Background Data. There is growing consensus on the importance of engaging and informing patients to improve the quality of significant medical decisions, yet there are no instruments currently available to measure decision quality. Methods. The herniated disc–decision quality instrument (HD-DQI) was developed with input from clinical experts, survey research experts, and patients. The HD-DQI produces 2 scores each scaled to 0% to 100%, with higher scores indicating better quality: (1) a total knowledge score and (2) a concordance score (indicating the percentage of patients who received treatments that matched their goals). We examined hypotheses relating to the acceptability, feasibility, validity, and reliability of the instrument, using data from 3 samples. Results. The HD-DQI survey was feasible to implement and acceptable to patients, with good response rates and low missing data. The knowledge score discriminated between patients who had seen a decision aid or no decision aid (55% vs. 38%, P < 0.001) and between providers and patients (73% vs. 46%, P < 0.001). The knowledge score also had good retest reliability (intraclass correlation coefficient = 0.85). Most patients (78%) received treatments that matched their goals. Patients who received treatments that matched their goals were less likely to regret the decision than those who did not (13% vs. 39%, P = 0.004). Conclusion. The HD-DQI met several criteria for high-quality patient-reported survey instruments. It can be used to determine the quality of decisions for treatment of herniated disc. More work is needed to examine acceptability for use as part of routine patient care.

Preparing to measure health coverage in federal surveys post-reform: lessons from Massachusetts.

In preparation for health reform in 2014, qualitative research was conducted with Massachusetts residents to explore how to adapt surveys to accommodate reporting information about health exchanges. Questions about exchange participation were effective when state-specific exchange program names were offered, but generic terms such as “marketplace” and “exchange” did not resonate with respondents. However, respondents were able to understand new questions about premiums and subsidies and to answer with a high degree of accuracy. These questions, taken in tandem with answers on plan type, were sufficient to distinguish among Medicaid, subsidized exchange coverage, and unsubsidized coverage, even without the benefit of state-specific exchange program names.

Development and Evaluation of CAHPS Questions to Assess the Impact of Health Information Technology on Patient Experiences with Ambulatory Care

Background:Little is known about whether health information technology (HIT) affects patient experiences with health care. Objective:To develop HIT questions that assess patients care experiences not evaluated by existing ambulatory Consumer Assessment of Health Plans and Systems (CAHPS) measures. Research Design:We reviewed published articles and conducted focus groups and cognitive testing to develop survey questions. We collected data, using mail and the internet, from patients of 69 physicians receiving care at an academic medical center and 2 regional integrated delivery systems in late 2009 and 2010. We evaluated questions and scales about HIT using factor analysis, item-scale correlations, and reliability (internal consistency and physician-level) estimates. Results:We found support for 3 HIT composites: doctor use of computer (2 items), e-mail (2 items), and helpfulness of provider’s website (4 items). Corrected item-scale correlations were 0.37 for the 2 doctor use of computer items and 0.71 for the 2 e-mail items, and ranged from 0.50 to 0.60 for the provider’s website items. Cronbach &agr; was high for e-mail (0.83) and provider’s website (0.75), but only 0.54 for doctor use of computer. As few as 50 responses per physician would yield reliability of 0.70 for e-mail and provider’s website. Two HIT composites, doctor use of computer (P<0.001) and provider’s website (P=0.02), were independent predictors of overall ratings of doctors. Conclusions:New CAHPS HIT items were identified that measure aspects of patient experiences not assessed by the CAHPS C&G 1.0 survey.

Coding Cognitive Interviews An Approach to Enhancing the Value of Cognitive Testing for Survey Question Evaluation

Cognitive testing has become routine for well-designed surveys. However, the protocols for cognitive testing vary widely, and observers have been concerned that analysis of the results is not systematic, that results are not replicable, and that the bases for conclusions are not transparent. To address some of those concerns, in this study the cognitive interviews were structured so that interviewers gathered information specifically about whether there was a problem with any of the four standard issues in question answering: comprehension, retrieval, estimation, and providing an answer. From the tape-recorded interviews, the percentage of respondents who had “no problem” with each element of question answering was tabulated for each question. In addition to being systematic and transparent, a strength of the approach is that it documents the positive features of questions, potentially providing evidence for the validity of answers as well as identifying possible question problems.

Decisions about Breast Reconstruction after Mastectomy: Patient Involvement, Knowledge, and Preferences.

Background: Most breast cancer patients who have a mastectomy do not have breast reconstruction, and rates of reconstruction vary by race, education, and geographic location, suggesting problems with decision making. We sought to assess the quality of decisions about breast reconstruction by measuring patient involvement in decision making, patient knowledge, and the degree to which decisions reflected patients9 goals. Methods: Breast cancer survivors from four sites who were treated with mastectomy in the past 3 years completed a mailed survey, as part of a larger study to validate decision quality instruments. The survey contained questions about the decision making process, factual questions, and questions about personal goals and concerns. Characteristics associated with knowledge were identified with linear regression. Goals/concerns associated with reconstruction were identified using logistic regression. The percent match between treatment preference and treatment received was calculated. Results: The larger study recruited 456 patients (overall response rate 59%). 91 patients completed the reconstruction module. Average age was 56.9 years, 82.6% were white, 63.7% had a college degree, and 64% had Stage I disease. 45.8% had reconstruction.Decision making: 78% of patients reported that their doctor mentioned reconstruction. Most reported a discussion of the pros of reconstruction (63.8%), whereas the minority reported a discussion of the cons (20.9%). 76% reported being asked for their preference about reconstruction. 3% said the doctor mainly made the decision, 74% said they made the decision, and 15% said both made the decision. Most (81%) felt their level of involvement was about right.Knowledge:The mean knowledge score was 32.9% (SD=19). 41% knew that reconstruction has little effect on cancer surveillance. 54% knew that recovery after implant surgery is easier than after flap surgery. 3.3% knew that about 1/3 of patients have a major complication. On bivariate analysis, reconstruction (43.3 vs. 32.6, p=0.053), higher income (43.4 vs. 26.3, p=0.008), a college degree (43.4 vs. 26.2, p Conclusions: Despite reporting high involvement in decisions about reconstruction, breast cancer patients undergoing mastectomy had major knowledge deficits, and many reported having treatment they did not prefer. In addition to involving patients in decisions about reconstruction, surgeons should discuss both the pros and the cons and should explicitly ask patients for their preference about reconstruction. Citation Information: Cancer Res 2009;69(24 Suppl):Abstract nr 3103.