David H. Gustafson

Quality collaboratives: lessons from research

Quality improvement collaboratives are increasingly being used in many countries to achieve rapid improvements in health care. However, there is little independent evidence that they are more cost effective than other methods, and little knowledge about how they could be made more effective. A number of systematic evaluations are being performed by researchers in North America, the UK, and Sweden. This paper presents the shared ideas from two meetings of these researchers. The evidence to date is that some collaboratives have stimulated improvements in patient care and organisational performance, but there are significant differences between collaboratives and teams. The researchers agreed on the possible reasons why some were less successful than others, and identified 10 challenges which organisers and teams need to address to achieve improvement. In the absence of more conclusive evidence, these guidelines are likely to be useful for collaborative organisers, teams and their managers and may also contribute to further research into collaboratives and the spread of innovations in health care.

Effect of Computer Support on Younger Women with Breast Cancer

AbstractOBJECTIVE: Assess impact of a computer-based patient support system on quality of life in younger women with breast cancer, with particular emphasis on assisting the underserved. DESIGN: Randomized controlled trial conducted between 1995 and 1998. SETTING: Five sites: two teaching hospitals (Madison, Wis, and Chicago, Ill), two nonteaching hospitals (Chicago, Ill), and a cancer resource center (Indianapolis, Ind). The latter three sites treat many underserved patients. PARTICIPANTS: Newly diagnosed breast cancer patients (N=246) under age 60. INTERVENTIONS: Experimental group received Comprehensive Health Enhancement Support System (CHESS), a home-based computer system providing information, decision-making, and emotional support. MEASUREMENTS AND MAIN RESULTS: Pretest and two posttest surveys (at two- and five-month follow-up) measured aspects of participation in care, social/information support, and quality of life. At two-month follow-up, the CHESS group was significantly more competent at seeking information, more comfortable participating in care, and had greater confidence in doctor(s). At five-month follow-up, the CHESS group had significantly better social support and also greater information competence. In addition, experimental assignment interacted with several indicators of medical underservice (race, education, and lack of insurance), such that CHESS benefits were greater for the disadvantaged than the advantaged group. CONCLUSIONS: Computer-based patient support systems such as CHESS may benefit patients by providing information and social support, and increasing their participation in health care. These benefits may be largest for currently underserved populations.

A Smartphone Application to Support Recovery From Alcoholism A Randomized Clinical Trial

IMPORTANCE Patients leaving residential treatment for alcohol use disorders are not typically offered evidence-based continuing care, although research suggests that continuing care is associated with better outcomes. A smartphone-based application could provide effective continuing care. OBJECTIVE To determine whether patients leaving residential treatment for alcohol use disorders with a smartphone application to support recovery have fewer risky drinking days than control patients. DESIGN, SETTING, AND PARTICIPANTS An unmasked randomized clinical trial involving 3 residential programs operated by 1 nonprofit treatment organization in the Midwestern United States and 2 residential programs operated by 1 nonprofit organization in the Northeastern United States. In total, 349 patients who met the criteria for DSM-IV alcohol dependence when they entered residential treatment were randomized to treatment as usual (n = 179) or treatment as usual plus a smartphone (n = 170) with the Addiction-Comprehensive Health Enhancement Support System (A-CHESS), an application designed to improve continuing care for alcohol use disorders. INTERVENTIONS Treatment as usual varied across programs; none offered patients coordinated continuing care after discharge. A-CHESS provides monitoring, information, communication, and support services to patients, including ways for patients and counselors to stay in contact. The intervention and follow-up period lasted 8 and 4 months, respectively. MAIN OUTCOMES AND MEASURES Risky drinking days--the number of days during which a patients drinking in a 2-hour period exceeded 4 standard drinks for men and 3 standard drinks for women, with standard drink defined as one that contains roughly 14 g of pure alcohol (12 oz of regular beer, 5 oz of wine, or 1.5 oz of distilled spirits). Patients were asked to report their risky drinking days in the previous 30 days on surveys taken 4, 8, and 12 months after discharge from residential treatment. RESULTS For the 8 months of the intervention and 4 months of follow-up, patients in the A-CHESS group reported significantly fewer risky drinking days than did patients in the control group, with a mean of 1.39 vs 2.75 days (mean difference, 1.37; 95% CI, 0.46-2.27; P = .003). CONCLUSIONS AND RELEVANCE The findings suggest that a multifeatured smartphone application may have significant benefit to patients in continuing care for alcohol use disorders. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT01003119.

Experiences of women with breast cancer: exchanging social support over the CHESS computer network.

Using an existential¨phenomenological approach, this paper describes how women with breast cancer experience the giving and receiving of social support in a computer-mediated context. Women viewed their experiences with the computer mediated support group as an additional and unique source of support in facing their illness. Anonymity within the support group fostered equalized participation and allowed women to communicate in ways that would have been more difficult in a face-to-face context. The asynchronous communication was a frustration to some participants, but some indicated that the format allowed for more thoughtful inter action. Motivations for seeking social support appeared to be a dynamic process, with a consistent progression from a position of receiving support to that of giving support. The primary benefits women received from participation in the group were communicating with other people who shared similar problems and helping others, which allowed them to change their focus from a preoccupation with their own sickness to thinking of others. Consistent with past research is the finding that women in this study expressed that social support is a multidimensional phenomenon and that their computer-mediated support group provided abundant emotional support, encouragement, and informational support. Excerpts from the phenomenological interviews are used to review and highlight key theoretical concepts from the research literatures on computer-mediated communication, social support, and the psychosocial needs of women with breast cancer.Using an existential-phenomenological approach, this paper describes how women with breast cancer experience the giving and receiving of social support in a computer-mediated context. Women viewed their experiences with the computer-mediated support group as an additional and unique source of support in facing their illness. Anonymity within the support group fostered equalized participation and allowed women to communicate in ways that would have been more difficult in a face-to-face context. The asynchronous communication was a frustration to some participants, but some indicated that the format allowed for more thoughtful interaction. Motivations for seeking social support appeared to be a dynamic process, with a consistent progression from a position of receiving support to that of giving support. The primary benefits women received from participation in the group were communicating with other people who shared similar problems and helping others, which allowed them to change their focus from a preoccupation with their own sickness to thinking of others. Consistent with past research is the finding that women in this study expressed that social support is a multidimensional phenomenon and that their computer-mediated support group provided abundant emotional support, encouragement, and informational support. Excerpts from the phenomenological interviews are used to review and highlight key theoretical concepts from the research literatures on computer-mediated communication, social support, and the psychosocial needs of women with breast cancer.

A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups☆

Abstract This research evaluated four methods of eliciting subjective likelihood ratio estimates. The methods differed in terms of amount and structure of interaction permitted between estimators. These processes were individual estimates, and three group processes: a Talk-Estimate process approximating an interacting group, an Estimate-Feedback-Estimate process as an approximation of a Delphi group, an Estimate-Talk-Estimate process as combination of nominal and interacting groups. In this study the Estimate-Talk-Estimate group process was superior in approaching correct estimates in this judgmental task. This is consistent with the long research tradition which favors group as opposed to individual problem-solving in judgmental situations. The individual Estimate process and the Estimate-Feedback-Estimate technique performed about equally well with respect to both error and variability. If anything, written feedback appeared to lead to a reduction in the quality of estimates. Finally, the relatively poor results from the Talk-Estimate process are consistent with other studies which have pointed out dysfunctions of interacting group processes for judgmental tasks.

Evaluation of quality improvement programmes

In response to increasing concerns about quality, many countries are carrying out large scale programmes which include national quality strategies, hospital programmes, and quality accreditation, assessment and review processes. Increasing amounts of resources are being devoted to these interventions, but do they ensure or improve quality of care? There is little research evidence as to their effectiveness or the conditions for maximum effectiveness. Reasons for the lack of evaluation research include the methodological challenges of measuring outcomes and attributing causality to these complex, changing, long term social interventions to organisations or health systems, which themselves are complex and changing. However, methods are available which can be used to evaluate these programmes and which can provide decision makers with research based guidance on how to plan and implement them. This paper describes the research challenges, the methods which can be used, and gives examples and guidance for future research. It emphasises the important contribution which such research can make to improving the effectiveness of these programmes and to developing the science of quality improvement.

CHESS: 10 years of research and development in consumer health informatics for broad populations, including the underserved

This paper reviews the research and development around a consumer health informatics system CHESS (The Comprehensive Health Enhancement Support System) developed and tested by the Center for Health Systems Research and Analysis at the University of Wisconsin. The review places particular emphasis on what has been found with regard to the acceptance and use of such systems by high risk and underserved groups.

Assessing the unmet information, support and care delivery needs of men with prostate cancer

This study identified the key Unmet Needs of men with localized prostate cancer. A series of Nominal Groups were used to identify needs, from which a 135-item survey was developed to assess both the Importance and Unmet Need of each item. An Importance-Weighted Unmet Need score was calculated for each item, incorporating both the Importance and the degree to which the need was unmet. Surveys (n=500) were distributed in four geographically distinct areas, with a response rate of 46%. Respondents were 90% Caucasian, 80% married, with a mean age of 66 years, and mean education of 14 years. Care delivery needs were most important and least unmet, while Support needs were least important and most unmet. However, when degree to which needs were unmet was weighted by Importance, information needs had the highest Importance-Weighted Unmet Need scores. The greatest Unmet Needs for information were in knowledge of recurrence issues and in side effects of the illness and its treatment.

Use and Impact of eHealth System by Low-income Women With Breast Cancer

ABSTRACT This article is the second of a two-part series reporting on a population-based study intended to use an eHealth system to examine the feasibility of reaching underserved women with breast cancer (Gustafson, McTavish et al., Reducing the digital divide for low-income women with breast cancer, 2004; Madison Center for Health Systems Research and Analysis, University of Wisconsin; Comprehensive Health Enhancement Support System [CHESS]) and determine how they use the system and what impact it had on them. Participants included women recently diagnosed with breast cancer whose income was at or below 250% of poverty level and were living in rural Wisconsin (n = 144; all Caucasian) or Detroit (n = 85; all African American). Because this was a population-based study all 229 participants received CHESS. A comparison group of patients (n = 51) with similar demographics was drawn from a separate recently completed randomized clinical trial. Use rates (e.g., frequency and length of use as well as type of use) as well as impact on several dimensions of quality of life and participation in health care are reported. Low-income subjects in this study logged on and spent more time on CHESS than more affluent women in a previous study. Urban African Americans used information and analysis services more and communication services less than rural Caucasians. When all low-income women from this study are combined and compared with a low-income control group from another study, the CHESS group was superior to that control group in 4 of 8 outcome variables at both statistically and practically significant levels (social support, negative emotions, participation in health care, and information competence). When African Americans and Caucasians are separated the control groups sample size becomes 30 and 21 thus reducing power. Statistical significance is retained, however, in all four outcomes for Caucasians and in two of four for African Americans. Practical significance is retained for all four outcomes. We conclude that an eHealth system like CHESS will be used extensively and have a positive impact on low-income women with breast cancer.

Effects of a Quality Improvement Collaborative on the Outcome of Care of Patients with HIV Infection: The EQHIV Study

Context Multi-institutional quality improvement collaboratives are popular, but are they effective? Contribution This controlled study evaluated an HIV care Breakthrough Series program that emphasized provider teams, sessions on quality improvement theory and techniques, and report backs about implementing quality improvement. A review of the medical records of 9986 HIV-infected patients showed no important differences in quality of care (viral load suppression, pneumocystitis prophylaxis, and screening for tuberculosis and hepatitis) between the 44 intervention clinics and the 25 control clinics. Cautions Patient adherence and satisfaction were not measured, and some control clinics may have used quality improvement techniques similar to those recommended by the collaborative program. The Editors In the pastdecade, tremendous improvements have occurred in measuring and monitoring the quality of medical care in the United States. Despite these advances, striking problems with quality persist (1, 2). The quality of care for patients with HIV infection is of particular concern. Substantial evidence shows that obtaining medical services and treatment for patients with HIV infection may lead to longer survival and better quality of life (3, 4), yet serious quality-of-care problems and striking disparities in quality by race and social class have been documented (4-6). In the 1980s, continuous quality improvement techniques were introduced into health care (7, 8). These strategies emphasize that most quality problems are a result of system failings rather than problems with individual practitioners (9). In 1995, the Institute for Healthcare Improvement introduced the concept of the Breakthrough Series, which brings together health care organizations dedicated to improving the quality of care in particular clinical areas through the application of continuous quality improvement techniques (10). These techniques (known as Plan/Do/Study/Act or PDSA cycles; Figure 1) first identify deficiencies in quality, next repeatedly implement small-scale interventions and measure changes, and then refine and expand interventions to improve processes of care (11, 12). Typically, each Breakthrough Series collaborative is composed of 20 to 40 participating health care organizations and a faculty with expertise in the clinical area and quality improvement methods (13). To date, the Institute for Healthcare Improvement has conducted collaboratives with more than 700 teams working on 23 clinical conditions or treatment processes, including improving asthma care and reducing medication errors. Although some evaluations of quality before and after a collaborative support the validity of this approach, only a few limited controlled trials have been conducted (14, 15). Figure 1. Theoretical construct of continuous quality improvement. An important source of funding for HIV care is the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act, which is administered by the HIV/AIDS Bureau of the Health Resources and Services Administration. Title III of the CARE Act supports comprehensive primary health care for HIV-infected individuals and currently supports primary care services for more than 150000 patients receiving care in more than 200 community health centers, hospital-based clinics, and city or county health services (16). In 1999, the Health Resources and Services Administration required all clinical sites that were newly awarded funding under Title III of the CARE Act to participate in a quality improvement collaborative conducted by the Institute for Healthcare Improvement. Other sites already receiving Title III funding were also invited to participate. This study evaluates the impact of the collaborative by examining pre- and postimplementation quality-of-care information on samples of patients from both participating and matched nonparticipating clinics. Methods Study Site Selection and Controls Of the 200 Title III sites in the continental United States in May 2000, we excluded 16 that reported caseloads of fewer than 100 cases per year, 12 that were initially slated to participate in the Breakthrough collaborative but elected not to do so, and 1 that lost its CARE Act funding shortly before the collaborative began. Of the remaining 171 sites, 62 participated in the collaborative. Among these sites, 54 agreed to participate in the study and 44 (including 11 mandatory participants and 33 voluntary participants; 71% of collaborative participants) provided chart review data. Of the 109 nonparticipating sites eligible to be selected as control sites, 65 provided information needed for matching. The potential control sites were matched with intervention sites on the basis of the type of site (community health center, community-based organization, health department, hospital, or university medical center), location (rural or urban), number of locations delivering care, region, and number of patients with active HIV infection. When these criteria were used, 40 sites were selected as potential controls and 37 of them (93%) agreed to participate in the study. Of these, 25 (63% of potential control sites) participated in the chart review portion of the study. The Committee on Human Studies of Harvard Medical School approved the study protocol. Quality Improvement Intervention Each participating clinic selected a team, usually consisting of at least one administrator and one or more clinicians, and a population of focus on which the teams interventions would be tested. Usually, the population of focus consisted of all HIV-infected patients in a particular site, but participants sometimes chose to focus on a subset of patients, such as those under the care of a particular group of clinicians. Originally, the design of the collaborative extended for 12 months and included a kickoff meeting and 2 subsequent 2-day meetings called learning sessions. The kickoff learning session included instruction in the theory and practice of quality improvement by identifying problems in HIV care and then introducing the techniques of continuously implementing, measuring, and refining changes (the Plan/Do/Study/Act cycles) (11, 12) to improve the care of HIV-infected patients. Each learning session included additional instruction in quality improvement techniques and breakout sessions that focused on improving specific aspects of care, developing an information infrastructure to track progress, and specific aspects of quality improvement theory. In addition, teams exchanged ideas and presented storyboards of their progress to date. At each session, teams reported on activities, methods, and results. Toward the end of the 12-month period, the Health Resources and Services Administration decided to extend the collaborative by 4 months and add a third learning session. Between the sessions (action periods), team members implemented concepts and ideas. Each site had access to a collaborative listserve, participated in monthly conference calls with the collaborative faculty, and submitted monthly reports of its improvements, which included charts that tracked the sites improvements to date in the required key quality measures described in the next section. Detailed descriptions of the Breakthrough Series collaboratives are available elsewhere (10, 17-19). Quality-of-Care Monitors We selected quality-of-care measures (Table 1) to coincide with required and optional quality measures selected by the collaborative faculty as areas for improvement. These measures were selected by the faculty after reviewing the literature to identify areas of quality deficiency in the delivery of HIV care, particularly for underserved populations targeted by the CARE Act. Because of the paramount importance of antiretroviral therapy to the treatment of HIV infection, the faculty focused on measures related to antiretroviral treatment, including the percentage of patients receiving highly active antiretroviral therapy, the percentage of patients with a controlled viral load, and the percentage of patients who received adherence counseling, as required key measures for the collaborative. Measures were then developed on the basis of consensus guidelines appropriate for the period of care (20). Our primary measures were rates of highly active antiretroviral therapy use and control of HIV viral load for appropriate patients. Patients eligible for highly active antiretroviral therapy included those with CD4+cell counts less than 0.350 109 cells/L, those with CD4+counts between 0.350 and 0.500 109 cells/L and a viral load greater than 5000 copies/mL, all patients with a viral load greater than 30000 copies/mL, and patients already receiving highly active antiretroviral therapy, as per the guidelines. We also assessed the use of highly active antiretroviral therapy for those with CD4+counts less than 0.350 109 cells/L to reflect recommendations that were published after the end of the collaborative (21). Because of the variability in viral load assays available at the time, viral load was considered controlled if it was undetectable or if the total viral load was less than 400 copies/mL. We also assessed the use of screening and prophylaxis, as well as access to care. The only key measure followed by the collaborative that we could not assess was related to adherence counseling because this information is not reliably available from medical records. Table 1. Quality of Care Indicators* Quality-of-Care Data Collection To identify pre- and postintervention samples of patients, we requested lists of all HIV-infected patients in care at each of the sites during the 2 time periods (Figure 2). For the first sample, sites were asked to provide encrypted lists of all HIV-infected patients age 18 years or older as of June 2000 seen at the site between 1 January and 30 June 2000. For the second sample, sites were asked to provide a similar list of active patients age 18 years or older as of Dec