Keith McInnes

Quality collaboratives: lessons from research

Quality improvement collaboratives are increasingly being used in many countries to achieve rapid improvements in health care. However, there is little independent evidence that they are more cost effective than other methods, and little knowledge about how they could be made more effective. A number of systematic evaluations are being performed by researchers in North America, the UK, and Sweden. This paper presents the shared ideas from two meetings of these researchers. The evidence to date is that some collaboratives have stimulated improvements in patient care and organisational performance, but there are significant differences between collaboratives and teams. The researchers agreed on the possible reasons why some were less successful than others, and identified 10 challenges which organisers and teams need to address to achieve improvement. In the absence of more conclusive evidence, these guidelines are likely to be useful for collaborative organisers, teams and their managers and may also contribute to further research into collaboratives and the spread of innovations in health care.

Effects of a Quality Improvement Collaborative on the Outcome of Care of Patients with HIV Infection: The EQHIV Study

Context Multi-institutional quality improvement collaboratives are popular, but are they effective? Contribution This controlled study evaluated an HIV care Breakthrough Series program that emphasized provider teams, sessions on quality improvement theory and techniques, and report backs about implementing quality improvement. A review of the medical records of 9986 HIV-infected patients showed no important differences in quality of care (viral load suppression, pneumocystitis prophylaxis, and screening for tuberculosis and hepatitis) between the 44 intervention clinics and the 25 control clinics. Cautions Patient adherence and satisfaction were not measured, and some control clinics may have used quality improvement techniques similar to those recommended by the collaborative program. The Editors In the pastdecade, tremendous improvements have occurred in measuring and monitoring the quality of medical care in the United States. Despite these advances, striking problems with quality persist (1, 2). The quality of care for patients with HIV infection is of particular concern. Substantial evidence shows that obtaining medical services and treatment for patients with HIV infection may lead to longer survival and better quality of life (3, 4), yet serious quality-of-care problems and striking disparities in quality by race and social class have been documented (4-6). In the 1980s, continuous quality improvement techniques were introduced into health care (7, 8). These strategies emphasize that most quality problems are a result of system failings rather than problems with individual practitioners (9). In 1995, the Institute for Healthcare Improvement introduced the concept of the Breakthrough Series, which brings together health care organizations dedicated to improving the quality of care in particular clinical areas through the application of continuous quality improvement techniques (10). These techniques (known as Plan/Do/Study/Act or PDSA cycles; Figure 1) first identify deficiencies in quality, next repeatedly implement small-scale interventions and measure changes, and then refine and expand interventions to improve processes of care (11, 12). Typically, each Breakthrough Series collaborative is composed of 20 to 40 participating health care organizations and a faculty with expertise in the clinical area and quality improvement methods (13). To date, the Institute for Healthcare Improvement has conducted collaboratives with more than 700 teams working on 23 clinical conditions or treatment processes, including improving asthma care and reducing medication errors. Although some evaluations of quality before and after a collaborative support the validity of this approach, only a few limited controlled trials have been conducted (14, 15). Figure 1. Theoretical construct of continuous quality improvement. An important source of funding for HIV care is the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act, which is administered by the HIV/AIDS Bureau of the Health Resources and Services Administration. Title III of the CARE Act supports comprehensive primary health care for HIV-infected individuals and currently supports primary care services for more than 150000 patients receiving care in more than 200 community health centers, hospital-based clinics, and city or county health services (16). In 1999, the Health Resources and Services Administration required all clinical sites that were newly awarded funding under Title III of the CARE Act to participate in a quality improvement collaborative conducted by the Institute for Healthcare Improvement. Other sites already receiving Title III funding were also invited to participate. This study evaluates the impact of the collaborative by examining pre- and postimplementation quality-of-care information on samples of patients from both participating and matched nonparticipating clinics. Methods Study Site Selection and Controls Of the 200 Title III sites in the continental United States in May 2000, we excluded 16 that reported caseloads of fewer than 100 cases per year, 12 that were initially slated to participate in the Breakthrough collaborative but elected not to do so, and 1 that lost its CARE Act funding shortly before the collaborative began. Of the remaining 171 sites, 62 participated in the collaborative. Among these sites, 54 agreed to participate in the study and 44 (including 11 mandatory participants and 33 voluntary participants; 71% of collaborative participants) provided chart review data. Of the 109 nonparticipating sites eligible to be selected as control sites, 65 provided information needed for matching. The potential control sites were matched with intervention sites on the basis of the type of site (community health center, community-based organization, health department, hospital, or university medical center), location (rural or urban), number of locations delivering care, region, and number of patients with active HIV infection. When these criteria were used, 40 sites were selected as potential controls and 37 of them (93%) agreed to participate in the study. Of these, 25 (63% of potential control sites) participated in the chart review portion of the study. The Committee on Human Studies of Harvard Medical School approved the study protocol. Quality Improvement Intervention Each participating clinic selected a team, usually consisting of at least one administrator and one or more clinicians, and a population of focus on which the teams interventions would be tested. Usually, the population of focus consisted of all HIV-infected patients in a particular site, but participants sometimes chose to focus on a subset of patients, such as those under the care of a particular group of clinicians. Originally, the design of the collaborative extended for 12 months and included a kickoff meeting and 2 subsequent 2-day meetings called learning sessions. The kickoff learning session included instruction in the theory and practice of quality improvement by identifying problems in HIV care and then introducing the techniques of continuously implementing, measuring, and refining changes (the Plan/Do/Study/Act cycles) (11, 12) to improve the care of HIV-infected patients. Each learning session included additional instruction in quality improvement techniques and breakout sessions that focused on improving specific aspects of care, developing an information infrastructure to track progress, and specific aspects of quality improvement theory. In addition, teams exchanged ideas and presented storyboards of their progress to date. At each session, teams reported on activities, methods, and results. Toward the end of the 12-month period, the Health Resources and Services Administration decided to extend the collaborative by 4 months and add a third learning session. Between the sessions (action periods), team members implemented concepts and ideas. Each site had access to a collaborative listserve, participated in monthly conference calls with the collaborative faculty, and submitted monthly reports of its improvements, which included charts that tracked the sites improvements to date in the required key quality measures described in the next section. Detailed descriptions of the Breakthrough Series collaboratives are available elsewhere (10, 17-19). Quality-of-Care Monitors We selected quality-of-care measures (Table 1) to coincide with required and optional quality measures selected by the collaborative faculty as areas for improvement. These measures were selected by the faculty after reviewing the literature to identify areas of quality deficiency in the delivery of HIV care, particularly for underserved populations targeted by the CARE Act. Because of the paramount importance of antiretroviral therapy to the treatment of HIV infection, the faculty focused on measures related to antiretroviral treatment, including the percentage of patients receiving highly active antiretroviral therapy, the percentage of patients with a controlled viral load, and the percentage of patients who received adherence counseling, as required key measures for the collaborative. Measures were then developed on the basis of consensus guidelines appropriate for the period of care (20). Our primary measures were rates of highly active antiretroviral therapy use and control of HIV viral load for appropriate patients. Patients eligible for highly active antiretroviral therapy included those with CD4+cell counts less than 0.350 109 cells/L, those with CD4+counts between 0.350 and 0.500 109 cells/L and a viral load greater than 5000 copies/mL, all patients with a viral load greater than 30000 copies/mL, and patients already receiving highly active antiretroviral therapy, as per the guidelines. We also assessed the use of highly active antiretroviral therapy for those with CD4+counts less than 0.350 109 cells/L to reflect recommendations that were published after the end of the collaborative (21). Because of the variability in viral load assays available at the time, viral load was considered controlled if it was undetectable or if the total viral load was less than 400 copies/mL. We also assessed the use of screening and prophylaxis, as well as access to care. The only key measure followed by the collaborative that we could not assess was related to adherence counseling because this information is not reliably available from medical records. Table 1. Quality of Care Indicators* Quality-of-Care Data Collection To identify pre- and postintervention samples of patients, we requested lists of all HIV-infected patients in care at each of the sites during the 2 time periods (Figure 2). For the first sample, sites were asked to provide encrypted lists of all HIV-infected patients age 18 years or older as of June 2000 seen at the site between 1 January and 30 June 2000. For the second sample, sites were asked to provide a similar list of active patients age 18 years or older as of Dec

Science-based policy for psychosocial interventions in refugee camps: a Cambodian example.

No longer are the high rates of psychiatric morbidity associated with mass violence in refugee populations invisible to the humanitarian assistance community. However, identification of mental health risk and protective factors that can be utilized by policy planners is still lacking. The objective of this report is to provide an analytic approach to determining these factors. A description is provided from the first large-scale epidemiological study of Cambodian refugees confined to the Thailand-Cambodian border in the 1980s and 1990s. The original data from this study are reanalyzed to evaluate the mental health impact of psychosocial factors subject to the influence of camp authorities, such as opportunities in the refugee camp environment and personal behaviors, in addition to trauma. The results suggest the extraordinary capacity of refugees to protect themselves against mental illness despite horrific life experiences. The recommendation emerges for refugee policy makers to create programs that support work, indigenous religious practices, and culture-based altruistic behavior among refugees. As refugee mental health policy receives increasing attention from the international community, it must consist of recommendations and practices based on scientific analysis and empirical evidence.

Correlations among measures of quality in HIV care in the United States: cross sectional study

Objective To determine whether a selected set of indicators can represent a single overall quality construct. Design Cross sectional study of data abstracted during an evaluation of an initiative to improve quality of care for people with HIV. Setting 69 sites in 30 states. Data sources Medical records of 9020 patients. Main outcome measures Adjusted performance rates at site level for eight measures of quality of care specific to HIV and a site level summary performance score (the number of measures for which the site was in the top quarter of the distribution). Results Of 28 site level correlations between measures, two were greater than 0.40, two were between 0.30 and 0.39, four were between 0.20 and 0.29, and the 20 remaining were all less than 0.20. One site was in the top quarter for seven measures, but no sites were in the top quarter for six or eight of the measures. Across the eight quality measures, sites were in the top quarter no more often than predicted by a chance (binomial) distribution. Conclusions The quality suggested by one measured indicator cannot necessarily be generalised to unmeasured indicators, even if this might be expected for clinical or other reasons.

Differences in patient and clinic characteristics at CARE Act funded versus non-CARE Act funded HIV clinics.

The Ryan White CARE Act supports comprehensive care to persons with HIV infection. With an annual budget of over

Associations between organizational characteristics and quality improvement activities of clinics participating in a quality improvement collaborative.

1 billion, it is the largest federally funded programme for HIV care in the USA. We analysed data from the HIV Costs and Services Utilization Study, a nationally representative sample of HIV patients. Patient data were collected in 1996–97 and clinic data were collected in 1998–99. We examined whether CARE Act funded clinics differed from other HIV clinics in (1) the characteristics of their patients, and (2) their organization, staffing, and services. We found that patients at CARE Act clinics were younger, less educated, poorer, and more likely to be female, non-white, unemployed, uninsured, and have heterosexual contact as an HIV risk factor, compared to patients at other HIV clinics. CARE Act clinics tended to specialize in HIV care, had more infectious disease specialists, had fewer total patients, and provided more support services (e.g. mental health, nutrition, case management, child care). These results are consistent with findings of other studies that were limited by non-probability samples or restricted geographical areas.

Disability Associated With Psychiatric Comorbidity and Health Status in Bosnian Refugees Living in Croatia

Background:Few studies have rigorously evaluated the associations between organizational characteristics and intervention activities of health care organizations participating in quality improvement collaboratives (QICs). Objective:To examine the relationship between clinic characteristics and intervention activities by primary care clinics that provide HIV care and that participated in a QIC. Design:Cross-sectional study of Ryan White CARE Act (now called Ryan White HIV/AIDS Treatment Modernization Act) funded clinics that participated in a QIC over 16 months in 2000 and 2001. The QIC was originally planned to be a more typical 12 months long, but was extended to increase the likelihood of success. Data were collected using surveys of clinicians and administrators in participating clinics and monthly reports of clinic improvement activities. Measures:Number of interventions attempted, percent of interventions repeated, percent of interventions evaluated, and organizational characteristics. Results:Clinics varied significantly in their intervention choices. Organizations with a more open culture and a greater emphasis on quality improvement attempted more interventions (P < 0.01, P < 0.05) and interventions that were more comprehensive (P < 0.01, P < 0.10). Presence of multidisciplinary teams and measurement of progress toward quantifiable goals also were associated with comprehensiveness of interventions (P < 0.01, P < 0.05). Conclusion:Clinic characteristics predicted intervention activities during a QIC. Further research is needed on how these organizational characteristics affect quality of care through their influence on intervention activities.