Simon Lewin

Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).

Simon Lewin and colleagues present a methodology for increasing transparency and confidence in qualitative research synthesis.

Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making

Assessment of applicability is an essential part of the systematic review process. In the context of systematic reviews of the effects of interventions, applicability is an assessment of whether the findings of a review can be applied in a particular context or population. For more complex interventions, assessing applicability can be challenging because of greater diversity of, and interactions within and between, the intended population, intervention components, comparison conditions, and outcomes as well as a range of further considerations related to intervention context and theoretical basis. We recommend that review authors plan and conduct analyses to explain variations in effect and answer questions about mechanisms of action and influence of different settings, contexts, and populations. We also recommend that review authors provide rich descriptions of the setting, implementation details, resource use, and contexts of included studies and assess applicability for at least one target population, setting, and context. This should facilitate applicability assessments by end users. Consensus on terminology is needed and guidance should be developed for the synthesis of implementation information within reviews as well as the documentation of applicability judgments by review authors.

Outreach education for integration of HIV/AIDS care, antiretroviral treatment, and tuberculosis care in primary care clinics in South Africa: PALSA PLUS pragmatic cluster randomised trial

Objective To investigate whether PALSA PLUS, an on-site educational outreach programme of non-didactic, case based, iterative clinical education of staff, led by a trainer, can increase access to and comprehensiveness of care for patients with HIV/AIDS. Design Cluster randomised trial. Setting Public primary care clinics offering HIV/AIDS care, antiretroviral treatment (ART), tuberculosis care, and ambulatory primary care in Free State province, South Africa. Participants Fifteen clinics all implementing decentralisation and task shifting were randomised. The clinics cared for 400 000 general primary care patients and 10 136 patients in an HIV/AIDS/ART programme. There were 150 nurses. Intervention On-site outreach education in eight clinics; no such education in seven (control). Main outcome measures Provision of co-trimoxazole prophylaxis among patients referred to the HIV/AIDS/ART programme, and detection of cases of tuberculosis among those in the programme. Proportion of patients in the programme enrolled through general primary care consultations. Results Patients referred to the HIV/AIDS programme through general primary care at intervention clinics were more likely than those at control clinics to receive co-trimoxazole prophylaxis (41%, (2253/5523) v 32% (1340/4210); odds ratio 1.95, 95% confidence interval 1.11 to 3.40), and tuberculosis was more likely to be diagnosed among patients with HIV/AIDS/ART (7% (417/5793) v 6% (245/4343); 1.25, 1.01 to 1.55). Enrolment in the HIV/AIDS and ART programme through HIV testing in general primary care was not significantly increased (53% v 50%; 1.19, 0.51 to 2.77). Secondary outcomes were similar, except for weight gain, which was higher in the intervention group (2.3 kg v 1.9 kg, P<0.001). Conclusion Though outreach education is an effective and feasible strategy for improving comprehensiveness of care and wellbeing of patients with HIV/AIDS, there is no evidence that it increases access to the ART programme. It is now being widely implemented in South Africa. Trial registration Current Controlled Trials ISRCTN 24820584.

Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers

Feasibility studies are increasingly undertaken in preparation for randomised controlled trials in order to explore uncertainties and enable trialists to optimise the intervention or the conduct of the trial. Qualitative research can be used to examine and address key uncertainties prior to a full trial. We present guidance that researchers, research funders and reviewers may wish to consider when assessing or undertaking qualitative research within feasibility studies for randomised controlled trials. The guidance consists of 16 items within five domains: research questions, data collection, analysis, teamwork and reporting. Appropriate and well conducted qualitative research can make an important contribution to feasibility studies for randomised controlled trials. This guidance may help researchers to consider the full range of contributions that qualitative research can make in relation to their particular trial. The guidance may also help researchers and others to reflect on the utility of such qualitative research in practice, so that trial teams can decide when and how best to use these approaches in future studies.

Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series

The GRADE-CERQual (‘Confidence in the Evidence from Reviews of Qualitative research’) approach provides guidance for assessing how much confidence to place in findings from systematic reviews of qualitative research (or qualitative evidence syntheses). The approach has been developed to support the use of findings from qualitative evidence syntheses in decision-making, including guideline development and policy formulation. Confidence in the evidence from qualitative evidence syntheses is an assessment of the extent to which a review finding is a reasonable representation of the phenomenon of interest. CERQual provides a systematic and transparent framework for assessing confidence in individual review findings, based on consideration of four components: (1) methodological limitations, (2) coherence, (3) adequacy of data, and (4) relevance. A fifth component, dissemination (or publication) bias, may also be important and is being explored. As with the GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach for effectiveness evidence, CERQual suggests summarising evidence in succinct, transparent, and informative Summary of Qualitative Findings tables. These tables are designed to communicate the review findings and the CERQual assessment of confidence in each finding. This article is the first of a seven-part series providing guidance on how to apply the CERQual approach. In this paper, we describe the rationale and conceptual basis for CERQual, the aims of the approach, how the approach was developed, and its main components. We also outline the purpose and structure of this series and discuss the growing role for qualitative evidence in decision-making. Papers 3, 4, 5, 6, and 7 in this series discuss each CERQual component, including the rationale for including the component in the approach, how the component is conceptualised, and how it should be assessed. Paper 2 discusses how to make an overall assessment of confidence in a review finding and how to create a Summary of Qualitative Findings table. The series is intended primarily for those undertaking qualitative evidence syntheses or using their findings in decision-making processes but is also relevant to guideline development agencies, primary qualitative researchers, and implementation scientists and practitioners.

Advancing the field of health systems research synthesis

Those planning, managing and working in health systems worldwide routinely need to make decisions regarding strategies to improve health care and promote equity. Systematic reviews of different kinds can be of great help to these decision-makers, providing actionable evidence at every step in the decision-making process. Although there is growing recognition of the importance of systematic reviews to inform both policy decisions and produce guidance for health systems, a number of important methodological and evidence uptake challenges remain and better coordination of existing initiatives is needed. The Alliance for Health Policy and Systems Research, housed within the World Health Organization, convened an Advisory Group on Health Systems Research (HSR) Synthesis to bring together different stakeholders interested in HSR synthesis and its use in decision-making processes. We describe the rationale of the Advisory Group and the six areas of its work and reflects on its role in advancing the field of HSR synthesis. We argue in favour of greater cross-institutional collaborations, as well as capacity strengthening in low- and middle-income countries, to advance the science and practice of health systems research synthesis. We advocate for the integration of quasi-experimental study designs in reviews of effectiveness of health systems intervention and reforms. The Advisory Group also recommends adopting priority-setting approaches for HSR synthesis and increasing the use of findings from systematic reviews in health policy and decision-making.

Fair publication of qualitative research in health systems: a call by health policy and systems researchers

Fair publication of qualitative research in health systems : a call by health policy and systems researchers

Key concepts that people need to understand to assess claims about treatment effects.

People are confronted with claims about the effects of treatments and health policies daily. Our objective was to develop a list of concepts that may be important for people to understand when assessing claims about treatment effects.

Assessing the complexity of interventions within systematic reviews: development, content and use of a new tool (iCAT_SR).

BackgroundHealth interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing ‘complex interventions’, but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews.MethodsWe developed the iCAT_SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool.ResultsThe tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1–6 are considered ‘core’ dimensions. Dimensions 7–10 are optional and may not be useful for all interventions.ConclusionsThe iCAT_SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

Communication strategies to promote the uptake of childhood vaccination in Nigeria: a systematic map

Background Effective communication is a critical component in ensuring that children are fully vaccinated. Although numerous communication interventions have been proposed and implemented in various parts of Nigeria, the range of communication strategies used has not yet been mapped systematically. This study forms part of the ‘Communicate to vaccinate’ (COMMVAC) project, an initiative aimed at building research evidence for improving communication with parents and communities about childhood vaccinations in low- and middle-income countries. Objective This study aims to: 1) identify the communication strategies used in two states in Nigeria; 2) map these strategies against the existing COMMVAC taxonomy, a global taxonomy of vaccination communication interventions; 3) create a specific Nigerian country map of interventions organised by purpose and target; and 4) analyse gaps between the COMMVAC taxonomy and the Nigerian map. Design We conducted the study in two Nigerian states: Bauchi State in Northern Nigeria and Cross River State in Southern Nigeria. We identified vaccination communication interventions through interviews carried out among purposively selected stakeholders in the health services and relevant agencies involved in vaccination information delivery; through observations and through relevant documents. We used the COMMVAC taxonomy to organise the interventions we identified based on the intended purpose of the communication and the group to which the intervention was targeted. Results The Nigerian map revealed that most of the communication strategies identified aimed to inform and educate and remind or recall. Few aimed to teach skills, enhance community ownership, and enable communication. We did not identify any intervention that aimed to provide support or facilitate decision-making. Many interventions had more than one purpose. The main targets for most interventions were caregivers and community members, with few interventions directed at health workers. Most interventions identified were used in the context of campaigns rather than routine immunisation programmes. Conclusions The identification and development of the Nigerian vaccination communication interventions map could assist programme managers to identify gaps in vaccination communication. The map may be a useful tool as part of efforts to address vaccine hesitancy and improve vaccination coverage in Nigeria and similar settings.Background Effective communication is a critical component in ensuring that children are fully vaccinated. Although numerous communication interventions have been proposed and implemented in various parts of Nigeria, the range of communication strategies used has not yet been mapped systematically. This study forms part of the ‘Communicate to vaccinate’ (COMMVAC) project, an initiative aimed at building research evidence for improving communication with parents and communities about childhood vaccinations in low- and middle-income countries. Objective This study aims to: 1) identify the communication strategies used in two states in Nigeria; 2) map these strategies against the existing COMMVAC taxonomy, a global taxonomy of vaccination communication interventions; 3) create a specific Nigerian country map of interventions organised by purpose and target; and 4) analyse gaps between the COMMVAC taxonomy and the Nigerian map. Design We conducted the study in two Nigerian states: Bauchi State in Northern Nigeria and Cross River State in Southern Nigeria. We identified vaccination communication interventions through interviews carried out among purposively selected stakeholders in the health services and relevant agencies involved in vaccination information delivery; through observations and through relevant documents. We used the COMMVAC taxonomy to organise the interventions we identified based on the intended purpose of the communication and the group to which the intervention was targeted. Results The Nigerian map revealed that most of the communication strategies identified aimed to inform and educate and remind or recall. Few aimed to teach skills, enhance community ownership, and enable communication. We did not identify any intervention that aimed to provide support or facilitate decision-making. Many interventions had more than one purpose. The main targets for most interventions were caregivers and community members, with few interventions directed at health workers. Most interventions identified were used in the context of campaigns rather than routine immunisation programmes. Conclusions The identification and development of the Nigerian vaccination communication interventions map could assist programme managers to identify gaps in vaccination communication. The map may be a useful tool as part of efforts to address vaccine hesitancy and improve vaccination coverage in Nigeria and similar settings.