Claire Glenton

Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study.

Objective To examine the use of qualitative approaches alongside randomised trials of complex healthcare interventions. Design Review of randomised controlled trials of interventions to change professional practice or the organisation of care. Data sources Systematic sample of 100 trials published in English from the register of the Cochrane Effective Practice and Organisation of Care Review Group. Methods Published and unpublished qualitative studies linked to the randomised controlled trials were identified through database searches and contact with authors. Data were extracted from each study by two reviewers using a standard form. We extracted data describing the randomised controlled trials and qualitative studies, the quality of these studies, and how, if at all, the qualitative and quantitative findings were combined. A narrative synthesis of the findings was done. Results 30 of the 100 trials had associated qualitative work and 19 of these were published studies. 14 qualitative studies were done before the trial, nine during the trial, and four after the trial. 13 studies reported an explicit theoretical basis and 11 specified their methodological approach. Approaches to sampling and data analysis were poorly described. For most cases (n=20) we found no indication of integration of qualitative and quantitative findings at the level of either analysis or interpretation. The quality of the qualitative studies was highly variable. Conclusions Qualitative studies alongside randomised controlled trials remain uncommon, even where relatively complex interventions are being evaluated. Most of the qualitative studies were carried out before or during the trials with few studies used to explain trial results. The findings of the qualitative studies seemed to be poorly integrated with those of the trials and often had major methodological shortcomings.

What about N? A methodological study of sample-size reporting in focus group studies

BackgroundFocus group studies are increasingly published in health related journals, but we know little about how researchers use this method, particularly how they determine the number of focus groups to conduct. The methodological literature commonly advises researchers to follow principles of data saturation, although practical advise on how to do this is lacking. Our objectives were firstly, to describe the current status of sample size in focus group studies reported in health journals. Secondly, to assess whether and how researchers explain the number of focus groups they carry out.MethodsWe searched PubMed for studies that had used focus groups and that had been published in open access journals during 2008, and extracted data on the number of focus groups and on any explanation authors gave for this number. We also did a qualitative assessment of the papers with regard to how number of groups was explained and discussed.ResultsWe identified 220 papers published in 117 journals. In these papers insufficient reporting of sample sizes was common. The number of focus groups conducted varied greatly (mean 8.4, median 5, range 1 to 96). Thirty seven (17%) studies attempted to explain the number of groups. Six studies referred to rules of thumb in the literature, three stated that they were unable to organize more groups for practical reasons, while 28 studies stated that they had reached a point of saturation. Among those stating that they had reached a point of saturation, several appeared not to have followed principles from grounded theory where data collection and analysis is an iterative process until saturation is reached. Studies with high numbers of focus groups did not offer explanations for number of groups. Too much data as a study weakness was not an issue discussed in any of the reviewed papers.ConclusionsBased on these findings we suggest that journals adopt more stringent requirements for focus group method reporting. The often poor and inconsistent reporting seen in these studies may also reflect the lack of clear, evidence-based guidance about deciding on sample size. More empirical research is needed to develop focus group methodology.

Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).

Simon Lewin and colleagues present a methodology for increasing transparency and confidence in qualitative research synthesis.

User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews.

OBJECTIVE To develop a Summary of Findings (SoF) table for use in Cochrane reviews that is understandable and useful for health professionals, acceptable to Cochrane Collaboration stakeholders, and feasible to implement. STUDY DESIGN AND SETTING We gathered stakeholder feedback on the format and content of an SoF table from an advisory group of more than 50 participants and their constituencies through e-mail consultations. We conducted user tests using a think-aloud protocol method, collecting feedback from 21 health professionals and researchers in Norway and the UK. We analyzed the feedback, defined problem areas, and generated new solutions in brainstorming workshops. RESULTS Stakeholders were concerned about precision in the data representation and about production feasibility. User testing revealed unexpected comprehension problems, mainly confusion about what the different numbers referred to (class reference). Resolving the tension between achieving table precision and table simplicity became the main focus of the working group. CONCLUSION User testing led to a table more useful and understandable for clinical audiences. We arrived at an SoF table that was acceptable to the stakeholders and in principle feasible to implement technically. Some challenges remain, including presenting continuous outcomes and technical/editorial implementation.

Presenting the Results of Cochrane Systematic Reviews to a Consumer Audience: A Qualitative Study

Objective. To develop and obtain feedback about a summary format for Cochrane reviews that is accessible to a consumer audience, without oversimplification or incorrect presentation. Methods. We developed 3 versions of a Plain Language Summary (PLS) format of a Cochrane Systematic Review. Using a semi-structured interview guide we tested these versions among 34 members of the public in Norway, Argentina, Canada, and Australia. The authors analyzed feedback, identified problems, and generated new solutions before retesting to produce a final version of a Plain Language Summary format. Results. Participants preferred results presented as words, supplemented by numbers in a table. There was a lack of understanding regarding the difference between a review and an individual study, that the effect is rarely an exact number, that evidence can be of low or high quality, and that level of quality is a separate issue from intervention effect. Participants also found it difficult to move between presentations of dichotomous and continuous outcomes. Rephrasing the introduction helped participants grasp the concept of a review. Confidence intervals were largely ignored or misunderstood. Our attempts to explain them were only partially successful. Text modifiers (‘‘probably,’’ ‘‘may’’) to convey different levels of quality were only partially understood, whereas symbols with explanations were more helpful. Participants often understood individual information elements about effect size and quality of these results, but did not always actively merge these elements. Conclusion. Through testing and iteration the authors identified and addressed several problems, using explanations, rephrasing, and symbols to present scientific concepts. Other problems remain, including how best to present confidence intervals and continuous outcomes. Future research should also test information elements in combination rather than in isolation. The new Plain Language Summary format is being evaluated in a randomized controlled trial.

Developing patient-centred information for back pain sufferers

Objective To identify information needs among a group of back pain sufferers as well as the barriers that may prevent them from accessing this information.

Portals to Wonderland: Health portals lead to confusing information about the effects of health care

BackgroundThe Internet offers a seemingly endless amount of health information of varying quality. Health portals, which provide entry points to quality-controlled collections of websites, have been hailed as a solution to this problem. The objective of this study is to assess the extent to which government-run health portals provide access to relevant, valid and understandable information about the effects of health care.MethodsWe selected eight clinically relevant questions for which there was a systematic review, searched four portals for answers, and compared the answers we found to the results of the systematic reviews.ResultsOur searches resulted in 3400 hits, 155 of which mentioned both the condition and the intervention in one of the eight questions. Sixty-three of the 155 web pages did not give any information about the effect of the intervention. Seventy-seven qualitatively described the effects of the intervention. Twenty-six of these had information that was too unclear to be categorised; 15 were not consistent with the systematic review; and 36 were consistent with the review, but usually did not mention what happens without the intervention, what outcomes have been measured or when they were measured. Fifteen web pages quantitatively described effects. Four of these were abstracts from the systematic review, nine had information that was incomplete and potentially misleading because of a lack of information about people not receiving the intervention and the length of follow-up; one had information that was consistent with the review, but only referred to three trials whereas the review included six; and one was consistent with the review.ConclusionInformation accessible through health portals is unlikely to be based on systematic reviews and is often unclear, incomplete and misleading. Portals are only as good as the websites they lead to. Investments in national health portals are unlikely to benefit consumers without investments in the production and maintenance of relevant, valid and understandable information to which the portals lead.

Can lay health workers increase the uptake of childhood immunisation? Systematic review and typology.

Objectives  Lay health workers (LHWs) are used in many settings to increase immunisation uptake among children. However, little is known about the effectiveness of these interventions. The objective of this review was to assess the effects of LHW interventions on childhood immunisation uptake.

Policymakers' and other stakeholders' perceptions of key considerations for health system decisions and the presentation of evidence to inform those considerations: An international survey

BackgroundThe DECIDE framework was developed to support evidence-informed health system decisions through evidence summaries tailored to health policymakers. The objective of this study was to determine policymakers’ perceptions regarding the criteria in the DECIDE framework and how best to summarise and present evidence to support health system decisions.MethodsWe conducted an online survey of a diverse group of stakeholders with health system decision experience from 15 countries and the World Health Organization. We asked about perceptions of criteria relevant to making health system decisions, use of evidence, grading systems, and evidence summaries.ResultsWe received 112 responses (70% response rate). Most respondents had healthcare (85%) and research (79%) experience. They (99%) indicated that systematic consideration of the available evidence would help to improve health system decision-making processes and supported the use of evidence from other countries (94%) and grading systems (81%). All ten criteria in the DECIDE framework were rated as important in the decision-making process. Respondents had divergent views regarding whether the same (38%) or different (45%) grading systems should be used across different types of health decisions. All components of our evidence summary were rated as important by over 90% of respondents.ConclusionsSurvey respondents were supportive of the DECIDE framework for health system decisions and the use of succinct summaries of the estimated size of effects and the quality of evidence. It is uncertain whether the findings of this survey represent the views of policymakers with little or no healthcare and research experience.

Communicate to vaccinate : the development of a taxonomy of communication interventions to improve routine childhood vaccination

BackgroundVaccination is a cost-effective public health measure and is central to the Millennium Development Goal of reducing child mortality. However, childhood vaccination coverage remains sub-optimal in many settings. While communication is a key feature of vaccination programmes, we are not aware of any comprehensive approach to organising the broad range of communication interventions that can be delivered to parents and communities to improve vaccination coverage. Developing a classification system (taxonomy) organised into conceptually similar categories will aid in: understanding the relationships between different types of communication interventions; facilitating conceptual mapping of these interventions; clarifying the key purposes and features of interventions to aid implementation and evaluation; and identifying areas where evidence is strong and where there are gaps. This paper reports on the development of the ‘Communicate to vaccinate’ taxonomy.MethodsThe taxonomy was developed in two stages. Stage 1 included: 1) forming an advisory group; 2) searching for descriptions of interventions in trials (CENTRAL database) and general health literature (Medline); 3) developing a sampling strategy; 4) screening the search results; 5) developing a data extraction form; and 6) extracting intervention data. Stage 2 included: 1) grouping the interventions according to purpose; 2) holding deliberative forums in English and French with key vaccination stakeholders to gather feedback; 3) conducting a targeted search of grey literature to supplement the taxonomy; 4) finalising the taxonomy based on the input provided.ResultsThe taxonomy includes seven main categories of communication interventions: inform or educate, remind or recall, teach skills, provide support, facilitate decision making, enable communication and enhance community ownership. These categories are broken down into 43 intervention types across three target groups: parents or soon-to-be-parents; communities, community members or volunteers; and health care providers.ConclusionsOur taxonomy illuminates and organises this field and identifies the range of available communication interventions to increase routine childhood vaccination uptake. We have utilised a variety of data sources, capturing information from rigorous evaluations such as randomised trials as well as experiences and knowledge of practitioners and vaccination stakeholders. The taxonomy reflects current public health practice and can guide the future development of vaccination programmes.