Rebecca Rees

Integrating qualitative research with trials in systematic reviews

An example review from public health shows how integration is possible and some potential benefits The value of including data from different types of studies in systematic reviews of health interventions is increasingly recognised. A recent editorial accepted that qualitative research should be included in systematic reviews, but pointed to a “daunting array of theoretical and practical problems.”1 This article presents an approach to combining qualitative and quantitative research in a systematic review. We describe how we used this approach in a systematic review of interventions to promote healthy eating among children, full details of which are available.2 The review question was: “What is known about the barriers to, and facilitators of, healthy eating among children aged 4-10 years?” The specific focus of the review was fruit and vegetable intake. We searched for two types of research: controlled trials (randomised or non-randomised) that examined interventions to promote healthy eating and studies that examined childrens perspectives and understandings (views studies), often by using qualitative research methods—for example, in-depth interviews and focus groups. ![][1] But will she eat her greens? Credit: PAUL DARRAH/REX We used conventional systematic review methods: sensitive searching, systematic screening, and independent quality assessment. These methods found 33 trials and eight qualitative studies that met our prespecified inclusion criteria. We assessed studies for quality and reliability according to standards for their specific study types; they were then synthesised individually by using methods appropriate to the study. We conducted a meta-analysis with the data extracted from trials, used qualitative methods to synthesise the textual data extracted from the qualitative studies, and then integrated the findings from the qualitative synthesis with those from the meta-analysis. This gave us one review with three syntheses (fig 1). Fig 1 Stages of the review We maintained the key principles of avoiding bias and maximising … [1]: /embed/graphic-1.gif

Acupuncture for chronic headache in primary care: large, pragmatic, randomised trial

Abstract Objective To determine the effects of a policy of “use acupuncture” on headache, health status, days off sick, and use of resources in patients with chronic headache compared with a policy of “avoid acupuncture.” Design Randomised, controlled trial. Setting General practices in England and Wales. Participants 401 patients with chronic headache, predominantly migraine. Interventions Patients were randomly allocated to receive up to 12 acupuncture treatments over three months or to a control intervention offering usual care. Main outcome measures Headache score, SF-36 health status, and use of medication were assessed at baseline, three, and 12 months. Use of resources was assessed every three months. Results Headache score at 12 months, the primary end point, was lower in the acupuncture group (16.2, SD 13.7, n = 161, 34% reduction from baseline) than in controls (22.3, SD 17.0, n = 140, 16% reduction from baseline). The adjusted difference between means is 4.6 (95% confidence interval 2.2 to 7.0; P = 0.0002). This result is robust to sensitivity analysis incorporating imputation for missing data. Patients in the acupuncture group experienced the equivalent of 22 fewer days of headache per year (8 to 38). SF-36 data favoured acupuncture, although differences reached significance only for physical role functioning, energy, and change in health. Compared with controls, patients randomised to acupuncture used 15% less medication (P = 0.02), made 25% fewer visits to general practitioners (P = 0.10), and took 15% fewer days off sick (P = 0.2). Conclusions Acupuncture leads to persisting, clinically relevant benefits for primary care patients with chronic headache, particularly migraine. Expansion of NHS acupuncture services should be considered.

Applying systematic review methods to studies of people's views: an example from public health research

Methods for systematic reviews are well developed for trials, but not for non-experimental or qualitative research. This paper describes the methods developed for reviewing research on people’s perspectives and experiences (“views” studies) alongside trials within a series of reviews on young people’s mental health, physical activity, and healthy eating. Reports of views studies were difficult to locate; could not easily be classified as “qualitative” or “quantitative”; and often failed to meet seven basic methodological reporting standards used in a newly developed quality assessment tool. Synthesising views studies required the adaptation of qualitative analysis techniques. The benefits of bringing together views studies in a systematic way included gaining a greater breadth of perspectives and a deeper understanding of public health issues from the point of view of those targeted by interventions. A systematic approach also aided reflection on study methods that may distort, misrepresent, or fail to pick up people’s views. This methodology is likely to create greater opportunities for people’s own perspectives and experiences to inform policies to promote their health.

A multidimensional conceptual framework for analysing public involvement in health services research

Objective To describe the development of a multidimensional conceptual framework capable of drawing out the implications for policy and practice of what is known about public involvement in research agenda setting.

An Emerging Framework for Including Different Types of Evidence in Systematic Reviews for Public Policy

Evidence-informed policy and practice call on research addressing a broad range of research questions: evaluating the need for, and development, implementation, acceptability and effectiveness of interventions. Synthesizing this evidence requires methods that integrate the findings from diverse study designs. This article reports the development of a new model of research synthesis for this purpose. On completion of a series of substantive reviews, methodological reflections addressed: the interrelationship between review questions, relevant theory and values within the review process; methodological similarities and differences with more conventional reviews of effectiveness; the added value in terms of conclusions and specific recommendations; and the relevance to public policy.

Prevalence of complementary therapy use by women with breast cancer: a population-based survey

The study measured the prevalence and costs of complementary therapy use by women diagnosed with breast cancer in the South Thames NHS region. A postal questionnaire was sent to a sample of 1023 women from the Thames Cancer Registry who had been diagnosed with breast cancer in the previous 7 years. Just over a fifth (22.4%) had consulted a complementary practitioner in the previous 12 months. Almost one third (31.5%) had done so since diagnosis. Almost pound17000 had been spent on visits in the previous year. The women using complementary medicine after diagnosis were slightly younger, more educated and more likely to have used complementary medicine before their diagnosis than non-complementary medicine users. In conclusion, significant numbers of women are visiting complementary therapists for health reasons following a breast cancer diagnosis. The out of pocket costs per user, with notable exceptions, are modest. Use of practitioners of complementary therapies following diagnosis is a significant and possibly growing phenomenon.

Assessment of generalisability in trials of health interventions: suggested framework and systematic review

Most evaluations of new treatments use highly selected populations, making it difficult to decide whether they would work elsewhere. Systematic evaluation and reporting of applicability is required

The views of young children in the UK about obesity, body size, shape and weight: a systematic review

BackgroundThere are high levels of concern about childhood obesity, with obese children being at higher risk of poorer health both in the short and longer terms. Childrens attitudes to, and beliefs about, their bodies have also raised concern. Children themselves have a stake in this debate; their perspectives on this issue can inform the ways in which interventions aim to work.This systematic review of qualitative and quantitative research aimed to explore the views of UK children about the meanings of obesity and body size, shape or weight and their own experiences of these issues.MethodsWe conducted sensitive searches of electronic databases and specialist websites, and contacted experts. We included studies published from the start of 1997 which reported the perspectives of UK children aged 4-11 about obesity or body size, shape or weight, and which described key aspects of their methods. Included studies were coded and quality-assessed by two reviewers independently.Findings were synthesised in two analyses: i) an interpretive synthesis of findings from open-ended questions; and ii) an aggregative synthesis of findings from closed questions. We juxtaposed the findings from the two syntheses. The effect of excluding the lowest quality studies was explored. We also consulted young people to explore the credibility of a subset of findings.ResultsWe included 28 studies. Instead of a focus on health, children emphasised the social impact of body size, describing experiences and awareness of abuse and isolation for children with a greater weight. Body size was seen as under the individuals control and children attributed negative characteristics to overweight people. Children actively assessed their own size; many wished their bodies were different and some were anxious about their shape.Reviewers judged that childrens engagement and participation in discussion had only rarely been supported in the included studies, and few study findings had depth or breadth.ConclusionsInitiatives need to consider the social aspects of obesity, in particular unhelpful beliefs, attitudes and discriminatory behaviours around body size. Researchers and policy-makers should involve children actively and seek their views on appropriate forms of support around this issue.

Acupuncture for migraine and headache in primary care: a protocol for a pragmatic, randomized trial

UNLABELLED This paper presents the protocol for a randomized trial of acupuncture for migraine and headache. SUBJECTS Four hundred patients with migraine or headache will be recruited from GP practices. INCLUSION CRITERIA Eighteen to 65 years old, contractable by telephone, onset at least 1 year prior at age less than 50, two headaches per month in the previous 6 months, adequate data completion and headache severity during pre-randomization baseline. EXCLUSION CRITERIA Pregnancy or malignancy, cluster headache, serious pathological aetiology, cranial neuralgia, acupuncture treatment in the past year. DESIGN Following a 4-week baseline, patients will be allocated to acupuncture or control by minimized randomization. Up to 12 acupuncture treatments will be provided by advanced members of the Acupuncture Association of Chartered Physiotherapists. The type of acupuncture given will be recorded. STUDY MEASURES: Outcome will be assessed by headache diary, medication diary and SF36 at 3 months and 1 year. Resource use and days off sick will be assessed by quarterly questionnaire. Adverse events will be monitored by self-report. The primary outcome measure will be the change in mean daily headache score between baseline and the 1 year follow-up. An economic evaluation will also be undertaken.

Randomised controlled trials for policy interventions: a review of reviews and meta-regression.

OBJECTIVES To determine whether randomised controlled trials (RCTs) lead to the same effect size and variance as non-randomised studies (NRSs) of similar policy interventions, and whether these findings can be explained by other factors associated with the interventions or their evaluation. DATA SOURCES Two RCTs were resampled to compare randomised and non-randomised arms. Comparable field trials were identified from a series of health promotion systematic reviews and a systematic review of transition for youths with disabilities. Previous methodological studies were sought from 14 electronic bibliographic databases (Applied Social Sciences Index and Abstracts, Australian Education Index, British Education Index, CareData, Dissertation Abstracts, EconLIT, Educational Resources Information Centre, International Bibliography of the Sociological Sciences, ISI Proceedings: Social Sciences and Humanities, PAIS International, PsycINFO, SIGLE, Social Science Citation Index, Sociological Abstracts) in June and July 2004. These were supplemented by citation searching for key authors, contacting review authors and searching key internet sites. REVIEW METHODS Analyses of previous resampling studies, replication studies, comparable field studies and meta-epidemiology investigated the relationship between randomisation and effect size of policy interventions. New resampling studies and new analyses of comparable field studies and meta-epidemiology were strengthened by testing pre-specified associations supported by carefully argued hypotheses. RESULTS Resampling studies offer no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. Prior methodological reviews and meta-analyses of existing reviews comparing effects from RCTs and non-randomised controlled trials (nRCTs) suggested that effect sizes from RCTs and nRCTs may indeed differ in some circumstances and that these differences may well be associated with factors confounded with design. No consistent explanations were found for randomisation being associated with changes in effect sizes of policy interventions in field trials. CONCLUSIONS From the resampling studies we have no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. At the level of individual studies, non-randomised trials may lead to different effect sizes, but this is unpredictable. Many of the examples reviewed and the new analyses in the current study reveal that randomisation is indeed associated with changes in effect sizes of policy interventions in field trials. Despite extensive analysis, we have identified no consistent explanations for these differences. Researchers mounting new evaluations need to avoid, wherever possible, allocation bias. New policy evaluations should adopt randomised designs wherever possible.