Julie Glanville

How to formulate research recommendations

“More research is needed” is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required

Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design

Background: systematic reviews are of increasing importance to health care professionals seeking to provide evidence-based health care, because they provide systematically prepared summaries of the current state of research knowledge on the effectiveness of health care interventions. To be able to make use of them, both researchers preparing systematic reviews and health care professionals need to be able to identify systematic reviews easily. In the past, systematic reviews have been difficult to identify easily among the mass of literature labelled ‘reviews’. Search strategies have been published which were intended to provide easier access to systematic reviews and meta-analyses. These strategies, mostly devised for the MEDLINE database, have been constructed using the authors’ knowledge of the subject area, the databases and past experience. Objective: the objective of this study was to design search strategies based on a more objective approach to strategy construction. Methods: word frequencies from the titles, abstracts and subject keywords of a collection of systematic reviews of the effective health care interventions were analysed to derive a highly sensitive search strategy. Results: the proposed strategy offers 98% sensitivity in retrieving systematic reviews, while retaining a low but acceptable level of precision (20%). Other strategies with other levels of sensitivity and precision are also presented. Discussion: this study shows that a frequency analysis approach can be used to generate highly sensitive strategies which retain adequate levels of precision when retrieving systematic reviews.

Effectiveness of probiotics on the duration of illness in healthy children and adults who develop common acute respiratory infectious conditions: a systematic review and meta-analysis

Recent systematic reviews have reported a positive, although modest, effect of probiotics in terms of preventing common cold symptoms. In this systematic review, the effect of probiotics, specifically Lactobacillus and Bifidobacterium strains, on the duration of acute respiratory infections in otherwise healthy children and adults was evaluated. To identify relevant trials, eight databases, including MEDLINE, Embase, the Cochrane Database of Systematic Reviews (CDSR), the Cochrane Central Register of Controlled Trials (CENTRAL), the Database of Abstracts of Reviews of Effects (DARE), Health Technology Assessment (HTA), Science Citation Index (SCI) and OAISTER, were searched from inception to 20 July 2012. Details regarding unpublished studies/databases were also obtained from probiotic manufacturers. Study selection, data extraction and quality assessment were carried out by two reviewers. Risk of bias was assessed using criteria adapted from those published by the Centre for Reviews and Dissemination. In this review, twenty randomised controlled trials (RCT) were included, of which twelve were considered to have a low risk of bias. Meta-analysis revealed significantly fewer numbers of days of illness per person (standardised mean difference (SMD) − 0·31 (95 % CI − 0·41, − 0·11), I 2= 3 %), shorter illness episodes by almost a day (weighted mean difference − 0·77 (95 % CI − 1·50, − 0·04), I 2= 80 %) (without an increase in the number of illness episodes), and fewer numbers of days absent from day care/school/work (SMD − 0·17 (95 % CI − 0·31, − 0·03), I 2= 67 %) in participants who received a probiotic intervention than in those who had taken a placebo. Reasons for heterogeneity between the studies were explored in subgroup analysis, but could not be explained, suggesting that the effect sizes found may differ between the population groups. This systematic review provides evidence from a number of good-quality RCT that probiotics reduce the duration of illness in otherwise healthy children and adults.

A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy

Search filters are increasingly used to search medical databases to identify specific topics or study designs. In particular, search filters have been designed to help health-care professionals identify systematic reviews of the effectiveness of health interventions. Identifying systematic reviews in databases such as MEDLINE is problematic and research has previously been undertaken into methods to design search filters that retrieve systematic reviews effectively. The aim of this study was to improve previously developed methods to derive a more objective search strategy to identify systematic reviews in MEDLINE. A ‘quasi-gold standard’ collection of known systematic reviews was identified. A frequency analysis of words within a subset of the ‘quasi-gold standard’ was undertaken followed by a statistical analysis of the most frequently occurring words. This analysis determined which terms would best distinguish between systematic reviews, non-systematic reviews and non-reviews. The performance of the best models was tested on the remaining subset of ‘quasi-gold standard’ records and then using the OVID interface to MEDLINE. The best model had a sensitivity of 73.4% for systematic reviews in the test set and 84.2% when used with the validation set. The best model had a specificity of 98.3% in the test set and 93% in the validation set. When tested on the same ‘quasi-gold standard’ using OVID MED-LINE the model showed 100% sensitivity and 4.4% precision. The number of times a term occurs in a record adds discriminatory power to the search strategy. Apparently highly relevant terms chosen subjectively do not perform as well as those derived by a statistical approach. Some search terms may not immediately seem useful in identifying systematic reviews, but when used in combination with other terms they prove to be highly discriminating. The best performing filters were tested on the OVID interface, but without frequency and term weightings. Their performance was also compared to previously published filters. One of the strategies was found to perform better with respect to sensitivity than previously published filters, although with lower precision.

Research output on primary care in Australia, Canada, Germany, the Netherlands, the United Kingdom, and the United States: bibliometric analysis

Objective To compare the volume and quality of original research in primary care published by researchers from primary care in the United Kingdom against five countries with well established academic primary care. Design Bibliometric analysis. Setting United Kingdom, United States, Australia, Canada, Germany, and the Netherlands. Studies reviewed Research publications relevant to comprehensive primary care and authored by researchers from primary care, recorded in Medline and Embase, with publication dates 2001-7 inclusive. Main outcome measures Volume of published activity of generalist primary care researchers and the quality of the research output by those publishing the most using citation metrics: numbers of cited papers, proportion of cited papers, and mean citation scores. Results 82 169 papers published between 2001 and 2007 in the six countries were classified as research on primary care. In a 15% pragmatic random sample of these records, 40% of research on primary care from the United Kingdom and 46% from the Netherlands was authored by researchers employed in a primary care setting or employed in academic departments of primary care. The 141 researchers with the highest volume of publications reporting research findings published between 2001 and 2007 (inclusive) authored or part authored 8.3% of the total sample of papers. For authors with the highest proportion of publications cited at least five times, the best performers came from the United States (n=5), United Kingdom (n=4), and the Netherlands (n=2). In the top 10 of authors with the highest proportions of publications achieving 20 or more citations, six were from the United Kingdom and four from the United States. The mean Hirsch index (measure of a researcher’s productivity and impact of the published work) was 14 for the Netherlands, 13 for the United Kingdom, 12 for the United States, 7 for Canada, 4 for Australia, and 3 for Germany. Conclusion This international comparison of the volume and citation rates of papers by researchers from primary care consistently placed UK researchers among the best performers internationally.

Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies

BackgroundNumerous types of digital health interventions (DHIs) are available to patients and the public but many factors affect their ability to engage and enrol in them. This systematic review aims to identify and synthesise the qualitative literature on barriers and facilitators to engagement and recruitment to DHIs to inform future implementation efforts.MethodsPubMed, MEDLINE, CINAHL, Embase, Scopus and the ACM Digital Library were searched for English language qualitative studies from 2000 – 2015 that discussed factors affecting engagement and enrolment in a range of DHIs (e.g. ‘telemedicine’, ‘mobile applications’, ‘personal health record’, ‘social networking’). Text mining and additional search strategies were used to identify 1,448 records. Two reviewers independently carried out paper screening, quality assessment, data extraction and analysis. Data was analysed using framework synthesis, informed by Normalization Process Theory, and Burden of Treatment Theory helped conceptualise the interpretation of results.ResultsNineteen publications were included in the review. Four overarching themes that affect patient and public engagement and enrolment in DHIs emerged; 1) personal agency and motivation; 2) personal life and values; 3) the engagement and recruitment approach; and 4) the quality of the DHI. The review also summarises engagement and recruitment strategies used. A preliminary DIgital Health EnGagement MOdel (DIEGO) was developed to highlight the key processes involved. Existing knowledge gaps are identified and a number of recommendations made for future research. Study limitations include English language publications and exclusion of grey literature.ConclusionThis review summarises and highlights the complexity of digital health engagement and recruitment processes and outlines issues that need to be addressed before patients and the public commit to digital health and it can be implemented effectively. More work is needed to create successful engagement strategies and better quality digital solutions that are personalised where possible and to gain clinical accreditation and endorsement when appropriate. More investment is also needed to improve computer literacy and ensure technologies are accessible and affordable for those who wish to sign up to them.Systematic review registrationInternational Prospective Register of Systematic Reviews CRD42015029846

Can we improve the morale of staff working in psychiatric units? A systematic review

Background: Those working in psychiatric units care for some of the most vulnerable and needy patients within health services, and suffer some of the highest levels of job dissatisfaction and burnout within healthcare workforce. Poor staff morale is bad for patient care and is economically wasteful. The same level of evidence of effectiveness and efficiency should be considered in workforce planning as is required in patient care. This has hitherto not occurred. Aim: To examine the impact and cost effectiveness of strategies to improve staff morale and reduce “burnout” amongst staff working in psychiatric units. Methods: We conducted a systematic review of robust evaluations of strategies designed to improve psychological wellbeing or the working experience of staff working in psychiatric units. We searched the following databases: EMBASE; MEDLINE; PsycINFO; CINAHL; Sociological Abstracts; HMIC; Management and Marketing Abstracts; Management Contents and Inside Conferences (all to 2004), and sought the following outcomes: Psychological wellbeing; Job satisfaction; staff burnout and stress; staff sickness and turnover; Direct and indirect costs. The following designs were included: Randomised Controlled Trails (RCTs); Controlled Clinical Trials (CCTs); Controlled Before and after studies (CBAs); and interrupted time series (ITSs). We conducted a narrative overview of key design features, endpoints and results. Results: We identified eight evaluations of strategies to improve staff morale (3 RCTs; 3 CCTs and 2 CBAs). Educational interventions designed to enhance the skill and competency of staff were the most commonly evaluated, and showed positive impact on at least one outcome of interest. Psycho-social interventions that sought out members of staff who were experiencing emotional problems and offered work-based support and enhanced social support networks were positive in US healthcare settings, but had been incompletely implemented and evaluated in UK settings. Organisational interventions, such as a shift to continuous care and primary nursing also showed a potential to be effective in reducing sickness rates and improving job satisfaction. Conclusions: There is substantial opportunity to design and implement interventions to improve the working experience of staff in psychiatric units. There is an onus to evaluate the longer term impact and cost effectiveness of these strategies. Unfortunately strategies are currently being implemented in the absence of any prospective evaluation. Declaration of interest: The authors have no conflict of interest.

THE U.K. NHS ECONOMIC EVALUATION DATABASE Economic Issues in Evaluations of Health Technology

OBJECTIVE The U.K. NHS Economic Evaluation Database (EED) project is commissioned to identify papers on economic evaluations of health technologies and to disseminate their findings to NHS decision makers by means of structured abstracts that are available through a public database and the Cochrane Library. This paper discusses current issues relating to the economic aspects of producing NHS EED abstracts. METHODS A review of NHS EED was undertaken between 1994 and 1999 to determine the methodologies adopted and issues that influence the usefulness of economic evaluations. Methods adopted to improve the quality of NHS EED abstracts are also reported. RESULTS Eighty-five percent of NHS EED abstracts are cost-effectiveness analyses (CEAs), 9.3% are cost-utility analyses (CUAs), and only 1.4% are cost-benefit analyses (CBAs). Of the total abstracts, 65.9% are based on single studies, 19.5% on reviews, 3.9% on estimates of effectiveness, and 10.7% on combinations of these sources. Models are utilized in 16.7% of CEAs, 60.2% of CUAs, and 20% of CBAs. Analyses of CBA studies reveal a degree of misuse of well-established definitions. NHS EED internal control mechanisms are reported that provide a means of ensuring that abstracts are based on sound academic principles. CONCLUSIONS Most economic evaluations are conducted by means of CEA, followed by CUA, while CBA accounts for an extreme minority of cases. Single studies form the principal source of effectiveness data, although models are widely used, principally in CUA. The structure of NHS EED abstracts provides decision makers with the principal results and an interpretation of the relative strengths and weaknesses of economic evaluations.

The prevalence of nursing staff stress on adult acute psychiatric in-patient wards. A systematic review.

Concerns about recent changes in acute in-patient mental healthcare environments have led to fears about staff stress and poor morale in acute in-patient mental healthcare staff. To review the prevalence of low staff morale, stress, burnout, job satisfaction and psychological well-being amongst staff working in in-patient psychiatric wards. Systematic review. Of 34 mental health studies identified, 13 were specific to acute in-patient settings, and 21 were specific to other non-specified ward-based samples. Most studies did not find very high levels of staff burnout and poor morale but were mostly small, of poor quality and provided incomplete or non-standardised prevalence data. The prevalence of indicators of low morale on acute in-patient mental health wards has been poorly researched and remains unclear. Multi-site, prospective epidemiological studies using validated measures of stress together with personal and organizational variables influencing staff stress in acute in-patient wards are required.

New Systematic Review Methodology for Visual Impairment and Blindness for the 2010 Global Burden of Disease Study

Abstract Purpose: To describe a systematic review of population-based prevalence studies of visual impairment (VI) and blindness worldwide over the past 32 years that informs the Global Burden of Diseases, Injuries and Risk Factors Study. Methods: A systematic review (Stage 1) of medical literature from 1 January 1980 to 31 January 2012 identified indexed articles containing data on incidence, prevalence and causes of blindness and VI. Only cross-sectional population-based representative studies were selected from which to extract data for a database of age- and sex-specific data of prevalence of four distance and one near vision loss categories (presenting and best-corrected). Unpublished data and data from studies using rapid assessment methodology were later added (Stage 2). Results: Stage 1 identified 14,908 references, of which 204 articles met the inclusion criteria. Stage 2 added unpublished data from 44 rapid assessment studies and four other surveys. This resulted in a final dataset of 252 articles of 243 studies, of which 238 (98%) reported distance vision loss categories. A total of 37 studies of the final dataset reported prevalence of mild VI and four reported near VI. Conclusion: We report a comprehensive systematic review of over 30 years of VI/blindness studies. While there has been an increase in population-based studies conducted in the 2000s compared to previous decades, there is limited information from certain regions (eg, Central Africa and Central and Eastern Europe, and the Caribbean and Latin America), and younger age groups, and minimal data regarding prevalence of near vision and mild distance VI.