Alison Booth

The nuts and bolts of PROSPERO: an international prospective register of systematic reviews

BackgroundFollowing publication of the PRISMA statement, the UK Centre for Reviews and Dissemination (CRD) at the University of York in England began to develop an international prospective register of systematic reviews with health-related outcomes. The objectives were to reduce unplanned duplication of reviews and provide transparency in the review process, with the aim of minimizing reporting bias.MethodsAn international advisory group was formed and a consultation undertaken to establish the key items necessary for inclusion in the register and to gather views on various aspects of functionality. This article describes the development of the register, now called PROSPERO, and the process of registration.ResultsPROSPERO offers free registration and free public access to a unique prospective register of systematic reviews across all areas of health from all around the world. The dedicated web-based interface is electronically searchable and available to all prospective registrants. At the moment, inclusion in PROSPERO is restricted to systematic reviews of the effects of interventions and strategies to prevent, diagnose, treat, and monitor health conditions, for which there is a health-related outcome.Ideally, registration should take place before the researchers have started formal screening against inclusion criteria but reviews are eligible as long as they have not progressed beyond the point of completing data extraction.The required dataset captures the key attributes of review design as well as the administrative details necessary for registration.Submitted registration forms are checked against the scope for inclusion in PROSPERO and for clarity of content before being made publicly available on the register, rejected, or returned to the applicant for clarification.The public records include an audit trail of major changes to planned methods, details of when the review has been completed, and links to resulting publications when provided by the authors.ConclusionsThere has been international support and an enthusiastic response to the principle of prospective registration of protocols for systematic reviews and to the development of PROSPERO.In October 2011, PROSPERO contained 200 records of systematic reviews being undertaken in 26 countries around the world on a diverse range of interventions.

An international registry of systematic-review protocols

Support for prospective registration of protocols for systematic reviews has been gathering momentum. The PRISMA statement, a guideline for reporting systematic reviews and meta-analyses of studies that evaluate health-care interventions, advocates registration. Well-conducted systematic reviews are accepted as the best-quality evidence to inform policy and practice, and the dramatic upward trend in the number of systematic reviews published annually (fi gure) is set to continue. However, there is currently no single facility for identifying this type of research in advance of the appearance of the results of the review. There is concern about and evidence of publication and selective outcome-reporting biases associated with systematic reviews. An open registry of reviews captured at the protocol stage would facilitate good practice in systematic reviews by providing transparency of the review process and outcomes. Discrepancies between the methods of the published review and those planned in the registered protocol could be more readily identifi ed. Registration might also encourage full publication of the review’s fi ndings and transparency in changes to methods that could bias fi ndings. In both the prevention and revelation of potential bias, registration should improve quality and increase confi dence that policy or practice informed by the fi ndings of systematic reviews are indeed drawing on best-quality evidence. A registry of protocols of systematic reviews could assist those planning new reviews and updating existing ones. Easy access to information about ongoing reviews should help to optimise the use of fi nite resources by enabling funding and commissioning agencies to avoid unnecessary duplication and encourage collaboration. A comprehensive registry could also create opportunities for methodological and other research, both within and across disciplines. Existing access to systematic-review protocols is limited to the outputs of individual organisations, such as the Cochrane and Campbell Collaborations and the Joanna Briggs Institute. The National Public Health Service for Wales is piloting an All Wales Systematic Reviews Register, but, up to now, there has been no centralised comprehensive registry of systematic-review protocols. We are developing an international facility to register the details of ongoing systematic reviews in health and social care. Our registry is being established with the existing platform and infrastructure that supports the Database of Abstracts of Reviews of Eff ects (DARE), the NHS Economic Evaluations Database (NHS EED) and the Health Technology Assessment (HTA) database produced by the Centre for Reviews and Dissemination. The initial scope of the registry will be limited to systematic reviews of the eff ectiveness of health interventions. However, the longterm aim is for the registry to include details of all ongoing systematic reviews with a health-related outcome in the broadest sense. Ultimately, inclusion will encompass systematic reviews of health-care interventions, and reviews of the social determinants of health, of service delivery, and of risk factors and genetic associations. Our web-based registry will off er free public access, be electronically searchable, and open to all prospective registrants. Registration will require the provision of a minimum dataset, the completeness of which will be checked automatically before registration. After acceptance, the registry’s entry and protocol for the review, if available, will be loaded on the database as a permanent entry and a unique identifi cation number issued. An audit trail for any amendments to the information will be available within the record. Links to resulting publications will be added. These provisions ensure that the registry will match the relevant criteria required of clinical trials registries.

PROSPERO at one year: an evaluation of its utility.

BackgroundPROSPERO, an international prospective register of systematic review protocols in health and social care, was launched in February 2011. After one year of operation we describe access and use, explore user experience and identify areas for future improvement.MethodsWe collated administrative data and web statistics and conducted an online survey of users’ experiences.ResultsOn 21 February 2012, there were 1,076 registered users and 359 registration records published on PROSPERO. The database usage statistics demonstrate the international interest in PROSPERO with high access around the clock and around the world. Based on 232 responses from PROSPERO users (response rate 22%), almost all respondents found joining and navigation was easy or very easy (99%); turn round time was good or excellent (96%); and supporting materials provided were helpful or very helpful (80%). The registration fields were found by 80% to be relevant to their review; 99% rated their overall experience of registering with PROSPERO as good or excellent. Most respondents (81%) had a written protocol before completing the registration form and 19% did not. The majority, 136 (79%), indicated they completed the registration form in 60 minutes or less. Of those who expressed an opinion, 167 (87%) considered the time taken to be about right.ConclusionsThe first year of PROSPERO has shown that registration of systematic review protocols is feasible and not overly burdensome for those registering their reviews. The evaluation has demonstrated that, on the whole, survey respondents are satisfied and the system allows registration of protocol details in a straightforward and acceptable way. The findings have prompted some changes to improve user experience and identified some issues for future consideration.

Establishing a Minimum Dataset for Prospective Registration of Systematic Reviews: An International Consultation

Background In response to growing recognition of the value of prospective registration of systematic review protocols, we planned to develop a web-based open access international register. In order for the register to fulfil its aims of reducing unplanned duplication, reducing publication bias, and providing greater transparency, it was important to ensure the appropriate data were collected. We therefore undertook a consultation process with experts in the field to identify a minimum dataset for registration. Methods and Findings A two-round electronic modified Delphi survey design was used. The international panel surveyed included experts from areas relevant to systematic review including commissioners, clinical and academic researchers, methodologists, statisticians, information specialists, journal editors and users of systematic reviews. Direct invitations to participate were sent out to 315 people in the first round and 322 in the second round. Responses to an open invitation to participate were collected separately. There were 194 (143 invited and 51 open) respondents with a 100% completion rate in the first round and 209 (169 invited and 40 open) respondents with a 91% completion rate in the second round. In the second round, 113 (54%) of the participants reported having previously taken part in the first round. Participants were asked to indicate whether a series of potential items should be designated as optional or required registration items, or should not be included in the register. After the second round, a 70% or greater agreement was reached on the designation of 30 of 36 items. Conclusions The results of the Delphi exercise have established a dataset of 22 required items for the prospective registration of systematic reviews, and 18 optional items. The dataset captures the key attributes of review design as well as the administrative details necessary for registration.

How to reduce unnecessary duplication: use PROSPERO

PROSPERO D Moher, A Booth, L Stewart a Faculty of Medicine, Clinical Epidemiology Program, Ottawa Hospital Research Institute, University of Ottawa, Ottawa, ON, Canada b NIHR Centre for Reviews and Dissemination, University of York, York, UK Correspondence: D Moher, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital General Campus, 501 Smyth Rd, Room L1288, Ottawa, ON K1H 8L6, Canada. Email dmoher@ohri.ca

After the gold rush? A systematic and critical review of general medical podcasts.

Summary Objectives Podcasts are increasingly used to enhance many forms of research communication and education. We set out to assess the extent of this podcast revolution by identifying and critically describing the content and quality of podcast services provided by leading general medical journals. Methods Summary of general and internal medicine journal podcasts identified in April 2008 by means of web-searching, with a brief commentary on their content and quality. Results Of the top 100 general medical and internal journals as ranked by impact factor only eight offer a regular podcast. The technical quality of most is of an acceptable but not of a high standard. The mode of delivery and resulting listening experience is variable with those utilizing an interview format more interesting to listen to than those reliant upon a single voice. Conclusions General medical podcasts are potentially a valuable resource for providing a digestible overview of the latest research, and for providing an opportunity to dip into areas outside ones own core interests. Although they represent a novel use of technology for disseminating knowledge, uptake has been limited and the quality of the listening experience is variable.

Social prescribing: less rhetoric and more reality.: A systematic review of the evidence

Objectives Social prescribing is a way of linking patients in primary care with sources of support within the community to help improve their health and well-being. Social prescribing programmes are being widely promoted and adopted in the UK National Health Service and so we conducted a systematic review to assess the evidence for their effectiveness. Setting/data sources Nine databases were searched from 2000 to January 2016 for studies conducted in the UK. Relevant reports and guidelines, websites and reference lists of retrieved articles were scanned to identify additional studies. All the searches were restricted to English language only. Participants Systematic reviews and any published evaluation of programmes where patient referral was made from a primary care setting to a link worker or facilitator of social prescribing were eligible for inclusion. Risk of bias for included studies was undertaken independently by two reviewers and a narrative synthesis was performed. Primary and secondary outcome measures Primary outcomes of interest were any measures of health and well-being and/or usage of health services. Results We included a total of 15 evaluations of social prescribing programmes. Most were small scale and limited by poor design and reporting. All were rated as a having a high risk of bias. Common design issues included a lack of comparative controls, short follow-up durations, a lack of standardised and validated measuring tools, missing data and a failure to consider potential confounding factors. Despite clear methodological shortcomings, most evaluations presented positive conclusions. Conclusions Social prescribing is being widely advocated and implemented but current evidence fails to provide sufficient detail to judge either success or value for money. If social prescribing is to realise its potential, future evaluations must be comparative by design and consider when, by whom, for whom, how well and at what cost. Trial registration number PROSPERO Registration: CRD42015023501.

Deconstructing media coverage of trastuzumab (Herceptin): an analysis of national newspaper coverage

Summary Objective To explore and critically describe the content and main narratives of UK national daily newspaper coverage of trastuzumab (Herceptin®). Design We used the NewsBank database to search eight national daily newspapers, and their Sunday equivalents, retrospectively from 19 February 2006 back to the earliest mention of trastuzumab or Herceptin (19 May 1998). Setting UK national newspapers. Main outcome measures To be eligible for inclusion, articles had to contain at least three sentences about trastuzumab. Articles that focused on the financial performance of companies associated with the drug were excluded from the analysis. For each included article, we extracted bibliographic details and data, and independently rated the reporting slant towards trastuzumab and, where relevant, the reporting slant towards access to treatment. Results We identified 361 articles that met the study inclusion criteria. The proprietary name of Herceptin was always used, with only eight articles mentioning the generic alternative. 294/361 included articles (81.5%) were rated as being positive towards trastuzumab, the remainder rated as neutral. Access to trastuzumab treatment was the main narrative running across included articles and reports of individual patients seeking treatment featured prominently throughout. In 208/361 of included articles (57%) the reporting slant towards access to trastuzumab treatment was rated as negative. 178/361 of included articles (49.3%) mentioned licensing, but rarely mentioned that licensing processes can only occur when the manufacturer applies for a licence. Only a minority of articles mentioned that the drug had to be licensed before it could be subject to the NICE approval process. Conclusions Newspaper coverage of trastuzumab has been characterized by uncritical reporting. Journalists (and consumers) should be more questioning when confronted with information about new drugs and of the motives of those who seek to set the news agenda.

Centre for Reviews and Dissemination databases: value, content, and developments.

BACKGROUND The Centre for Reviews and Dissemination (CRD) funded through the United Kingdom National Institute for Health Research undertakes systematic reviews evaluating the research evidence on health and public health questions of national and international importance. CRD is involved in methods research and produces internationally accepted guidelines for undertaking systematic reviews. A core product of the Centre are the CRD databases; a key resource for health professionals, policy makers, and researchers around the world. Research evidence evaluating the effects and cost-effectiveness of healthcare interventions is growing year on year, and can be difficult and time consuming to identify and appraise. The databases assist decision makers by systematically identifying and critically describing systematic reviews and economic evaluations, appraising their quality and highlighting their relative strengths and weaknesses; and providing a comprehensive listing of ongoing and published health technology assessments. METHODS AND RESULTS This article briefly outlines the processes behind each of the three databases, gives details of the records, the information they contain and the value added by CRD. An overview of the new public interface for the CRD databases to be launched later this year is given. Improvements and innovations to supplement existing functions are listed, including enhanced and new search options, the addition of quick links to relevant material from within individual records, an integrated PubMed search and the ability to select personal preferences. CONCLUSIONS CRDs databases provide free access to an important knowledge base and are used widely to underpin evidence informed healthcare decisions in the United Kingdom and internationally.

A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study

OBJECTIVES To examine outcome reporting bias of systematic reviews registered in PROSPERO. STUDY DESIGN AND SETTING Retrospective cohort study. The primary outcomes from systematic review publications were compared with those reported in the corresponding PROSPERO records; discrepancies in the primary outcomes were assessed as upgrades, additions, omissions, or downgrades. Relative risks (RRs) and 95% confidence intervals (CI) were calculated to determine the likelihood of having a change in primary outcome when the meta-analysis result was favorable and statistically significant. RESULTS Ninety-six systematic reviews were published. A discrepancy in the primary outcome occurred in 32% of the included reviews and 39% of the reviews did not explicitly specify a primary outcome(s); 6% of the primary outcomes were omitted. There was no significant increased risk of adding/upgrading (RR, 2.14; 95% CI: 0.53, 8.63) or decreased risk of downgrading (RR, 0.76; 95% CI: 0.27, 2.17) an outcome when the meta-analysis result was favorable and statistically significant. As well, there was no significant increased risk of adding/upgrading (RR, 0.89; 95% CI: 0.31, 2.53) or decreased risk of downgrading (RR, 0.56; 95% CI: 0.29, 1.08) an outcome when the conclusion was positive. CONCLUSIONS We recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.