Kirsty Loudon

The PRECIS-2 tool: designing trials that are fit for purpose

PRECIS is a tool to help trialists make design decisions consistent with the intended purpose of their trial. This paper gives guidance on how to use an improved, validated version, PRECIS-2, which has been developed with the help of over 80 international trialists, clinicians, and policymakers. Keeping the original simple wheel format, PRECIS-2 has nine domains—eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis—scored from 1 (very explanatory) to 5 (very pragmatic) to facilitate domain discussion and consensus. It is hoped PRECIS-2 will be valuable in supporting the explicit matching of design decisions to how the trial results are intended to be used

Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform

Randomised trials are at the heart of evidence-based healthcare, but the methods and infrastructure for conducting these sometimes complex studies are largely evidence free. Trial Forge (www.trialforge.org) is an initiative that aims to increase the evidence base for trial decision making and, in doing so, to improve trial efficiency.This paper summarises a one-day workshop held in Edinburgh on 10 July 2014 to discuss Trial Forge and how to advance this initiative. We first outline the problem of inefficiency in randomised trials and go on to describe Trial Forge. We present participants’ views on the processes in the life of a randomised trial that should be covered by Trial Forge.General support existed at the workshop for the Trial Forge approach to increase the evidence base for making randomised trial decisions and for improving trial efficiency. Agreed upon key processes included choosing the right research question; logistical planning for delivery, training of staff, recruitment, and retention; data management and dissemination; and close down. The process of linking to existing initiatives where possible was considered crucial. Trial Forge will not be a guideline or a checklist but a ‘go to’ website for research on randomised trials methods, with a linked programme of applied methodology research, coupled to an effective evidence-dissemination process. Moreover, it will support an informal network of interested trialists who meet virtually (online) and occasionally in person to build capacity and knowledge in the design and conduct of efficient randomised trials.Some of the resources invested in randomised trials are wasted because of limited evidence upon which to base many aspects of design, conduct, analysis, and reporting of clinical trials. Trial Forge will help to address this lack of evidence.

Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose.

BackgroundIf you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention.MethodsThe study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism.DiscussionWe have concrete suggestions for improving PRECIS and a growing list of enthusiastic individuals interested in contributing to this work. By early 2014 we expect to have a validated PRECIS-2.

Patient and public attitudes to and awareness of clinical practice guidelines: A systematic review with thematic and narrative syntheses

BackgroundClinical practice guidelines are typically written for healthcare providers but there is increasing interest in producing versions for the public, patients and carers. The main objective of this review is to identify and synthesise evidence of the public’s attitudes towards clinical practice guidelines and evidence-based recommendations written for providers or the public, together with their awareness of guidelines.MethodsWe included quantitative and qualitative studies of any design reporting on public, patient (and their carers) attitudes and awareness of guidelines written for providers or patients/public. We searched electronic databases including MEDLINE, PSYCHINFO, ERIC, ASSIA and the Cochrane Library from 2000 to 2012. We also searched relevant websites, reviewed citations and contacted experts in the field. At least two authors independently screened, abstracted data and assessed the quality of studies. We conducted a thematic analysis of first and second order themes and performed a separate narrative synthesis of patient and public awareness of guidelines.ResultsWe reviewed 5415 records and included 26 studies (10 qualitative studies, 13 cross sectional and 3 randomised controlled trials) involving 24 887 individuals. Studies were mostly good to fair quality. The thematic analysis resulted in four overarching themes: Applicability of guidelines; Purpose of guidelines for patient; Purpose of guidelines for health care system and physician; and Properties of guidelines. Overall, participants had mixed attitudes towards guidelines; some participants found them empowering but many saw them as a way of rationing care. Patients were also concerned that the information may not apply to their own health care situations. Awareness of guidelines ranged from 0-79%, with greater awareness in participants surveyed on national guideline websites.ConclusionThere are many factors, not only formatting, that may affect the uptake and use of guideline-derived material by the public. Producers need to make clear how the information is relevant to the reader and how it can be used to make healthcare improvements although there were problems with data quality. Awareness of guidelines is generally low and guideline producers cannot assume that the public has a more positive perception of their material than of alternative sources of health information.

The PRECIS-2 tool has good interrater reliability and modest discriminant validity

OBJECTIVES PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. STUDY DESIGN AND SETTING Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. RESULTS Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. CONCLUSION We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders.

Incomplete reporting of recruitment information in breast cancer trials published between 2003 and 2008.

OBJECTIVES To review the reporting of key items of recruitment information in trial reports and estimate the number needed to screen to recruit one additional participant. STUDY DESIGN AND SETTING Review of breast cancer trials published in the years 2003-2005, 2007, and 2008. RESULTS The search identified 1,570 potentially eligible studies. After a random selection of 20% from each year and checking against inclusion criteria, a total of 207 studies were included in the review. Some items of information were well reported, such as the number included in the analysis. Sample size calculations were often not presented, but reporting is slowly improving. Who recruits participants and how many individuals were screened are often not reported. The median number needed to screen to recruit one additional participant was two (range, 1-593). CONCLUSIONS Without reporting the when, where, by whom, and how many of recruitment, trialists deny readers part of the contextual description they need to judge whether a trials results are applicable to their own situation. Trialists and journal editors need to be more diligent in following the reporting recommendations of the Consolidated Standards of Reporting Trials statement.

Original ArticleThe PRECIS-2 tool has good interrater reliability and modest discriminant validity

OBJECTIVES PRagmatic Explanatory Continuum Indicator Summary (PRECIS)-2 is a tool that could improve design insight for trialists. Our aim was to validate the PRECIS-2 tool, unlike its predecessor, testing the discriminant validity and interrater reliability. STUDY DESIGN AND SETTING Over 80 international trialists, methodologists, clinicians, and policymakers created PRECIS-2 helping to ensure face validity and content validity. The interrater reliability of PRECIS-2 was measured using 19 experienced trialists who used PRECIS-2 to score a diverse sample of 15 randomized controlled trial protocols. Discriminant validity was tested with two raters to independently determine if the trial protocols were more pragmatic or more explanatory, with scores from the 19 raters for the 15 trials as predictors of pragmatism. RESULTS Interrater reliability was generally good, with seven of nine domains having an intraclass correlation coefficient over 0.65. Flexibility (adherence) and recruitment had wide confidence intervals, but raters found these difficult to rate and wanted more information. Each of the nine PRECIS-2 domains could be used to differentiate between trials taking more pragmatic or more explanatory approaches with better than chance discrimination for all domains. CONCLUSION We have assessed the validity and reliability of PRECIS-2. An elaboration study and web site provide guidance to help future users of the tool which is continuing to be tested by trial teams, systematic reviewers, and funders.

Understanding the applicability of results from primary care trials: lessons learned from applying PRECIS-2.

OBJECTIVE To compare two approaches for trial teams to apply PRECIS-2 to pragmatic trials: independent scoring and scoring following a group discussion. STUDY DESIGN AND SETTING We recruited multidisciplinary teams who were conducting or had conducted trials in primary care in collaboration with the Pragmatic Clinical Trials Unit, Queen Mary University of London. Each team carried out two rounds of scoring on the nine PRECIS-2 domains: first independently using an online version of PRECIS-2 and second following a discussion. RESULTS Seven teams took part in the study. Before the discussion, within-team agreement in scores was generally poor and not all raters were able to score all domains; agreement improved after the discussion. The PRECIS-2 wheels suggested that the trials were pragmatic, although some domains were more pragmatic than others. CONCLUSION PRECIS-2 can facilitate information exchange within trial teams. To apply PRECIS-2 successfully, we recommend a discussion between those with detailed understanding of what usual care is for the intervention, the trials design including operational and technical aspects, and the PRECIS-2 domains. For some cluster-randomized trials, greater insight may be gained by plotting two PRECIS-2 wheels, one at the individual participant level and another at the cluster level.

PRECIS-2: a tool to improve the applicability of randomised controlled trials

Results Brainstorming sessions identified the PRECIS presentation (a wheel), lack of a scoring system and domain weighting as issues for exploration in the Delphi process. Thirty four completed responses from 90 invitees were received in Round 1 of the Delphi; Round 2 involved 23 individuals (response rate 82%). 45% selected a 1-5 Likert scale, 56.5% wanted to use a table (to justify decisions) and a PRECIS wheel, 26% were in favour of weighting domains. Suggestions for extra domains included: recruitment process for participants and integration of the intervention into the healthcare system. An expert panel in Toronto used the Delphi suggestions to help create alternative versions of PRECIS-2 for user-testing in spring 2013. Conclusions PRECIS can be improved by the addition of a Likert scale and additional domains. We expect to have a validated PRECIS-2 by the beginning of 2014.

Patients, health information, and guidelines: A focus-group study

Background. Evidence-based clinical guidelines could support shared decision-making and help patients to participate actively in their care. However, it is not well known how patients view guidelines as a source of health information. This qualitative study aimed to assess what patients know about guidelines, and what they think of their presentation formats. Research question. What is the role of guidelines as health information for patients and how could the implementation of evidence-based information for patients be improved? Methods. A qualitative study with focus groups that were built around a semi-structured topic guide. Focus groups were audiotaped and transcribed and analysed using a phenomenographic approach. Results. Five focus groups were carried out in 2012 with a total of 23 participants. Patients searched for health information from the Internet or consulted health professionals or their personal networks. The concepts of guidelines included instructions or standards for health professionals, information given by a health professional to the patient, and material to protect and promote the interests of patients. Some patients did not have a concept for guidelines. Patients felt that health information was abundant and its quality sometimes difficult to assess. They respected conciseness, clarity, clear structure, and specialists or well-known organizations as authors of health information. Patients would like health professionals to deliver and clarify written materials to them or point out to them the relevant Internet sites. Conclusions. The concept of guidelines was not well known among our interviewees; however, they expressed an interest in having more communication on health information, both written information and clarifications with their health professionals.