Ruth Stewart

Teaching critical appraisal skills in healthcare settings

BACKGROUNDnCritical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individuals work. Within the last decade critical appraisal has been added as a topic to many medical school and UK Royal College curricula, and several continuing professional development ventures have been funded to provide further training. This is an update of a Cochrane review first published in 2001.nnnOBJECTIVESnTo assess the effects of teaching critical appraisal skills to health professionals on the process of care, patient outcomes and knowledge of health professionals. SEARCH METHODS. We updated the search (see Appendix 1 for search strategies by database) and used those search strategies to search the Cochrane Central Register of Controlled Trials (1997 to June 2011) and MEDLINE (from 1997 to June 2011). We also searched EMBASE, CINAHL and PsycINFO (up to January 2010). We searched LISA (up to January 2010), ERIC (up to January 2010), SIGLE (up to January 2010) and Web of Knowledge (up to January 2010). We also searched the Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effects (DARE) and the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register up to January 2010.nnnSELECTION CRITERIAnRandomised trials, controlled clinical trials, controlled before and after studies and interrupted time series analyses that examined the effectiveness of educational interventions teaching critical appraisal to health professionals. The outcomes included process of care, patient mortality, morbidity, quality of life and satisfaction. We included studies reporting on health professional knowledge/awareness only when based upon objective, standardised, validated instruments. We did not consider studies involving students.nnnDATA COLLECTION AND ANALYSISnTwo review authors independently extracted data and assessed risk of bias. We contacted authors of included studies to obtain missing data.nnnMAIN RESULTSnIn total, we reviewed a total of 11,057 titles and abstracts, of which 148 appeared potentially relevant to the review. We included three studies involving 272 people in this review. None of the included studies evaluated process of care or patient outcomes. Statistically significant improvements in participants knowledge were reported in domains of critical appraisal (variable approaches across studies) in two of the three studies. We determined risk of bias to be unclear and as such considered this to be plausible bias that raises some doubt about the results.nnnAUTHORS CONCLUSIONSnLow-intensity critical appraisal teaching interventions in healthcare populations may result in modest gains. Improvements to research examining the effectiveness of interventions in healthcare populations are required; specifically rigorous randomised trials employing interventions using appropriate adult learning theories.

Informed choice and public health screening for children: the case of blood spot screening

Objectiveu2002 To examine parents’ and health professionals’ views on informed choice in newborn blood spot screening, and assess information and communication needs.

Newborn screening information supports public health more than informed choice

Objective To appraise information resources on newborn blood spot screening currently available for parents and health professionals internationally. Method Health information on newborn blood spot screening was sourced internationally through the internet and, in the United Kingdom, through health service providers and support organisations. An appraisal tool was developed based on DISCERN criteria for evaluating information about patient treatment choices and UK National Screening Committee criteria for determining whether a screening programme should be implemented. Two researchers independently applied the appraisal tool to all the leaflets describing the heel prick, then discussed and resolved discrepancies. Results The majority of leaflets supported the public health agenda by informing parents of the benefits of screening, including the significance of early detection and treatment for these conditions. Few leaflets, however, supported the informed choice agenda by mentioning either the limitations of screening, or choice, or by being easy to read. Conclusion Most health information about newborn blood spot screening portrays it in a positive light, and relatively few leaflets address the difficulties openly. We recommend the use of criteria for high-quality patient information alongside topic or condition-specific guidelines as a starting point for developing patient information.

Evidence informed policy making for health communication

Background Health care decision-making requires research evidence and good judgement. This applies to policy making, as much as it does to deci sions about an individuals care. We have attempted to employ the principles of patient involvement and evidence-based medicine to the development of policies for communication about newborn blood spot screening. Methods We identified points of communication along the newborn screening pathway and gathered evidence about these communication events. We asked three questions: what research evidence is available about screening and communication; what are parents experiences of and preferences for communication about screening; and, to what extent does current parent information match the evidence, and meet parents needs? Drawing together these different forms of evidence, we employed consensus development methods, convening a multi-disciplined working group to guide our decisions in drafting communication resources and procedures. Finally we carried out a consultation to garner feedback on these draft communication policies. Results Feedback from our consultation was largely positive. We made changes to the format of the parent leaflet, and simplified the content. Where we faced contradictory advice, we followed NICE guidelines, prioritising the views of the majority of parents. Discussion We have employed a process of developing evidence-informed communication policy that incorporates a wide range of evidence, and incorporates the judgement of patients and health professionals, through the use of consensus development methods.

Purity or pragmatism? Reflecting on the use of systematic review methodology in development

Systematic review methodology pioneered in health care has been increasingly applied to development questions of importance in lower- and middle-income countries. This paper reports one such review on the topic of microfinance in sub-Saharan Africa and reflects on the number of pragmatic methodological compromises made when applying the method to a new field. These compromises relate to multidisciplinary teamwork, application of regional filters, drawing on evidence from additional study types and exploring mechanisms for change through the development and testing of a causal pathway. The paper concludes that a pragmatic rigorous approach to systematically reviewing evidence of effectiveness is needed for international development.

The size and nature of the evidence-base for smallholder farming in Africa: a systematic map

Relevant systematic reviews and impact evaluations were systematically sought and described in order to assess the size and nature of the evidence-base about the effectiveness of interventions for smallholder farmers in Africa. A total of 21 relevant systematic reviews and 415 reports of impact evaluations were identified. This paper describes this African evidence-base in terms of the interventions and outcomes assessed, and the geographical spread of the primary research across the continent. Gaps in the evidence-base are identified and recommendations made for future research.

Patient and public perspectives shaping scientific and medical research: panels for data, discussions, and decisions

This paper explores the role of patient panels for shaping research for health, scientific research about health and illness, and applied medical research. After examining the history and purposes of involving patients in discussions and decision making for research, it outlines the expertise and skills required if panels are to be successful. The paper also analyses existing guidance for panels that include patients. Panels benefit from the experiential knowledge of panel members, craft knowledge of panel facilitators, and organizational knowledge gained through previous experience of hosting panels. Guidance is available that addresses structures and resources (for panel funders) and interpersonal communication and group dynamics (for panel members and facilitators). This guidance is most comprehensive when it has itself been developed by all these types of stakeholders.

What have we learned from the application of systematic review methodology in international development? – a thematic overview

The importance of systematic review evidence in the design and implementation of policies and interventions is increasingly recognised in the field of international development. This article presents a stocktake of the primary years of systematic reviewing in international development, providing a thematic overview of what we have learned about conducting international development reviews. Applying a structured methodology to search for and categorise all relevant literature, it establishes that systematic reviews have been well received in international development and serve as a useful tool for evidence-informed development, which has led to important discoveries in the domain.

Exploring the evidence-practice gap: a workshop report on mixed and participatory training for HIV prevention in Southern Africa

BACKGROUNDnThe gap between what is known and what is done about public health (the evidence-practice gap) needs addressing. One solution may be through mixed and participatory training in accessing and appraising research.nnnAPPROACHnResidential workshops trained policy-makers, practitioners and researchers from seven southern-African countries in evidence-based decision-making for HIV prevention. They included training in accessing, critiquing and summarizing research, whilst remaining responsive to the priorities of the participants.nnnREFLECTIONSnDrawing on the participants feedback and our observations, we reflected on how these workshops may have addressed the evidence-practice gap. We identified three areas: access to research, understanding of research and the relevance of research. The workshops enabled a small group of people to access relevant research in a timely manner. However, more needs to be done to disseminate research findings appropriately as any long-term impact will be affected by the political and economic context in which participants work. We are confident that the participants went away with increased understanding of the purposes and processes of research, but for research to make a difference, the research community needs to emphasise more the publication of research findings written for potential users. The workshops were most successful in influencing researchers to consider bridging the evidence-practice gap by producing more relevant research, applicable to policy-makers and practitioners.nnnCONCLUSIONnThis intensive intervention has the potential to reduce the evidence-practice gap for HIV prevention in southern Africa by training non-researchers to engage with research, whilst providing an opportunity for researchers to engage with policy-makers and practitioners.

Walking the last mile on the long road to evidence-informed development: building capacity to use research evidence

The systematic application of rigorous evidence to inform the design and implementation of development policies and programmes has the potential to positively influence development outcomes. To achieve such evidence-informed development, a process of generating, transmitting, and using high-quality, policy-relevant evidence of development effectiveness is required. This article focuses on the final step in this casual chain – the use of evidence by national development policymakers. It proposes a people- and demand-focused approach to capacity building for the use of research evidence by policymakers. This support in building personal as well as team capacity and demand is assumed to encourage a growing institutionalisation of evidence use. The article integrates these capacity-building efforts into the wider theory of change for evidence-informed development, highlighting the importance of effective mechanisms to encourage research use in order to achieve the objective of improving lives through research evidence.