Geoff Wong

Tensions and Paradoxes in Electronic Patient Record Research: A Systematic Literature Review Using the Meta-narrative Method

CONTEXTnThe extensive research literature on electronic patient records (EPRs) presents challenges to systematic reviewers because it covers multiple research traditions with different underlying philosophical assumptions and methodological approaches.nnnMETHODSnUsing the meta-narrative method and searching beyond the Medline-indexed literature, this review used conflicting findings to address higher-order questions about how researchers had differently conceptualized and studied the EPR and its implementation.nnnFINDINGSnTwenty-four previous systematic reviews and ninety-four further primary studies were considered. Key tensions in the literature centered on (1) the EPR (container or itinerary); (2) the EPR user (information-processer or member of socio-technical network); (3) organizational context (the setting within which the EPR is implemented or the EPR-in-use); (4) clinical work (decision making or situated practice); (5) the process of change (the logic of determinism or the logic of opposition); (6) implementation success (objectively defined or socially negotiated); and (7) complexity and scale (the bigger the better or small is beautiful).nnnCONCLUSIONSnThe findings suggest that EPR use will always require human input to recontextualize knowledge; that even though secondary work (audit, research, billing) may be made more efficient by the EPR, primary clinical work may be made less efficient; that paper may offer a unique degree of ecological flexibility; and that smaller EPR systems may sometimes be more efficient and effective than larger ones. We suggest an agenda for further research.

RAMESES publication standards: realist syntheses

BackgroundThere is growing interest in realist synthesis as an alternative systematic review method. This approach offers the potential to expand the knowledge base in policy-relevant areas -for example, by explaining the success, failure or mixed fortunes of complex interventions. No previous publication standards exist for reporting realist syntheses. This standard was developed as part of the RAMESES (Realist And MEta-narrative Evidence Syntheses: Evolving Standards) project. The projects aim is to produce preliminary publication standards for realist systematic reviews.MethodsWe (a) collated and summarized existing literature on the principles of good practice in realist syntheses; (b) considered the extent to which these principles had been followed by published syntheses, thereby identifying how rigor may be lost and how existing methods could be improved; (c) used a three-round online Delphi method with an interdisciplinary panel of national and international experts in evidence synthesis, realist research, policy and/or publishing to produce and iteratively refine a draft set of methodological steps and publication standards; (d) provided real-time support to ongoing realist syntheses and the open-access RAMESES online discussion list so as to capture problems and questions as they arose; and (e) synthesized expert input, evidence syntheses and real-time problem analysis into a definitive set of standards.ResultsWe identified 35 published realist syntheses, provided real-time support to 9 on-going syntheses and captured questions raised in the RAMESES discussion list. Through analysis and discussion within the project team, we summarized the published literature and common questions and challenges into briefing materials for the Delphi panel, comprising 37 members. Within three rounds this panel had reached consensus on 19 key publication standards, with an overall response rate of 91%.ConclusionThis project used multiple sources to develop and draw together evidence and expertise in realist synthesis. For each item we have included an explanation for why it is important and guidance on how it might be reported. Realist synthesis is a relatively new method for evidence synthesis and as experience and methodological developments occur, we anticipate that these standards will evolve to reflect further methodological developments. We hope that these standards will act as a resource that will contribute to improving the reporting of realist syntheses.To encourage dissemination of the RAMESES publication standards, this article is co-published in the Journal of Advanced Nursing and is freely accessible on Wiley Online Library (http://www.wileyonlinelibrary.com/journal/jan). Please see related article http://www.biomedcentral.com/1741-7015/11/20 and http://www.biomedcentral.com/1741-7015/11/22

Realist methods in medical education research: what are they and what can they contribute?

Medical Education 2012: 46: 89–96

Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)

BackgroundThere is growing interest in theory-driven, qualitative and mixed-method approaches to systematic review as an alternative to (or to extend and supplement) conventional Cochrane-style reviews. These approaches offer the potential to expand the knowledge base in policy-relevant areas - for example by explaining the success, failure or mixed fortunes of complex interventions. However, the quality of such reviews can be difficult to assess. This study aims to produce methodological guidance, publication standards and training resources for those seeking to use the realist and/or meta-narrative approach to systematic review.Methods/designWe will: [a] collate and summarise existing literature on the principles of good practice in realist and meta-narrative systematic review; [b] consider the extent to which these principles have been followed by published and in-progress reviews, thereby identifying how rigour may be lost and how existing methods could be improved; [c] using an online Delphi method with an interdisciplinary panel of experts from academia and policy, produce a draft set of methodological steps and publication standards; [d] produce training materials with learning outcomes linked to these steps; [e] pilot these standards and training materials prospectively on real reviews-in-progress, capturing methodological and other challenges as they arise; [f] synthesise expert input, evidence review and real-time problem analysis into more definitive guidance and standards; [g] disseminate outputs to audiences in academia and policy. The outputs of the study will be threefold:1. Quality standards and methodological guidance for realist and meta-narrative reviews for use by researchers, research sponsors, students and supervisors2. A RAMESES (Realist and Meta-review Evidence Synthesis: Evolving Standards) statement (comparable to CONSORT or PRISMA) of publication standards for such reviews, published in an open-access academic journal.3. A training module for researchers, including learning outcomes, outline course materials and assessment criteria.DiscussionRealist and meta-narrative review are relatively new approaches to systematic review whose overall place in the secondary research toolkit is not yet fully established. As with all secondary research methods, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency of studies.

Realist RCTs of complex interventions - An oxymoron

Bonell et al. discuss the challenges of carrying out randomised controlled trials (RCTs) to evaluate complex interventions in public health, and consider the role of realist evaluation in enhancing this design (Bonell, Fletcher, Morton, Lorenc, & Moore, 2012). They argue for a synergistic, rather than oppositional relationship between realist and randomised evaluation and that it is possible to benefit from the insights provided by realist evaluation without relinquishing the RCT as the best means of examining intervention causality. We present counter-arguments to their analysis of realist evaluation and their recommendations for realist RCTs. Bonell et al. are right to question whether and how (quasi-)experimental designs can be improved to better evaluate complex public health interventions. However, the paper does not explain how a research design that is fundamentally built upon a positivist ontological and epistemological position can be meaningfully adapted to allow it to be used from within a realist paradigm. The recommendations for realist RCTs do not sufficiently take into account important elements of complexity that pose major challenges for the RCT design. They also ignore key tenets of the realist evaluation approach. We propose that the adjective realist should continue to be used only for studies based on a realist philosophy and whose analytic approach follows the established principles of realist analysis. It seems more correct to call the approach proposed by Bonell and colleagues theory informed RCT, which indeed can help in enhancing RCTs.

Known knowns, known unknowns, unknown unknowns: The predicament of evidence-based policy

The authors present a case study examining the potential for policies to be “evidence-based.” To what extent is it possible to say that a decision to implement a complex social intervention is warranted on the basis of available empirical data? The case chosen is whether there is sufficient evidence to justify banning smoking in cars carrying children. The numerous assumptions underpinning such legislation are elicited, the weight and validity of evidence for each is appraised, and a mixed picture emerges. Certain propositions seem well supported; others are not yet proven and possibly unknowable. The authors argue that this is the standard predicament of evidence-based policy. Evidence does not come in finite chunks offering certainty and security to policy decisions. Rather, evidence-based policy is an accumulative process in which the data pursue but never quite capture unfolding policy problems. The whole point is the steady conversion of “unknowns” to “knowns.”

Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment.

Realist review has increased in popularity as a methodology for complex intervention assessment. Our experience suggests that the process of designing a realist review requires its customization to areas under investigation. To elaborate on this idea, we first describe the logic underpinning realist review and then present critical reflections on our application experience, organized in seven areas. These are the following: (1) the challenge of identifying middle range theory; (2) addressing heterogeneity and lack of conceptual clarity; (3) the challenge of appraising the quality of complex evidence; (4) the relevance of capturing unintended outcomes; (5) understanding the process of context, mechanism, and outcome (CMO) configuring; (6) incorporating middle-range theory in the CMO configuration process; and (7) using middle range theory to advance the conceptualization of outcomes - both visible and seemingly hidden. One conclusion from our experience is that the degree of heterogeneity of the evidence base will determine whether theory can drive the development of review protocols from the outset, or will follow only after an intense period of data immersion. We hope that presenting a critical reflection on customizing realist review will convey how the methodology can be tailored to the often complex and idiosyncratic features of health research, leading to innovative evidence syntheses.

Policy guidance on threats to legislative interventions in public health: a realist synthesis

BackgroundLegislation is one of the most powerful weapons for improving population health and is often used by policy and decision makers. Little research exists to guide them as to whether legislation is feasible and/or will succeed. We aimed to produce a coherent and transferable evidence based framework of threats to legislative interventions to assist the decision making process and to test this through the case study of legislation to ban smoking in cars carrying children.MethodsWe conceptualised legislative interventions as a complex social interventions and so used the realist synthesis method to systematically review the literature for evidence. 99 articles were found through searches on five electronic databases (MEDLINE, HMIC, EMBASE, PsychINFO, Social Policy and Practice) and iterative purposive searching. Our initial searches sought any studies that contained information on smoking in vehicles carrying children. Throughout the review we continued where needed to search for additional studies of any type that would conceptually contribute to helping build and/or test our framework.ResultsOur framework identified a series of transferable threats to public health legislation. When applied to smoking bans in vehicles; problem misidentification; public support; opposition; and enforcement issues were particularly prominent threats. Our framework enabled us to understand and explain the nature of each threat and to infer the most likely outcome if such legislation were to be proposed in a jurisdiction where no such ban existed.Specifically, the micro-environment of a vehicle can contain highly hazardous levels of second hand smoke. Public support for such legislation is high amongst smokers and non-smokers and their underlying motivations were very similar - wanting to practice the Millian principle of protecting children from harm. Evidence indicated that the tobacco industry was not likely to oppose legislation and arguments that such a law would be unenforceable were unfounded.ConclusionIt is possible to develop a coherent and transferable evidence based framework of the ideas and assumptions behind the threats to legislative intervention that may assist policy and decision makers to analyse and judge if legislation is feasible and/or likely to succeed.

How Design of Places Promotes or Inhibits Mobility of Older Adults Realist Synthesis of 20 Years of Research

Objective: The objective of this study was to determine the environmental features that best support aging in place. Method: We conducted a realist synthesis, a theory-driven interpretive method of evidence synthesis, of 120+ articles (published 1991-2011) that attempts to explain how place may influence older adults’ decisions about mobility (e.g., physical activity). We developed an initial program theory, reviewed the literature, identified outcomes, analyzed and synthesized patterns, and created a final program theory. Results: Safety was a central mechanism, serving as one of the bridges between environmental components (e.g., connectivity, aesthetics, retail and services) and decisions about mobility. Population density, sidewalk presence, and park proximity did not emerge as key factors. Discussion: Safety considerations are one of the most prominent influences of older adults’ decisions about mobility. Street connectivity, pedestrian access and transit, and retail and services were also important. These factors are amenable to change and can help promote mobility for older adults.

Putting your course on the web: lessons from a case study and systematic literature review

Background u2002Education via the Internet offers enormous potential, but many online courses are pedagogically or technically weak and many good projects are never mainstreamed.