Kieran Walshe

Realist review - a new method of systematic review designed for complex policy interventions:

Evidence-based policy is a dominant theme in contemporary public services but the practical realities and challenges involved in using evidence in policy-making are formidable. Part of the problem is one of complexity. In health services and other public services, we are dealing with complex social interventions which act on complex social systems-things like league tables, performance measures, regulation and inspection, or funding reforms. These are not ‘magic bullets‘ which will always hit their target, but programmes whose effects are crucially dependent on context and implementation. Traditional methods of review focus on measuring and reporting on programme effectiveness, often find that the evidence is mixed or conflicting, and provide little or no clue as to why the intervention worked or did not work when applied in different contexts or circumstances, deployed by different stakeholders, or used for different purposes. This paper offers a model of research synthesis which is designed to work with complex social interventions or programmes, and which is based on the emerging ‘realist’ approach to evaluation. It provides an explanatory analysis aimed at discerning what works for whom, in what circumstances, in what respects and how. The first step is to make explicit the programme theory (or theories) - the underlying assumptions about how an intervention is meant to work and what impacts it is expected to have. We then look for empirical evidence to populate this theoretical framework, supporting, contradicting or modifying the programme theories as it goes. The results of the review combine theoretical understanding and empirical evidence, and focus on explaining the relationship between the context in which the intervention is applied, the mechanisms by which it works and the outcomes which are produced. The aim is to enable decision-makers to reach a deeper understanding of the intervention and how it can be made to work most effectively. Realist review does not provide simple answers to complex questions. It will not tell policy-makers or managers whether something works or not, but will provide the policy and practice community with the kind of rich, detailed and highly practical understanding of complex social interventions which is likely to be of much more use to them when planning and implementing programmes at a national, regional or local level.

Evidence-based Management: From Theory to Practice in Health Care

The rise of evidence-based clinical practice in health care has caused some people to start questioning how health care managers and policymakers make decisions, and what role evidence plays in the process. Though managers and policymakers have been quick to encourage clinicians to adopt an evidence-based approach, they have been slower to apply the same ideas to their own practice. Yet, there is evidence that the same problems (of the underuse of effective interventions and the overuse of ineffective ones) are as widespread in health care management as they are in clinical practice. Because there are important differences between the culture, research base, and decision-making processes of clinicians and managers, the ideas of evidence-based practice, while relevant, need to be translated for management rather than simply transferred. The experience of the Center for Health Management Research (CHMR) is used to explore how to bring managers and researchers together and promote the use of evidence in managerial decision-making. However, health care funders, health care organizations, research funders, and academic centers need wider and more concerted action to promote the development of evidence-based managerial practice.

The Top Patient Safety Strategies That Can Be Encouraged for Adoption Now

Over the past 12 years, since the publication of the Institute of Medicines report, “To Err is Human: Building a Safer Health System,” improving patient safety has been the focus of considerable public and professional interest. Although such efforts required changes in policies; education; workforce; and health care financing, organization, and delivery, the most important gap has arguably been in research. Specifically, to improve patient safety we needed to identify hazards, determine how to measure them accurately, and identify solutions that work to reduce patient harm. A 2001 report commissioned by the Agency for Healthcare Research and Quality, “Making Health Care Safer: A Critical Analysis of Patient Safety Practices” (1), helped identify some early evidence-based safety practices, but it also highlighted an enormous gap between what was known and what needed to be known.

Advancing the science of patient safety

Despite a decades worth of effort, patient safety has improved slowly, in part because of the limited evidence base for the development and widespread dissemination of successful patient safety practices. The Agency for Healthcare Research and Quality sponsored an international group of experts in patient safety and evaluation methods to develop criteria to improve the design, evaluation, and reporting of practice research in patient safety. This article reports the findings and recommendations of this group, which include greater use of theory and logic models, more detailed descriptions of interventions and their implementation, enhanced explanation of desired and unintended outcomes, and better description and measurement of context and of how context influences interventions. Using these criteria and measuring and reporting contexts will improve the science of patient safety.

Rules and guidelines in clinical practice: a qualitative study in operating theatres of doctors’ and nurses’ views

Background: The current orthodoxy within patient safety research and policy is characterised by a faith in rules based systems which limit the capacity for individual discretion, and hence fallibility. However, guidelines have been seen as stifling innovation and eroding trust. Our objectives were to explore the attitudes towards guidelines of doctors and nurses working together in surgical teams and to examine the extent to which trusting relationships are maintained in a context governed by explicit rules. Methods: Fourteen consultant grade surgeons of mixed specialty, 12 consultant anaesthetists, and 15 nurses were selected to reflect a range of roles. Participant observation was combined with semi-structured interviews. Results: Doctors’ views about the contribution of guidelines to safety and to clinical practice differed from those of nurses. Doctors rejected written rules, instead adhering to the unwritten rules of what constitutes acceptable behaviour for members of the medical profession. In contrast, nurses viewed guideline adherence as synonymous with professionalism and criticised doctors for failing to comply with guidelines. Conclusions: While the creation of a “safety culture” requires a shared set of beliefs, attitudes and norms in relation to what is seen as safe clinical practice, differences of opinion on these issues exist which cannot be easily reconciled since they reflect deeply ingrained beliefs about what constitutes professional conduct. While advocates of standardisation (such as nurses) view doctors as rule breakers, doctors may not necessarily regard guidelines as legitimate or identify with the rules written for them by members of other social groups. Future safety research and policy should attempt to understand the unwritten rules which govern clinical behaviour and examine the ways in which such rules are produced, maintained, and accepted as legitimate.

Effectiveness of quality improvement: learning from evaluations

The effectiveness of many quality improvement interventions has been studied, and research suggests that most have highly variable effects which depend heavily on the context in which they are used and the way they are implemented. This has three important implications. Firstly, it means that the approach to quality improvement used in an organisation probably matters less than how and by whom it is used. Rather than taking up, trying, and then discarding a succession of different quality improvement techniques, organisations should probably choose one carefully and then persevere to make it work. Secondly, future research into quality improvement interventions should be directed more at understanding how and why they work—the determinants of effectiveness—rather than measuring whether they work. Thirdly, some element of evaluation should be incorporated into every quality improvement programme so that its effectiveness can be monitored and the information can be used to improve the systems for improvement.

Primary care trusts

Premature reorganisation, with mergers, may be harmful

The “redisorganisation” of the NHS: Another reorganisation involving unhappy managers can only worsen the service

The NHS is being reorganised—again. Having declared on taking office in 1997 that it recognised that the NHS had suffered too much structural reform, the re-elected Labour government has embarked on the largest, and least debated, reorganisation of the NHS for two decades.1 A consultation document, “Shifting the balance of power in the NHS: securing delivery,”2 published in July proposed abolishing the executive regional offices of the NHS and two thirds of health authorities and creating new primary care trusts to take on a raft of responsibilities from health authorities. Only the acute NHS trusts emerge from these changes relatively unscathed. The consultation, which lasted six weeks, closed in early September and the government has yet to publish its results. But the reorganisation is steaming ahead regardless, with the aim of completing all the changes by April 2002. Few people outside the NHS management community seem to be aware of the exact nature and implications of these changes, which have their roots in growing public and political impatience with the quality of NHS services. From two recently published surveys 3 4 it appears that …

The rise of regulation in the NHS

The current British government has created five national agencies to regulate the NHS. These new arrivals are simply additions to an already crowded regulatory landscape. But, if politicians can be persuaded to let go, the new regulators of the NHS could provide a genuinely new approach to improving performance and management During the past four years the British government has created five new national agencies to regulate the NHS in England (box).1 2 3 4 The government has moved away from using markets, competition, and contracting to manage performance in the NHS. But it has been unwilling to rely on traditional bureaucratic structures to exert control, and has turned increasingly to regulation. In this article, I describe how and why organisational regulation in the NHS in England has grown in recent years. I examine how regulation was used in the NHS in the past and describe the characteristics of the new regulatory agencies. Finally, I use information from the wider literature on regulation to examine the regulatory model adopted by these new agencies and to explore what they might learn from regulation in other settings. #### Summary points Over the past 20 years, regulation has increased in Britains private and public sectors In the past four years the British government has created five new national agencies to regulate the NHS The new agencies could be the start of a new approach to improving performance and management in the NHS The new agencies could learn from regulators in other sectors and adopt a responsive approach to regulation #### New national agencies to regulate the NHS National Institute for Clinical Excellence Commission for Health Improvement Modernisation Agency National Patient Safety Agency National Clinical Assessment Authority Regulation is “sustained and focused control exercised by a public agency over activities which are valued by a community.”5 The key features of regulation are that …

The relationship between organizational culture and performance in acute hospitals.

This paper examines the relationship between senior management team culture and organizational performance in English acute hospitals (NHS Trusts) over three time periods between 2001/2002 and 2007/2008. We use a validated culture rating instrument, the Competing Values Framework, to measure senior management team culture. Organizational performance is assessed using a wide range of routinely collected indicators. We examine the associations between organizational culture and performance using ordered probit and multinomial logit models. We find that organizational culture varies across hospitals and over time, and this variation is at least in part associated in consistent and predictable ways with a variety of organizational characteristics and routine measures of performance. Moreover, hospitals are moving towards more competitive culture archetypes which mirror the current policy context, though with a stronger blend of cultures. The study provides evidence for a relationship between culture and performance in hospital settings.