Jill Russell

Adoption, non-adoption, and abandonment of a personal electronic health record: case study of HealthSpace

Objective To evaluate the policy making process, implementation by NHS organisations, and patients’ and carers’ experiences of efforts to introduce an internet accessible personal electronic health record (HealthSpace) in a public sector healthcare system. Design Mixed method, multilevel case study. Setting English National Health Service; the basic HealthSpace technology (available throughout England) and the advanced version (available in a few localities where this option had been introduced) were considered. Main outcome measures National statistics on invitations sent, HealthSpace accounts created, and interviews and ethnographic observation of patients and carers. Data analysis was informed by a socio-technical approach which considered macro and micro influences on both adoption and non-adoption of innovations, and by the principles of critical discourse analysis. Participants 56 patients and carers (of whom 21 opened a basic HealthSpace account, 20 had diabetes but were not initially using HealthSpace, and 15 used advanced HealthSpace accounts to exchange messages with their general practitioner), 3000 pages of documents (policies, strategies, business plans, minutes of meetings, correspondence), observational field notes, and 160 interviews with policy makers, project managers, and clinical staff. Results Between 2007 and October 2010, 172 950 people opened a basic HealthSpace account. 2913 (0.13% of those invited) opened an advanced account, compared with 5-10% of the population anticipated in the original business case. Overall, patients perceived HealthSpace as neither useful nor easy to use and its functionality aligned poorly with their expectations and self management practices. Those who used email-style messaging were positive about its benefits, but enthusiasm beyond three early adopter clinicians was low, and fewer than 100 of 30 000 patients expressed interest. Policy makers’ hopes that “deploying” HealthSpace would lead to empowered patients, personalised care, lower NHS costs, better data quality, and improved health literacy were not realised over the three year evaluation period. Conclusion Unless personal electronic health records align closely with people’s attitudes, self management practices, identified information needs, and the wider care package (including organisational routines and incentive structures for clinicians), the risk that they will be abandoned or not adopted at all is substantial. Conceptualising such records dynamically (as components of a socio-technical network) rather than statically (as containers for data) and employing user centred design techniques might improve their chances of adoption and use. The findings raise questions about how eHealth programmes in England are developed and approved at policy level.

Introduction of shared electronic records: multi-site case study using diffusion of innovation theory

Objective To explore the introduction of a centrally stored, shared electronic patient record (the summary care record (SCR)) in England and draw wider lessons about the implementation of large scale information technology projects in health care. Design Multi-site, mixed method case study applying utilisation focused evaluation. Setting Four early adopter sites for the SCR in England—three in urban areas of relative socioeconomic deprivation and the fourth in a relatively affluent rural area. Data sources and analysis Data included 250 staff interviews, 1500 hours of ethnographic observation, interviews and focus groups with 170 patients and carers, 2500 pages of correspondence and documentary evidence, and incorporation of relevant surveys and statistics produced by others. These were analysed by using a thematic approach drawing on (and extending) a theoretical model of complex change developed in a previous systematic review. Main findings The mixed fortunes of the SCR programme in its first year were largely explained by eight interacting influences. The first was the SCR’s material properties (especially technical immaturity and lack of interoperability) and attributes (especially the extent to which potential adopters believed the benefits outweighed the risks). The second was adopters’ concerns (especially about workload and the ethicality of sharing “confidential” information on an implied consent model). The third influence was interpersonal influence (for example, opinion leaders, champions, facilitators), and the fourth was organisational antecedents for innovation (for example past experience with information technology projects, leadership and management capacity, effective data capture systems, slack resources). The fifth was organisational readiness for the SCR (for example, innovation-system fit, tension for change, power balances between supporters and opponents, baseline data quality). The sixth was the implementation process (including the nature of the change model and the extent to which new routines associated with the SCR aligned with existing organisational routines). The seventh influence was the nature and quality of links between different parts of the system, and the final one was the wider environment (especially the political context of the programme). Conclusion Shared electronic records are not plug-in technologies. They are complex innovations that must be accepted by individual patients and staff and also embedded in organisational and inter-organisational routines. This process is heavily influenced at the micro-level by the material properties of the technology, individuals’ attitudes and concerns, and interpersonal influence; at the meso-level by organisational antecedents, readiness, and operational aspects of implementation; and at the macro-level by institutional and socio-political forces. A case study approach and multi-level theoretical analysis can illuminate how contextual factors shape, enable, and constrain new, technology supported models of patient care.

Adoption and non-adoption of a shared electronic summary record in England: a mixed-method case study

Objective To evaluate a national programme to develop and implement centrally stored electronic summaries of patients’ medical records. Design Mixed-method, multilevel case study. Setting English National Health Service 2007-10. The summary care record (SCR) was introduced as part of the National Programme for Information Technology. This evaluation of the SCR considered it in the context of national policy and its frontline implementation and use in three districts. Participants and methods Quantitative data (cumulative records created nationally plus a dataset of 416 325 encounters in participating primary care out-of-hours and walk-in centres) were analysed statistically. Qualitative data (140 interviews including policy makers, managers, clinicians, and software suppliers; 2000 pages of ethnographic field notes including observation of 214 clinical consultations; and 3000 pages of documents) were analysed thematically and interpretively. Results Creating individual SCRs and supporting their adoption and use was a complex, technically challenging, and labour intensive process that occurred more slowly than planned. By early 2010, 1.5 million such records had been created. In participating primary care out-of-hours and walk-in centres, an SCR was accessed in 4% of all encounters and in 21% of encounters where one was available; these figures were rising in some but not all sites. The main determinant of SCR access was the identity of the clinician: individual clinicians accessed available SCRs between 0 and 84% of the time. When accessed, an SCR seemed to support better quality care and increase clinician confidence in some encounters. There was no direct evidence of improved safety, but findings were consistent with a rare but important positive impact on preventing medication errors. SCRs sometimes contained incomplete or inaccurate data, but clinicians drew judiciously on these data along with other sources. SCR use was not associated with shorter consultations or reduction in onward referral. Successful introduction of SCRs depended on interaction between multiple stakeholders from different worlds (clinical, political, technical, commercial) with different values, priorities, and ways of working. The programme’s fortunes seemed to turn on the ability of change agents to bridge these different institutional worlds, align their conflicting logics, and mobilise implementation effort. Conclusions Benefits of centrally stored electronic summary records seem more subtle and contingent than many stakeholders anticipated, and clinicians may not access them. Complex interdependencies, inherent tensions, and high implementation workload should be expected when they are introduced on a national scale.

Why Do Evaluations of eHealth Programs Fail? An Alternative Set of Guiding Principles

Trisha Greenhalgh and Jill Russell discuss the relative merits of “scientific” and “social practice” approaches to evaluation and argue that eHealth evaluation is in need of a paradigm shift.

Narrative methods in quality improvement research

This paper reviews and critiques the different approaches to the use of narrative in quality improvement research. The defining characteristics of narrative are chronology (unfolding over time); emplotment (the literary juxtaposing of actions and events in an implicitly causal sequence); trouble (that is, harm or the risk of harm); and embeddedness (the personal story nests within a particular social, historical and organisational context). Stories are about purposeful action unfolding in the face of trouble and, as such, have much to offer quality improvement researchers. But the quality improvement report (a story about efforts to implement change), which is common, must be distinguished carefully from narrative based quality improvement research (focused systematic enquiry that uses narrative methods to generate new knowledge), which is currently none. We distinguish four approaches to the use of narrative in quality improvement research—narrative interview; naturalistic story gathering; organisational case study; and collective sense-making—and offer a rationale, describe how data can be collected and analysed, and discuss the strengths and limitations of each using examples from the quality improvement literature. Narrative research raises epistemological questions about the nature of narrative truth (characterised by sense-making and emotional impact rather than scientific objectivity), which has implications for how rigour should be defined (and how it might be achieved) in this type of research. We offer some provisional guidance for distinguishing high quality narrative research in a quality improvement setting from other forms of narrative account such as report, anecdote, and journalism.

Recognizing rhetoric in health care policy analysis

Critiques of the ‘naïve rationalist’ model of policy-making abound in the sociological and political science literature. Yet academic debate on health care policy-making continues to be couched in the dominant discourse of evidence-based medicine, whose underlying assumptions - that policies are driven by facts rather than values and these can be clearly separated; that ‘evidence’ is context-free, can be objectively weighed up and placed unproblematically in a ‘hierarchy’; and that policy-making is essentially an exercise in decision science - have constrained both thinking and practice. In this paper, drawing on theoretical work from political science and philosophy, and innovative empirical work in the health care sector, we argue that health care is well overdue for a re-defining of what policy-making is. Policy-making is the formal struggle over ideas and values, played out by the rhetorical use of language and the enactment of social situations. While the selection, evaluation and implementation of research evidence are important in the policy-making process, they do not equate to that process. The study of argument in the construction of policy has the potential to illuminate dimensions of the process that are systematically occluded when policy-making is studied through a naïve rationalist lens. In particular, a rhetorical perspective highlights the struggle over ideas, the ‘naming and framing’ of policy problems, the centrality of audience and the rhetorical use of language in discussion to increase the audiences adherence to particular framings and proposals. Rhetorical theory requires us to redefine what counts as ‘rationality’ - which must extend from what is provably true (by logic) and probably true (by Bayesian reasoning) to embrace, in addition, that which is plausibly true (i.e. can convince a reasonable audience). Future research into health care policy-making needs to move beyond the study of ‘getting evidence into practice’ and address the language, arguments and discourse through which policy is constructed and enacted.

Evidence-Based Policymaking: A Critique

The idea that policy should be based on best research evidence might appear to be self-evident. But a closer analysis reveals a number of problems and paradoxes inherent in the concept of “evidence-based policymaking.” The current conflict over evidence-based policymaking parallels a long-standing “paradigm war” in social research between positivist, interpretivist, and critical approaches. This article draws from this debate in order to inform the discussions over the appropriateness of evidence- based policymaking and the related question of what is the nature of policymaking. The positivist, empiricist worldview that underpins the theory and practice of evidence-based medicine (EBM) fails to address key elements of the policymaking process. In particular, a narrowly “evidence-based” framing of policymaking is inherently unable to explore the complex, context-dependent, and value-laden way in which competing options are negotiated by individuals and interest groups. Sociolinguistic tools such as argumentation theory offer opportunities for developing richer theories about how policymaking happens. Such tools also have potential practical application in the policymaking process: by enhancing participants’ awareness of their own values and those of others, the quality of the collective deliberation that lies at the heart of policymaking may itself improve.

Transferability of principles of evidence based medicine to improve educational quality: systematic review and case study of an online course in primary health care

The success of evidence based medicine has led to pressure to make medical education more evidence based. Greenhalgh and colleagues tested the transferability of these principles when developing a postgraduate course

Receptionist input to quality and safety in repeat prescribing in UK general practice: ethnographic case study

Objective To describe, explore, and compare organisational routines for repeat prescribing in general practice to identify contributors and barriers to safety and quality. Design Ethnographic case study. Setting Four urban UK general practices with diverse organisational characteristics using electronic patient records that supported semi-automation of repeat prescribing. Participants 395 hours of ethnographic observation of staff (25 doctors, 16 nurses, 4 healthcare assistants, 6 managers, and 56 reception or administrative staff), and 28 documents and other artefacts relating to repeat prescribing locally and nationally. Main outcome measures Potential threats to patient safety and characteristics of good practice. Methods Observation of how doctors, receptionists, and other administrative staff contributed to, and collaborated on, the repeat prescribing routine. Analysis included mapping prescribing routines, building a rich description of organisational practices, and drawing these together through narrative synthesis. This was informed by a sociological model of how organisational routines shape and are shaped by information and communications technologies. Results Repeat prescribing was a complex, technology-supported social practice requiring collaboration between clinical and administrative staff, with important implications for patient safety. More than half of requests for repeat prescriptions were classed as “exceptions” by receptionists (most commonly because the drug, dose, or timing differed from what was on the electronic repeat list). They managed these exceptions by making situated judgments that enabled them (sometimes but not always) to bridge the gap between the idealised assumptions about tasks, roles, and interactions that were built into the electronic patient record and formal protocols, and the actual repeat prescribing routine as it played out in practice. This work was creative and demanded both explicit and tacit knowledge. Clinicians were often unaware of this input and it did not feature in policy documents or previous research. Yet it was sometimes critical to getting the job done and contributed in subtle ways to safeguarding patients. Conclusion Receptionists and administrative staff make important “hidden” contributions to quality and safety in repeat prescribing in general practice, regarding themselves accountable to patients for these contributions. Studying technology-supported work routines that seem mundane, standardised, and automated, but which in reality require a high degree of local tailoring and judgment from frontline staff, opens up a new agenda for the study of patient safety.

Using the online environment in assessment for learning: a case‐study of a web‐based course in primary care

The development of e‐learning has opened up new opportunities for innovation in assessment practices in higher education. This descriptive case study draws upon staff and student experiences of teaching and learning on a web‐based Masters programme in primary health care to explore how specific features of the online environment can be exploited to promote assessment as part of learning. It begins by identifying different ways of conceptualising assessment in order to highlight the fundamental value choices facing those developing and delivering assessment systems, and then describes our own approach to assessment. In the second part of the paper we explore two key ways in which the online learning environment enables assessment to contribute to learning—through its potential to support collaborative learning, and through facilitating high quality feedback between teachers and students.