The “inconvenient truth” about AI in healthcare

Abstract

As the UK sits in painful deadlock over Brexit, it is important to remember that governments are regularly faced with crises, and their responses can create enduring benefit for future generations. Back in 1858, for example, the UK parliament was dealing with another messy crisis: “the great stink.” In a world before sanitation, the river Thames had become an open latrine, and as summer blossomed parliament was engulfed in a pestilential stench. £2.5 million (about £300 million in today’s money) was hastily approved to build a network of sewers throughout the capital. This particular model of sanitation, developed by Bazalgette, was adopted by other cities around the world and the rest, as they say, is history. It is now unthinkable that a developed nation would not have sanitation infrastructure. However, back in 1858 the debate was whether sanitation infrastructure was worthy of investment and whether it was a public or private good. A similar debate has been simmering for some time regarding health data infrastructure, defined as the hardware and software to securely aggregate, store, process and transmit healthcare data. Is data infrastructure necessary for healthcare organizations and if so, is it the responsibility of individual healthcare organizations, of local health systems, or is it a public good? In the 21st Century, the age of big data and artificial intelligence (AI), each healthcare organization has built its own data infrastructure to support its own needs, typically involving onpremises computing and storage. Data is balkanized along organizational boundaries, severely constraining the ability to provide services to patients across a care continuum within one organization or across organizations. This situation evolved as individual organizations had to buy and maintain the costly hardware and software required for healthcare, and has been reinforced by vendor lock-in, most notably in electronic medical records (EMRs). With increasing cost pressure and policy imperatives to manage patients across and between care episodes, the need to aggregate data across and between departments within a healthcare organization and across disparate organizations has become apparent not only to realize the promise of AI but also to improve the efficiency of existing data intensive tasks such as any population level segmentation and patient safety monitoring. The rapid explosion in AI has introduced the possibility of using aggregated healthcare data to produce powerful models that can automate diagnosis and also enable an increasingly precision approach to medicine by tailoring treatments and targeting resources with maximum effectiveness in a timely and dynamic manner. However, “the inconvenient truth” is that at present the algorithms that feature prominently in research literature are in fact not, for the most part, executable at the frontlines of clinical practice. This is for two reasons: first, these AI innovations by themselves do not re-engineer the incentives that support existing ways of working. A complex web of ingrained political and economic factors as well as the proximal influence of medical practice norms and commercial interests determine the way healthcare is delivered. Simply adding AI applications to a fragmented system will not create sustainable change. Second, most healthcare organizations lack the data infrastructure required to collect the data needed to optimally train algorithms to (a) “fit” the local population and/or the local practice patterns, a requirement prior to deployment that is rarely highlighted by current AI publications, and (b) interrogate them for bias to guarantee that the algorithms perform consistently across patient cohorts, especially those who may not have been adequately represented in the training cohort. For example, an algorithm trained on mostly Caucasian patients is not expected to have the same accuracy when applied to minorities. In addition, such rigorous evaluation and re-calibration must continue after implementation to track and capture those patient demographics and practice patterns which inevitably change over time. Some of these issues can be addressed through external validation, the importance of which is not unique to AI, and it is timely that existing standards for prediction model reporting are being updated specifically to incorporate standards applicable to this end. In the United States, there are islands of aggregated healthcare data in the ICU, and in the Veterans Administration. These aggregated data sets have predictably catalyzed an acceleration in AI development; but without broader development of data infrastructure outside these islands it will not be possible to generalize these innovations. Elsewhere in the economy, the development of cloud computing, secure high-performance general use data infrastructure and services available via the Internet (the “cloud”), has been a significant enabler for large and small technology companies alike, providing significantly lower fixed costs and higher performance as well as supporting the aforementioned opportunities for AI. Healthcare, with its abundance of data, is in theory well-poised to benefit from growth in cloud computing. The largest and arguably most valuable store of data in healthcare rests in EMRs. However, clinician satisfaction with EMRs remains low, resulting in variable completeness and quality of data entry, and interoperability between different providers remains elusive. The typical lament of a harried clinician is still “why does my EMR still suck and why don’t all these systems just talk to each other?” Policy imperatives have attempted to address these dilemmas, however progress has been minimal. In spite of the widely touted benefits of “data