Artificial neural networks for disease trajectory prediction in the context of sepsis
AA RTIFICIAL NEURAL NETWORKS FOR DISEASE TRAJECTORYPREDICTION IN THE CONTEXT OF SEPSIS
A P
REPRINT
Dale Larie ∗ Department of SurgeryUniversity of VermontBurlington, VT [email protected]
Gary An
Department of SurgeryUniversity of VermontBurlington, VT [email protected]
Chase Cockrell
Department of SurgeryUniversity of VermontBurlington, VT [email protected]
July 28, 2020 A BSTRACT
Introduction:
The disease trajectory for clinical sepsis, in terms of temporal cytokine and phenotypicdynamics, can be interpreted as a random dynamical system. The ability to make accurate predictionsabout patient state from clinical measurements has eluded the biomedical community, primarilydue to the paucity of relevant and high-resolution data.
Methods:
We have utilized two distinctneural network architectures, Long Short-Term Memory and Multi-Layer Perceptron, to take atime sequence of five measurements of eleven simulated serum cytokine concentrations as inputand to return both the future cytokine trajectories as well as an aggregate metric representing thepatient’s state of health. We performed this work with two distinct training sets: one consisted ofcytokine trajectories starting immediately after the simulated injury and one consisted of cytokinetrajectories beginning 24 hours after the simulated injury.
Results:
The neural networks convergedwithin 50 epochs for cytokine trajectory predictions and health-metric regressions, with the expectedamount of error (due to stochasticity in the simulation). The mapping from a specific cytokineprofile to a state-of-health is not unique, and increased levels of inflammation result in less accuratepredictions and increased amounts of stochasticity. The Multi-Layer Perceptron neural network,trained on trajectories beginning 24 hours post-injury performed the best. Due to the propagationof machine learning error combined with computational model stochasticity over time, the networkshould be re-grounded in reality daily as predictions can diverge from the true model trajectory asthe system evolves towards a probabilistic basin of attraction.
Discussion:
This work serves as aproof-of-concept for the use of artificial neural networks to predict disease progression in sepsis. Thiswork is not intended to replace a trained clinician, rather the goal is to augment their intuition withquantifiable statistical information to help them make the best decisions. We note that this relies on avalid computational model of the system in question, in this case, the innate immune system, as thereis not now, nor will there be in the near future, sufficient data to inform a machine-learning trained,artificially intelligent, controller.
Keywords
Sepsis · Inflammation · Machine Learning · Artificial Neural Networks · Multiscale Models · Simulation
There are approximately 1 million cases of sepsis in the United States each year, with a mortality rate between 28-50% [1]. Sepsis is a highly dynamic process, characterized phenotypically by features such as multi-system organfailure, and molecularly, by dysregulation of the body’s internal cytokine signaling network [2, 3, 4]. While careprocess improvements in the treatment of sepsis, such as the development of treatment bundles and practice guidelines, ∗ Use footnote for providing further information about author (webpage, alternative address)— not for acknowledging fundingagencies. a r X i v : . [ q - b i o . Q M ] J u l rtificial neural networks for disease trajectory prediction in the context of sepsis A P
REPRINT have improved clinical outcomes in the past few decades, the search for new drugs to treat the biological-basis ofsepsis has been marked by complete failure: there is currently not a single drug approved by the U.S. Food andDrug Administration that targets the underlying pathophysiology of sepsis [5, 6]. One of the major challenges indesigning therapies for sepsis is an inability to effectively forecast the disease trajectories of individual patients, therebylimiting the effective sub-stratification of this heterogeneous population into those biologically similar enough tocontrol. Existing means of classifying sepsis patients, such as with the Sequential Organ Failure Score (SOFA) [7]or various biomarker panels [8, 9, 10], while potentially useful for coarse-grained outcome risk stratification, areonly able to provide population-level projections that cannot effectively be updated to an individual patient’s diseasecourse. Adding to the limitations of data-centric population-based scoring systems is the inherent stochasticity ofthe biological processes driving sepsis. The presence of stochasticity in the system governing inflammation makesaccurately predicting the entire trajectory of the disease, or accurately predicting the patient state 30 days into the future,given one point of assessment, an impossibility.Ultimately, the biological heterogeneity seen clinically is a combination of inter-patient (genetic variability) and intra-patient (stochastic processes) heterogeneity. The result is that it is impossible to comprehensively enumerate all possiblebiomarker states and configurations (i.e. phenotypes) that can be generated from a specific systemic perturbation orinjury. The challenge (and solution) is similar to that faced by Q-Learning [11] (now Deep Reinforcement Learning);Q-learning is a type of reinforcement learning in which agents determine what action to take (a) by looking up theircurrent state (s) in the lookup table, Q(s,a) that lists the probability of a desirable outcome based on that decision.Because the lookup table needs to provide this probability to guide the decision process it requires a finite (andcomputationally tractable) state space. In order to work effectively in continuous (infinite states) search spaces, Q-learning takes utilizes the Universal Approximation Theorem [12], which states that a feed-forward neural network canapproximate, to arbitrary fidelity, a real and continuous function. In the case of Q-learning, it is the lookup table thatis being approximated; we note that the lookup table does not necessarily meet the strict mathematical definition forcontinuity, however the technique works in practice. Personalized medicine faces a similar challenge: it is impossibleto comprehensively enumerate the set of all possible clinical observables (cytokine profiles, vital signs, etc.) which apatient can present – these state predictions must then be approximated. Furthermore, being able to eventually discoverand evaluate potential therapeutic regimens/control strategies requires the ability to evaluate counterfactuals: e.g. whatwould have happened had an intervention not been done? The ability to represent counterfactuals requires beingable to depict some future horizon of system behavior while accounting for the inherent stochasticity and behavioralheterogeneity of the system.Due to the stochastic nature of biomedical systems, predictions should be thought of as analogous to predicting the pathof a hurricane – making short-term predictions is possible, making long-term (and accurate/precise) predictions is not;in order to effectively predict the path of a hurricane, and mitigate the risks associated with it, the model is continuouslyupdated and new putative trajectories are calculated. Such an approach requires the ability to capture the dynamicprocesses that drive the behavior of the system, and as such mandate the use of mechanistic/quasi-mechanistic modelsthat can dynamically generate system trajectories.In previous work, we have demonstrated that the cytokine signaling network which controls the inflammatory processcan be modeled as a random dynamical system [2, 4], which is a system that evolves in time according to fixed rules,but also incorporates stochasticity [13, 14]. Knowledge of the underlying cellular and molecular processes of acuteinflammation has been used to create a dynamic model, the Innate Immune Response Agent-based Model (IIRABM)[15], that can serve as a proxy model for the development of more advanced prediction and forecasting methods. TheIIRABM has been used to demonstrate the use of in silico clinical trials as a means of evaluating the plausibility ofplanned potential interventions [15], provided fundamental insights into the mathematical and dynamic propertiesof sepsis that account for patient heterogeneity [2], demonstrating the futility of standard biomarker-based outcomeprediction [2], and served as a proxy model [16] for control discovery for sepsis. This most recent control discoverywork has employed advanced computational methods such as genetic algorithms/evolutionary computing [4] and deepreinforcement learning/artificial intelligence [17] to describe what would be required for multi-modal treatment ofsepsis. While the IIRABM is nearly 20 years old its central component structure remains valid and has predicted aseries of behaviors associated with sepsis that have since been recognized in the subsequent years, specifically thetemporal concurrence of pro- and anti-inflammatory cytokine responses (as opposed to sequential pro- and compensatoryresponses) [18, 19] and the importance of the immunoparalyzed recovery phase of sepsis, particularly with respect toits prolonged duration [20, 21, 22, 23]. The current work utilizes the IIRABM as a means of developing a method fordynamic trajectory prediction through the training and use of artificial neural networks (ANN).In the hospital setting, there is a paucity of population-level comprehensive data which can characterize a patient’shealth trajectory, causing machine-learning models that are informed by data alone, without an underlying mechanisticmodel or biological theory, to be brittle and of limited utility [24, 25]. The use of a properly validated computationalmodel, which can generate unlimited amounts of data, can address this shortcoming, however, in order for multiscale2rtificial neural networks for disease trajectory prediction in the context of sepsis
A P
REPRINT modeling and simulation to be deployed in clinical practice, it must be practical to utilize the models in a clinical setting.As we have shown in previous work [2], this requires an immense amount of computational power as the simulation hasto be repeated for many stochastic replicates. Compressing/approximating the information and dynamics containedwithin the computational model using an ANN allows for a computationally cheap and tractable method of rapidlyupdating predictions about patient disease trajectory as new information becomes available.
The foresting procedure is divided into two principal tasks: 1) predict future cytokine trajectories in an 11-dimensionalspace; and 2) regress the overall ‘health’ of the simulation as a function of its current cytokine profile. Training andvalidation data was generated using a previously validated computational model of the human immune response toinjury, the Innate Immune Response Agent-Based Model (IIRABM) [2]. The training/validation set was composed ofcytokine measurements for 11 unique cytokines over 10,000 time-steps in 66,000 in silico patients. Networks wereconstructed Using Keras [26], a TensorFlow based deep learning library for Python.
In order to forecast future values in the cytokine time series, we utilized long short-term memory (LSTM) recursiveneural networks (RNN). RNNs are different from standard multi-layer-perceptron networks because they have a neuralnetwork contained within a cell which takes information from the current input to help determine the adjusted state ofthe cell based on its current cell state. This adjusted cell state becomes the new cell state, and an output is determinedfor the network.LSTM networks’ memory cells have a unique structure, characterized by an input gate, two update layers, and an outputgate to determine the adjusted cell state [27]. The memory cells in LSTM networks allow for more long term memorythan typical RNNs which make them well suited for time-series analysis and prediction [28]. Noting this, an LSTMnetwork will likely be able to predict future cytokine levels, given that they are continuous and previous cytokine levelswill likely have a large impact on near-future values.We constructed a unique network for each cytokine that was to be predicted; each LSTM network takes 5 sequential11-d cytokine profiles as input and predicts the subsequent value(s). The first three layers of the network are 100-nodeLSTM layers; the output from these layers are fed into two fully connected layers of 300 and 200 nodes respectively,then to a single output node, resulting in 296,301 trainable parameters. Training data was arranged into 5 sequential11-dimensional points as training input features and the next 11-dimensional point as the training label. The data wasthen shuffled to avoid biasing the training. After data preprocessing, 8,576,100 data sequences and labels were used totrain the network. The loss metric used to train this network is mean absolute error (MAE), and the Adam optimizer[29]. Each network was trained until loss converged to a minimumFor the ultimate utilization of this network, 11 cytokine values are observed for 5 time steps, then a prediction for each ofthe next values is made using its own LSTM network. This set of 11 observations is combined into one 11-dimensionalpoint, which is then added to the original 5 samples as the next sample. Predictions are made recursively in this mannerfor 100 time steps after the initial observation. Accuracy of this algorithm was measured using the average MSE acrossthe 11 cytokine values at 1, 2, 3, 4, 5, 10, 25, 50, and 100 time steps after the initial observation. Prediction varianceand error bars were calculated through stochastic variations to the dropout layer [30], as demonstrated with regards toActive Learning for regression in [31].
The IIRABM uses the ‘Oxygen Deficit’ metric as a measure of health, where a low oxygen deficit is good, and ahigh oxygen deficit is bad. We note that, both in silico and in vivo, cytokine profiles provide a non-unique mappingto state-of-health (a concept which is more nebulously defined in vivo than in our in silico model). As such, error isexpected when attempting to regress from an 11d cytokine profile to a single health metric. In order to perform thisregression, we utilized a fully connected deep network which takes an 11-dimensional cytokine vector as input, feedinginto two fully connected layers with 1,500 nodes each, then into a layer with 150 nodes, and finally to a single outputnode. The loss metric used to train this algorithm is MSE. Using the regression network, a prediction of oxygen deficittrajectory can be made from the 11-dimensional matrix created by the LSTM network. Overall accuracy was measuredby comparing the oxygen deficit path to the predicted path and calculating the MSE.The prediction of whether an in silico patient will live or die, or the decision on whether or not pharmacologictherapeutics are more likely to be beneficial than detrimental, is ultimately based on the temporal trajectory of the3rtificial neural networks for disease trajectory prediction in the context of sepsis
A P
REPRINT
Figure 1: In Panel A we present the variance in oxygen deficit as a function of the sum of cytokine concentrations inthe whole area of simulated tissue. In Panel B, we show the mean absolute error in the regression of the oxygen deficitas a function of cytokine profile, as a function of training epoch.patient’s state of health (in this case, a measure of systemic oxygen deficit). The predicted trajectory in 11d cytokinespace is then fed into the health-metric regression network to forecast the most-likely outcome, time-to-outcome, andtime-horizon for potentially effective therapeutic interventions.Additionally, we created a multi-layer perceptron (MLP) to predict the future oxygen deficit trajectory as a functionof past values only, effectively treating the simulation output as a Markov chain. This network expects an input of 5sequential oxygen deficit values and will return a single future oxygen deficit value predicted for the next time step.The structure of this MLP begins with a fully connected layer of 1000 nodes, followed by a 1% permanent dropoutlayer, then two fully connected layers of 150 nodes each, followed by a single output node. This network was trainedusing a loss function to minimize MSE. Trajectory prediction for this network is made in the same recursive manner asthe cytokine prediction networks.Lastly as a comparison of the efficacy of LSTM neural networks, MLP prediction networks for each cytokine were alsocreated. Each network functionally acts the same as the LSTM networks, accepting 5 sequential 11-dimensional pointsin cytokine space and predicting the future value for a single cytokine. Each network has a structure beginning with afully connected layer of 1000 nodes, followed by a function to flatten the output shape from a 5 by 1000 array to asingle vector of length 5000. Next is another fully connected layer of 1000 nodes, then a 1% permanent dropout layer,feeding into a fully connected layer of 500 nodes, then another fully connected layer of 500 nodes, then finally to asingle output node. These networks were trained using a loss function to minimize MSE.
It is important to note that the map which translates a cytokine profile into its associated oxygen deficit (and vice-versa)is non-unique, and therefore some amount of error is expected and unavoidable. In Figure 1a, we present the variance inoxygen deficit as a function of the sum of cytokine concentrations in the whole area of simulated tissue. The sum ofcytokine concentrations is a coarse metric that roughly represents the amount of inflammation (no distinction is madebetween pro- and anti-inflammatory signals) and inflammatory signaling present in the model. This is analogous towhat is seen clinically – patients that see ostensibly identical insults/infections/injuries will invariably present a range ofresponses, in terms of temporal cytokine profiles or other clinical observables (heart-rate, blood pressure, temperature,etc.).This figure also illustrates a key difference between the structure of the noise in the IIRABM and the stochastic structurein a stochastic differential equation; the noise present in the IIRABM varies spatio-temporally and cannot be representedwith a closed-form analytical expression. Very generally speaking, the reason for this is that when cell-signaling is high,there is lots of activity in the model, and therefore lots of opportunities for stochastic events. This can be illustrated witha simple thought experiment: consider two system states, one with a single infected cell and one with 10 infected cells,and each infected cell has a probability of infecting a single neighbor, and some probability of healing. If we evolve thesimulation a single time step, system 1 can have 0, 1, or 2 infected cells, while the range of infected cells in system 2can vary from 0 to 20 (depending on the spatial configuration of the infected cells). In Figure 1b, we show the meanabsolute error as a function of training epoch when training the health regression neural net. The error quickly converges4rtificial neural networks for disease trajectory prediction in the context of sepsis
A P
REPRINT to a minimum with a relatively constant error of approximately 200 units (on a scale of 8160), with the caveat that thepredicted error would be lower when the true oxygen deficit is lower, and higher when the true oxygen deficit is higher.Figure 2: The mean squared error (in arbitrary units) asa function of training epoch for TNF α . The training ofthis cytokine prediction network was representative of allsimulated cytokine prediction networks.Cytokine trajectories present similar stochastic propertiesas the oxygen deficit: when levels are high, the plausi-ble range of cytokine expression for the subsequent timestep is larger than when levels are low. We present themean squared error (in arbitrary units) as a function oftraining epoch for TNF α , which is representative of thefull cytokine set in Figure 2. Once again, the networkquickly converges to a low and constant level of error.We note that the total error quickly and significantly in-creases as we extend the time-prediction horizon past 100time-steps. This distinguishes this methodology from thatof ML-augmented surrogate modeling [32] because wedo not claim the ability to accurately represent the entirecourse of a sepsis disease trajectory (up to 90 days in ourcomputational model) using neural-network approxima-tions.The use of the dropout layer allows for the simple creationof an ensemble of predictive networks by stochasticallyvarying the specific node(s) in the layer that are droppedout. Using this, we have visualized the probability cloudfor future health trajectories generated using the MLPnetwork (shaded in red), future health trajectories generated using the LSTM network (shaded in blue) and plottedthe true trajectory (red line) in Figure 3. This figure visualizes a single prediction iteration (predict future cytokinetrajectory, regress state of health) for the above-described workflow. As new data is fed into the model about the truetrajectory of the system, the forecast cloud is updated. The actual health trajectory typically lies in the center of theprobability cloud, which is a clear benefit of the ensemble approach. In Figure 4, we display the probability cone for theentire simulation run, starting at the 240th time step, and then updating the trajectory cone on every subsequent timestep.Figure 3: The future health-trajectory probability cloudpredicted using the MLP network is shaded in red; thefuture health-trajectory probability cloud predicted usingthe LSTM network is shaded in blue; the true trajectory isplotted in red. In Figure 5, Panel A, we contrast predictions that consid-ered the full time evolution of the system when trainingthe neural network model, shaded in red, with predic-tions that only used training data collected after the 240thtime step, representing approximately 1 day. The networkthat only utilizes data collected more than 24 hrs post-injury performs substantially better. This is primarilydue to the massive amount of stochasticity introducedat the time of injury; the degree of this stochasticity issignificantly larger in magnitude than later in the simu-lation, as discussed below. In Panel B, we display thesame oxygen-deficit probability cone as in Panel A, how-ever also reseed the simulation’s random number gen-erator at this time step to generate 100 stochastic repli-cates of the time evolution of that specific instantiationof the IIRABM. We see that the predicted probabilitycone has a greater spread than the actual probability cone,however we note that the MLP predictor is constantlyupdating its trajectory predictions: the set of observa-tions { t a − , t a − , t a − , t a − , t a − } , where t a − has the super-script, ‘a’, representing an actual observation, and thesubscript, ‘-5’ to denote that the time point is 5 pointsprior to the starting reference point, is used to predict t p , where the superscript, ‘p’, indicates a predicted ob-servation. Eventually, the set of points used to generatethe prediction will consist entirely of previously predicted points, allowing for the compounding of any errors. In5rtificial neural networks for disease trajectory prediction in the context of sepsis A P
REPRINT
Panel C, we show the same probability cone as in Panel A, however this time, we have reseeded the simulationevery 100 time steps at t=1100 to t=1800, for 100 stochastic replicates each. This is a more direct comparisonsince the MLP predictor effectively reseeds itself every time step. We observe that the actual probability cone issignificantly wider than in Panel B, but still not as wide as the predicted cone. This is discussed in detail below.
A P
REPRINT
Figure 5: In Panel A, we contrast the future health-trajectory probability cones with networks that used data collectedin the first 24 hours (shaded in red) and networks that excluded the first-day data (shaded in blue); we note that the bluearea appears purple as it is entirely contained within the red area. In Panel B, we present 100 stochastic replicates ofthe actual simulation health-trajectory, reseeded at the time of the first prediction (red) compared with the predictedprobability cone trajectories (blue). In Panel C, we generated the simulation trajectory cone through reseeding thesimulation’s random number generator at the upper and lower boundaries of the trajectory cone every 100 time stepsfrom t=1100 to t=1800.models [35], these do not directly address complex agent-based models, such as the IIRABM, which are explicitly usedbecause they have no equivalent equation-based formulation. The key aspect of the current work is that it represents afirst-step of a development workflow that integrates mechanism-based simulation with machine learning in order totrain predictive ANNs that can inform what sort of sensing technology and capabilities need to be developed in the realworld. Rather that sporadically connected (at best) suggestions of which mediators to target, and at what time interval,the implementation of workflows using model-driven investigation will bring biomedicine in line with nearly everyother technological and scientific field.
This work was supported by National Institutes of Health grant 1RO1GM115839-01. Additionally, this research usedhigh performance computing resources provided by the Vermont Advanced Computing Core (VACC).
References [1] Wood KA, Angus DC. Pharmacoeconomic implications of new therapies in sepsis. Pharmacoeconomics.2004;22(14):895–906. doi:10.2165/00019053-200422140-00001.[2] Cockrell C, An G. Sepsis reconsidered: Identifying novel metrics for behavioral landscape characterizationwith a high-performance computing implementation of an agent-based model. J Theor Biol. 2017;430:157–168.doi:10.1016/j.jtbi.2017.07.016.[3] Cockrell C, An G. Genetic Algorithms for model refinement and rule discovery in a high-dimensional agent-basedmodel of inflammation. bioRxiv. 2019; p. 790394.[4] Cockrell RC, An G. Examining the controllability of sepsis using genetic algorithms on an agent-based model ofsystemic inflammation. PLoS Comput Biol. 2018;14(2):e1005876. doi:10.1371/journal.pcbi.1005876.[5] Angus DC. The search for effective therapy for sepsis: back to the drawing board? JAMA. 2011;306(23):2614–2615. doi:306/23/2614 [pii];10.1001/jama.2011.1853 [doi].[6] Buchman TG, Billiar TR, Elster E, Kirk AD, Rimawi RH, Vodovotz Y, et al. Precision medicine for critical illnessand injury. Crit Care Med. 2016;44(9):1635–1638. doi:10.1097/ccm.0000000000002028.[7] Vincent J. The SOFA (Sepsis-related Organ Failure Assessment) score to described organ dysfunction/failure.Springer; 1996.[8] Gibot S, Béné MC, Noel R, Massin F, Guy J, Cravoisy A, et al. Combination biomarkers to diagnose sepsis in thecritically ill patient. American journal of respiratory and critical care medicine. 2012;186(1):65–71.[9] Samraj RS, Zingarelli B, Wong HR. Role of biomarkers in sepsis care. Shock (Augusta, Ga). 2013;40(5):358.[10] Riedel S. Procalcitonin and the role of biomarkers in the diagnosis and management of sepsis. Diagnosticmicrobiology and infectious disease. 2012;73(3):221–227.7rtificial neural networks for disease trajectory prediction in the context of sepsis
A P