Masoumeh T. Izadi

An infrastructure for real-time population health assessment and monitoring

The fragmented nature of population health information is a barrier to public health practice. Despite repeated demands by policymakers, administrators, and practitioners to develop information systems that provide a coherent view of population health status, there has been limited progress toward developing such an infrastructure. We are creating an informatics platform for describing and monitoring the health status of a defined population by integrating multiple clinical and administrative data sources. This infrastructure, which involves a population health record, is designed to enable development of detailed portraits of population health, facilitate monitoring of population health indicators, enable evaluation of interventions, and provide clinicians and patients with population context to assist diagnostic and therapeutic decision-making. In addition to supporting public health professionals, clinicians, and the public, we are designing the infrastructure to provide a platform for public health informatics research. This early report presents the requirements and architecture for the infrastructure and describes the initial implementation of the population health record, focusing on indicators of chronic diseases related to obesity.

Point-based planning for predictive state representations

Predictive state representations (PSRs) have been proposed recently as an alternative representation for environments with partial observability. The representation is rooted in actions and observations, so it holds the promise of being easier to learn than Partially Observable Markov Decision Processes (POMDPs). However, comparatively little work has explored planning algorithms using PSRs. Exact methods developed to date are no faster than existing exact planning approaches for POMDPs, and only memory-based PSRs have been shown so far to have an advantage in terms of planning speed. In this paper, we present an algorithm for approximate planning in PSRs, based on an approach similar to point-based value iteration in POMDPs. The point-based approach turns out to be a natural match for the PSR state representation. We present empirical results showing that our approach is either comparable or better than POMDP point-based planning.

Belief selection in point-based planning algorithms for POMDPs

Current point-based planning algorithms for solving partially observable Markov decision processes (POMDPs) have demonstrated that a good approximation of the value function can be derived by interpolation from the values of a specially selected set of points. The performance of these algorithms can be improved by eliminating unnecessary backups or concentrating on more important points in the belief simplex. We study three methods designed to improve point-based value iteration algorithms. The first two methods are based on reachability analysis on the POMDP belief space. This approach relies on prioritizing the beliefs based on how they are reached from the given initial belief state. The third approach is motivated by the observation that beliefs which are the most overestimated or underestimated have greater influence on the precision of value function than other beliefs. We present an empirical evaluation illustrating how the performance of point-based value iteration (Pineau et al., 2003) varies with these approaches.

Sensitivity Analysis of POMDP Value Functions

In sequential decision making under uncertainty, as in many other modeling endeavors, researchers observe a dynamical system and collect data measuring its behavior over time. These data are often used to build models that explain relationships between the measured variables, and are eventually used for planning and control purposes. However, these measurements cannot always be exact, systems can change over time, and discovering these facts or fixing these problems is not always feasible. Therefore it is important to formally describe the degree to which the model can tolerate noise, in order to keep near optimal behavior. The problem of finding tolerance bounds has been the focus of many studies for Markov Decision Processes (MDPs) due to their usefulness in practical applications. In this paper, we consider Partially Observable MDPs (POMDPs), which is a more realistic extension of MDPs with a wider scope of applications. We address two types of perturbations in POMDP model parameters, namely additive and multiplicative, and provide theoretical bounds for the impact of these changes in the value function. Experimental results are provided to illustrate our POMDP perturbation analysis in practice.

Using rewards for belief state updates in partially observable markov decision processes

Partially Observable Markov Decision Processes (POMDP) provide a standard framework for sequential decision making in stochastic environments. In this setting, an agent takes actions and receives observations and rewards from the environment. Many POMDP solution methods are based on computing a belief state, which is a probability distribution over possible states in which the agent could be. The action choice of the agent is then based on the belief state. The belief state is computed based on a model of the environment, and the history of actions and observations seen by the agent. However, reward information is not taken into account in updating the belief state. In this paper, we argue that rewards can carry useful information that can help disambiguate the hidden state. We present a method for updating the belief state which takes rewards into account. We present experiments with exact and approximate planning methods on several standard POMDP domains, using this belief update method, and show that it can provide advantages, both in terms of speed and in terms of the quality of the solution obtained.

Quantifying the determinants of outbreak detection performance through simulation and machine learning

OBJECTIVE To develop a probabilistic model for discovering and quantifying determinants of outbreak detection and to use the model to predict detection performance for new outbreaks. MATERIALS AND METHODS We used an existing software platform to simulate waterborne disease outbreaks of varying duration and magnitude. The simulated data were overlaid on real data from visits to emergency department in Montreal for gastroenteritis. We analyzed the combined data using biosurveillance algorithms, varying their parameters over a wide range. We then applied structure and parameter learning algorithms to the resulting data set to build a Bayesian network model for predicting detection performance as a function of outbreak characteristics and surveillance system parameters. We evaluated the predictions of this model through 5-fold cross-validation. RESULTS The model predicted performance metrics of commonly used outbreak detection methods with an accuracy greater than 0.80. The model also quantified the influence of different outbreak characteristics and parameters of biosurveillance algorithms on detection performance in practically relevant surveillance scenarios. In addition to identifying characteristics expected a priori to have a strong influence on detection performance, such as the alerting threshold and the peak size of the outbreak, the model suggested an important role for other algorithm features, such as adjustment for weekly patterns. CONCLUSION We developed a model that accurately predicts how characteristics of disease outbreaks and detection methods will influence on detection. This model can be used to compare the performance of detection methods under different surveillance scenarios, to gain insight into which characteristics of outbreaks and biosurveillance algorithms drive detection performance, and to guide the configuration of surveillance systems.

PHIO: a knowledge base for interpretation and calculation of public health indicators.

Existing population health indicators tend to be out-of-date, not fully available at local levels of geography, and not developed in a coherent/consistent manner, which hinders their use in public health. The PopHR platform aims to deliver an electronic repository that contains multiple aggregated clinical, administrative, and environmental data sources to provide a coherent view of the health status of populations in the province of Quebec, Canada. This platform is designed to provide representative information in near-real time with high geographical resolution, thereby assisting public health professionals, analysts, clinicians and the public in decision-making. This paper presents our ongoing efforts to develop an integrated population health indicator ontology (PHIO) that captures the knowledge required for calculation and interpretation of health indicators within a PopHR semantic framework.

Optimizing the response to surveillance alerts in automated surveillance systems

Although much research effort has been directed toward refining algorithms for disease outbreak alerting, considerably less attention has been given to the response to alerts generated from statistical detection algorithms. Given the inherent inaccuracy in alerting, it is imperative to develop methods that help public health personnel identify optimal policies in response to alerts. This study evaluates the application of dynamic decision making models to the problem of responding to outbreak detection methods, using anthrax surveillance as an example. Adaptive optimization through approximate dynamic programming is used to generate a policy for decision making following outbreak detection. We investigate the degree to which the model can tolerate noise theoretically, in order to keep near optimal behavior. We also evaluate the policy from our model empirically and compare it with current approaches in routine public health practice for investigating alerts. Timeliness of outbreak confirmation and total costs associated with the decisions made are used as performance measures. Using our approach, on average, 80 per cent of outbreaks were confirmed prior to the fifth day of post-attack with considerably less cost compared to response strategies currently in use. Experimental results are also provided to illustrate the robustness of the adaptive optimization approach and to show the realization of the derived error bounds in practice.

Prediction-Directed Compression of POMDPs

High dimensionality of belief space in partially observable Markov decision processes (POMDPs) is one of the major causes that severely restricts the applicability of this model. Previous studies have demonstrated that the dimensionality of a POMDP can eventually be reduced by transforming it into an equivalent predictive state representation (PSR). In this paper, we address the problem of finding an approximate and compact PSR model corresponding to a given POMDP model. We formulate this problem in an optimization framework. Our algorithm tries to minimize the potential error that missing some core tests may cause. We also present an empirical evaluation on benchmark problems, illustrating the performance of this approach.

Using Dynamic Bayesian Networks for Incorporating Nontraditional Data Sources in Public Health Surveillance

The estimation of disease prevalence based on public health surveillance data requires the accurate identification of cases from limited information (e.g., diagnostic codes). These data sources typically consist of routinely collected records of population healthcare utilization, such as administrative and clinical data, that specifies diagnostic codes or terms for each encounter. These data sources include, for example, emergency department visits, pharmaceutical (drug) dispensations, and laboratory test orders. The case definitions depend on the data source and are typically based on the presence of diagnostic codes or key words in a prespecified time frame. Each data source will result in a certain degree of misclassification bias when estimating prevalence. Inaccuracies can occur at each stage from the time the disease process is initiated to the stage at which diagnostic codes are entered into the database. Indeed, when relying on these data sources, asymptomatic cases will be missed, as well as those not seeking health care. Even patients that seek care may be inaccurately diagnosed or the diagnostic code that is entered in the system may not represent the diagnosis or may not be a code or key word used in the definition. In addition to misclassification bias, these data sources are not usually available in a timely manner. Timeliness is an important factor for prevalence estimation in certain contexts such as the prevalence of infectious diseases during an epidemic. For instance, in an influenza pandemic, such estimates must be obtained within days. In recent years, several nonclinical and nontraditional data sources have been introduced to public health surveillance with the potential to provide more timely signals of changing prevalence trends. Ideally, combining the new and traditional data sources, there is greater potential to overcome bias and provide more timely signals. However, building a construct capable of incorporating data from these various sources in a coherent manner is not trivial. In this research, we consider the case of the 2009–2010 H1N1 pandemic as the context of interest and we use media reports of deaths from H1N1 on the web as a nontraditional data source. We propose to use dynamic Bayesian networks from the class of probabilistic graphical models in order to combine this new data source with traditional ones through exploration of the possible probabilistic relationships between these data streams. This is an initial step toward building a framework that can potentially support aggregation of heterogeneous data for a real-time estimation of disease prevalence. Our preliminary results show that the proposed model can be used in accurate prediction of short-term future counts of the data sources. This is particularly useful in timely prediction of epidemic changes over a defined population.