Anya Okhmatovskaia

Validation of population-based disease simulation models: a review of concepts and methods

BackgroundComputer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models.MethodsWe developed a comprehensive framework for validating population-based chronic disease simulation models and used this framework in a review of published model validation guidelines. Based on the review, we formulated a set of recommendations for gathering evidence of model credibility.ResultsEvidence of model credibility derives from examining: 1) the process of model development, 2) the performance of a model, and 3) the quality of decisions based on the model. Many important issues in model validation are insufficiently addressed by current guidelines. These issues include a detailed evaluation of different data sources, graphical representation of models, computer programming, model calibration, between-model comparisons, sensitivity analysis, and predictive validity. The role of external data in model validation depends on the purpose of the model (e.g., decision analysis versus prediction). More research is needed on the methods of comparing the quality of decisions based on different models.ConclusionAs the role of simulation modeling in population health is increasing and models are becoming more complex, there is a need for further improvements in model validation methodology and common standards for evaluating model credibility.

Developing syndrome definitions based on consensus and current use.

OBJECTIVE Standardized surveillance syndromes do not exist but would facilitate sharing data among surveillance systems and comparing the accuracy of existing systems. The objective of this study was to create reference syndrome definitions from a consensus of investigators who currently have or are building syndromic surveillance systems. DESIGN Clinical condition-syndrome pairs were catalogued for 10 surveillance systems across the United States and the representatives of these systems were brought together for a workshop to discuss consensus syndrome definitions. RESULTS Consensus syndrome definitions were generated for the four syndromes monitored by the majority of the 10 participating surveillance systems: Respiratory, gastrointestinal, constitutional, and influenza-like illness (ILI). An important element in coming to consensus quickly was the development of a sensitive and specific definition for respiratory and gastrointestinal syndromes. After the workshop, the definitions were refined and supplemented with keywords and regular expressions, the keywords were mapped to standard vocabularies, and a web ontology language (OWL) ontology was created. LIMITATIONS The consensus definitions have not yet been validated through implementation. CONCLUSION The consensus definitions provide an explicit description of the current state-of-the-art syndromes used in automated surveillance, which can subsequently be systematically evaluated against real data to improve the definitions. The method for creating consensus definitions could be applied to other domains that have diverse existing definitions.

Projections of preventable risks for cardiovascular disease in Canada to 2021: a microsimulation modelling approach

BACKGROUND Reductions in preventable risks associated with cardiovascular disease have contributed to a steady decrease in its incidence over the past 50 years in most developed countries. However, it is unclear whether this trend will continue. Our objective was to examine future risk by projecting trends in preventable risk factors in Canada to 2021. METHODS We created a population-based microsimulation model using national data on births, deaths and migration; socioeconomic data; cardiovascular disease risk factors; and algorithms for changes in these risk factors (based on sociodemographic characteristics and previous cardiovascular disease risk). An initial population of 22.5 million people, representing the Canadian adult population in 2001, had 13 characteristics including the risk factors used in clinical risk prediction. There were 6.1 million potential exposure profiles for each person each year. Outcome measures included annual prevalence of risk factors (smoking, obesity, diabetes, hypertension and lipid levels) and of co-occurring risks. RESULTS From 2003 to 2009, the projected risks of cardiovascular disease based on the microsimulation model closely approximated those based on national surveys. Except for obesity and diabetes, all risk factors were projected to decrease through to 2021. The largest projected decreases were for the prevalence of smoking (from 25.7% in 2001 to 17.7% in 2021) and uncontrolled hypertension (from 16.1% to 10.8%). Between 2015 and 2017, obesity was projected to surpass smoking as the most prevalent risk factor. INTERPRETATION Risks of cardiovascular disease are projected to decrease modestly in Canada, leading to a likely continuing decline in its incidence.

Knowledge-based surveillance for preventing postoperative surgical site infection.

At least one out of every twenty people admitted to a Canadian hospital will acquire an infection. These hospital-acquired infections (HAIs) take a profound individual and system-wide toll, resulting in thousands of deaths and hundreds of millions of dollars in additional expenses each year. Surveillance for HAIs is essential to develop and evaluate prevention and control efforts. In nearly all healthcare institutions, however, surveillance for HAIs is a manual process, requiring highly trained infection control practitioners to consult multiple information systems and paper charts. The amount of effort required for discovery and integration of relevant data from multiple sources limits the current effectiveness of HAIs surveillance. In this research, we apply knowledge modeling and semantic technologies to facilitate the integration of disparate data and enable automatic reasoning with these integrated data to identify events of clinical interest. In this paper, we focus on Surgical Site Infections (SSIs), which account for a relatively large fraction of all hospital acquired infections.

PopHR: a knowledge‐based platform to support integration, analysis, and visualization of population health data

Population health decision makers must consider complex relationships between multiple concepts measured with differential accuracy from heterogeneous data sources. Population health information systems are currently limited in their ability to integrate data and present a coherent portrait of population health. Consequentially, these systems can provide only basic support for decision makers. The Population Health Record (PopHR) is a semantic web application that automates the integration and extraction of massive amounts of heterogeneous data from multiple distributed sources (e.g., administrative data, clinical records, and survey responses) to support the measurement and monitoring of population health and health system performance for a defined population. The design of the PopHR draws on the theories of the determinants of health and evidence‐based public health to harmonize and explicitly link information about a population with evidence about the epidemiology and control of chronic diseases. Organizing information in this manner and linking it explicitly to evidence is expected to improve decision making related to the planning, implementation, and evaluation of population health and health system interventions. In this paper, we describe the PopHR platform and discuss the architecture, design, key modules, and its implementation and use.

Effects of Reductions in Body Mass Index on the Future Osteoarthritis Burden in Canada: A Population‐Based Microsimulation Study

Osteoarthritis (OA) is the most common joint disease and a major cause of disability. Incidence and prevalence of OA are expected to increase due to population aging and increased levels of obesity. The purpose of this study was to project the effect of hypothetical interventions that change the distribution of body mass index (BMI) on OA burden in Canada.

SimPHO: an ontology for simulation modeling of population health

Simulation modeling of population health is being used increasingly for epidemiology research and public health policy-making. However, the impact of population health simulation models is inhibited by their complexity and the lack of established standards to describe these models. To address this issue, we are developing the Ontology for Simulation Modeling of Population Health (SimPHO) - a formal, explicit, computer-readable approach to describing population health simulation models. SimPHO builds on previous work to classify and formally represent knowledge about simulation models, and incorporates the semantics of the epidemiology and public health domains. SimPHO will allow model developers to make explicit their assumptions, to describe their models in a formal, consistent and interoperable manner, and to facilitate model reuse and integration. To illustrate the use of SimPHO, we describe one software application driven by this ontology, an automated visualization tool for generating interactive web-based diagrams of population health simulation models.

PHIO: a knowledge base for interpretation and calculation of public health indicators.

Existing population health indicators tend to be out-of-date, not fully available at local levels of geography, and not developed in a coherent/consistent manner, which hinders their use in public health. The PopHR platform aims to deliver an electronic repository that contains multiple aggregated clinical, administrative, and environmental data sources to provide a coherent view of the health status of populations in the province of Quebec, Canada. This platform is designed to provide representative information in near-real time with high geographical resolution, thereby assisting public health professionals, analysts, clinicians and the public in decision-making. This paper presents our ongoing efforts to develop an integrated population health indicator ontology (PHIO) that captures the knowledge required for calculation and interpretation of health indicators within a PopHR semantic framework.

Addressing the challenge of encoding causal epidemiological knowledge in formal ontologies: a practical perspective.

The paper presents an overview of approaches to encoding uncertain causal knowledge in formal ontologies and demonstrates how these approaches can be used in a semantic-driven application for public health using the Population Health Record (PopHR) platform as an example.

Ophiucus: RDF-based visualization tool for health simulation models.

Simulation modeling of population health is becoming increasingly popular for epidemiology research and public health policy-making. However, the acceptability of population health simulation models is inhibited by their complexity and the lack of established standards to describe these models. To address this issue, we propose Ophiuchus - an RDF (Resource Description Framework: http://www.w3.org/RDF/)-based visualization tool for generating interactive 2D diagrams of population health simulation models, which describe these models in an explicit and formal manner. We present the results of a preliminary system assessment and discuss current limitations of the system.