Ben Y. Reis

HealthMap: Global Infectious Disease Monitoring through Automated Classification and Visualization of Internet Media Reports

Abstract Objective Unstructured electronic information sources, such as news reports, are proving to be valuable inputs for public health surveillance. However, staying abreast of current disease outbreaks requires scouring a continually growing number of disparate news sources and alert services, resulting in information overload. Our objective is to address this challenge through the HealthMap.org Web application, an automated system for querying, filtering, integrating and visualizing unstructured reports on disease outbreaks. Design This report describes the design principles, software architecture and implementation of HealthMap and discusses key challenges and future plans. Measurements We describe the process by which HealthMap collects and integrates outbreak data from a variety of sources, including news media (e.g., Google News), expert-curated accounts (e.g., ProMED Mail), and validated official alerts. Through the use of text processing algorithms, the system classifies alerts by location and disease and then overlays them on an interactive geographic map. We measure the accuracy of the classification algorithms based on the level of human curation necessary to correct misclassifications, and examine geographic coverage. Results As part of the evaluation of the system, we analyzed 778 reports with HealthMap, representing 87 disease categories and 89 countries. The automated classifier performed with 84% accuracy, demonstrating significant usefulness in managing the large volume of information processed by the system. Accuracy for ProMED alerts is 91% compared to Google News reports at 81%, as ProMED messages follow a more regular structure. Conclusion HealthMap is a useful free and open resource employing text-processing algorithms to identify important disease outbreak information through a user-friendly interface.

Stability of the Memory of Eye Position in a Recurrent Network of Conductance-Based Model Neurons

Studies of the neural correlates of short-term memory in a wide variety of brain areas have found that transient inputs can cause persistent changes in rates of action potential firing, through a mechanism that remains unknown. In a premotor area that is responsible for holding the eyes still during fixation, persistent neural firing encodes the angular position of the eyes in a characteristic manner: below a threshold position the neuron is silent, and above it the firing rate is linearly related to position. Both the threshold and linear slope vary from neuron to neuron. We have reproduced this behavior in a biophysically plausible network model. Persistence depends on precise tuning of the strength of synaptic feedback, and a relatively long synaptic time constant improves the robustness to mistuning.

Surveillance Sans Frontières: Internet-Based Emerging Infectious Disease Intelligence and the HealthMap Project

John Brownstein and colleagues discuss HealthMap, an automated real-time system that monitors and disseminates online information about emerging infectious diseases.

Time series modeling for syndromic surveillance

BackgroundEmergency department (ED) based syndromic surveillance systems identify abnormally high visit rates that may be an early signal of a bioterrorist attack. For example, an anthrax outbreak might first be detectable as an unusual increase in the number of patients reporting to the ED with respiratory symptoms. Reliably identifying these abnormal visit patterns requires a good understanding of the normal patterns of healthcare usage. Unfortunately, systematic methods for determining the expected number of (ED) visits on a particular day have not yet been well established. We present here a generalized methodology for developing models of expected ED visit rates.MethodsUsing time-series methods, we developed robust models of ED utilization for the purpose of defining expected visit rates. The models were based on nearly a decade of historical data at a major metropolitan academic, tertiary care pediatric emergency department. The historical data were fit using trimmed-mean seasonal models, and additional models were fit with autoregressive integrated moving average (ARIMA) residuals to account for recent trends in the data. The detection capabilities of the model were tested with simulated outbreaks.ResultsModels were built both for overall visits and for respiratory-related visits, classified according to the chief complaint recorded at the beginning of each visit. The mean absolute percentage error of the ARIMA models was 9.37% for overall visits and 27.54% for respiratory visits. A simple detection system based on the ARIMA model of overall visits was able to detect 7-day-long simulated outbreaks of 30 visits per day with 100% sensitivity and 97% specificity. Sensitivity decreased with outbreak size, dropping to 94% for outbreaks of 20 visits per day, and 57% for 10 visits per day, all while maintaining a 97% benchmark specificity.ConclusionsTime series methods applied to historical ED utilization data are an important tool for syndromic surveillance. Accurate forecasting of emergency department total utilization as well as the rates of particular syndromes is possible. The multiple models in the system account for both long-term and recent trends, and an integrated alarms strategy combining these two perspectives may provide a more complete picture to public health authorities. The systematic methodology described here can be generalized to other healthcare settings to develop automated surveillance systems capable of detecting anomalies in disease patterns and healthcare utilization.

Using temporal context to improve biosurveillance

Current efforts to detect covert bioterrorist attacks from increases in hospital visit rates are plagued by the unpredictable nature of these rates. Although many current systems evaluate hospital visit data 1 day at a time, we investigate evaluating multiple days at once to lessen the effects of this unpredictability and to improve both the timeliness and sensitivity of detection. To test this approach, we introduce simulated disease outbreaks of varying shapes, magnitudes, and durations into 10 years of historical daily visit data from a major tertiary-care metropolitan teaching hospital. We then investigate the effectiveness of using multiday temporal filters for detecting these simulated outbreaks within the noisy environment of the historical visit data. Our results show that compared with the standard 1-day approach, the multiday detection approach significantly increases detection sensitivity and decreases latency while maintaining a high specificity. We conclude that current biosurveillance systems should incorporate a wider temporal context to improve their effectiveness. Furthermore, for increased robustness and performance, hybrid systems should be developed to capitalize on the complementary strengths of different types of temporal filters.

The Autapse: A Simple Illustration of Short-Term Analog Memory Storage by Tuned Synaptic Feedback

According to a popular hypothesis, short-term memories are stored as persistent neural activity maintained by synaptic feedback loops. This hypothesis has been formulated mathematically in a number of recurrent network models. Here we study an abstraction of these models, a single neuron with a synapse onto itself, or autapse. This abstraction cannot simulate the way in which persistent activity patterns are distributed over neural populations in the brain. However, with proper tuning of parameters, it does reproduce the continuously graded, or analog, nature of many examples of persistent activity. The conditions for tuning are derived for the dynamics of a conductance-based model neuron with a slow excitatory autapse. The derivation uses the method of averaging to approximate the spiking model with a nonspiking, reduced model. Short-term analog memory storage is possible if the reduced model is approximately linear and if its feedforward bias and autapse strength are precisely tuned.

Predicting Adverse Drug Events Using Pharmacological Network Models

A network-based method that uses available pharmacosafety data can predict yet-to-be-discovered adverse drug events to help reduce drug-associated morbidity and mortality. The Power of Prediction We’ve all done it: googled a combination of medical terms to describe how we feel after taking a new medication. The result is a seemingly infinite list of Web sites telling us that the nausea is normal, or that the headaches warrant another visit to the doctor. Oftentimes, important adverse effects of drugs are discovered and added to the drug label only years after a drug goes on the market. But what if scientists could know about certain adverse drug effects before they are clinically discovered? Cami and colleagues develop a mathematical approach to predicting such adverse events associated with the drugs we take, in hopes of reducing drug-related morbidity—and mortality. After its release to the market, any given drug undergoes rigorous evaluation to determine associated ADEs (adverse drug effects). This post hoc analysis is usually unable to detect rare or delayed-onset ADEs until enough clinical evidence accumulates–a process that may take years. The method devised by Cami and coauthors does not need to wait for such evidence to accumulate. Instead, it can inform drug safety practitioners early on of likely ADEs that will be detected down the line. The authors first collected a “snapshot” of 809 drugs and their 852 related adverse events that had been documented in 2005. These drug-safety associations were combined with taxonomic and biological data to construct a network that is reminiscent of a web. Cami et al. then used this drug-ADE network to train a logistic regression predictive model—basically creating a formula that would indicate the likelihood of unknown side effects of any drug in the network. The predictive capabilities of the model were prospectively validated using drug-ADE associations newly reported between 2006 and 2010. Such prospective evaluation preserves the chronological order of drug adverse event reporting, making it a realistic method for predicting future ADEs. With their network, the authors were able to predict with high specificity seven of eight drug ADEs identified by pharmacological experts as having emerged after 2005, including the relationship between the anti-diabetes drug rosiglitazone (Avandia) and heart attack. The benefit for patients? With this powerful model in place, certain unknown adverse drug effects may be discovered earlier, helping to prevent drug-related morbidity and mortality through appropriate consumer label warnings. Early and accurate identification of adverse drug events (ADEs) is critically important for public health. We have developed a novel approach for predicting ADEs, called predictive pharmacosafety networks (PPNs). PPNs integrate the network structure formed by known drug-ADE relationships with information on specific drugs and adverse events to predict likely unknown ADEs. Rather than waiting for sufficient post-market evidence to accumulate for a given ADE, this predictive approach relies on leveraging existing, contextual drug safety information, thereby having the potential to identify certain ADEs earlier. We constructed a network representation of drug-ADE associations for 809 drugs and 852 ADEs on the basis of a snapshot of a widely used drug safety database from 2005 and supplemented these data with additional pharmacological information. We trained a logistic regression model to predict unknown drug-ADE associations that were not listed in the 2005 snapshot. We evaluated the model’s performance by comparing these predictions with the new drug-ADE associations that appeared in a 2010 snapshot of the same drug safety database. The proposed model achieved an AUROC (area under the receiver operating characteristic curve) statistic of 0.87, with a sensitivity of 0.42 given a specificity of 0.95. These findings suggest that predictive network methods can be useful for predicting unknown ADEs.

Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population.

Objectives: (1) To determine the value of emergency department chief complaint (CC) and International Classification of Disease diagnostic codes for identifying respiratory illness in a pediatric population and (2) to modify standard respiratory CC and diagnostic code sets to better identify respiratory illness in children. Results: We determined the sensitivity and specificity of CC and diagnostic codes by comparing code groups with a criterion standard. CC and diagnostic codes for 500 pediatric emergency department patients were retrospectively classified as respiratory or nonrespiratory. Respiratory diagnostic codes were further classified as upper or lower respiratory. The criterion standard was a blinded, reviewer-assigned illness category based on history, physical examination, test results, and treatment. We also modified our respiratory code sets to better identify respiratory illness in this population. Methods: Four hundred ninety-six charts met inclusion criteria. By the criterion standard, 87 (18%) patients had upper and 47 (10%) had lower respiratory illness. The specificity of CC and diagnostic codes groups was >0.97 [95% confidence interval (CI) 0.95-0.98]. The code group sensitivities were as follows: CC was 0.47 (95% CI 0.38-0.55), upper respiratory diagnostic was 0.56 (95% CI 0.45-0.67), lower respiratory diagnostic was 0.87 (95% CI 0.74-0.95), and combined CC and/or diagnostic was 0.72 (95% CI 0.63-0.79). Modifying the respiratory code sets to better identify respiratory illness increased sensitivity but decreased specificity. Conclusions: Diagnostic and CC codes have substantial value for emergency department syndromic surveillance. Adapting our respiratory code sets to a pediatric population forced a tradeoff between sensitivity and specificity.

A self-scaling, distributed information architecture for public health, research, and clinical care.

OBJECTIVE This study sought to define a scalable architecture to support the National Health Information Network (NHIN). This architecture must concurrently support a wide range of public health, research, and clinical care activities. STUDY DESIGN The architecture fulfils five desiderata: (1) adopt a distributed approach to data storage to protect privacy, (2) enable strong institutional autonomy to engender participation, (3) provide oversight and transparency to ensure patient trust, (4) allow variable levels of access according to investigator needs and institutional policies, (5) define a self-scaling architecture that encourages voluntary regional collaborations that coalesce to form a nationwide network. RESULTS Our model has been validated by a large-scale, multi-institution study involving seven medical centers for cancer research. It is the basis of one of four open architectures developed under funding from the Office of the National Coordinator of Health Information Technology, fulfilling the biosurveillance use case defined by the American Health Information Community. The model supports broad applicability for regional and national clinical information exchanges. CONCLUSIONS This model shows the feasibility of an architecture wherein the requirements of care providers, investigators, and public health authorities are served by a distributed model that grants autonomy, protects privacy, and promotes participation.

AEGIS: a robust and scalable real-time public health surveillance system

In this report, we describe the Automated Epidemiological Geotemporal Integrated Surveillance system (AEGIS), developed for real-time population health monitoring in the state of Massachusetts. AEGIS provides public health personnel with automated near-real-time situational awareness of utilization patterns at participating healthcare institutions, supporting surveillance of bioterrorism and naturally occurring outbreaks. As real-time public health surveillance systems become integrated into regional and national surveillance initiatives, the challenges of scalability, robustness, and data security become increasingly prominent. A modular and fault tolerant design helps AEGIS achieve scalability and robustness, while a distributed storage model with local autonomy helps to minimize risk of unauthorized disclosure. The report includes a description of the evolution of the design over time in response to the challenges of a regional and national integration environment.