Marc Paladini

Reassessing Google Flu Trends Data for Detection of Seasonal and Pandemic Influenza: A Comparative Epidemiological Study at Three Geographic Scales

The goal of influenza-like illness (ILI) surveillance is to determine the timing, location and magnitude of outbreaks by monitoring the frequency and progression of clinical case incidence. Advances in computational and information technology have allowed for automated collection of higher volumes of electronic data and more timely analyses than previously possible. Novel surveillance systems, including those based on internet search query data like Google Flu Trends (GFT), are being used as surrogates for clinically-based reporting of influenza-like-illness (ILI). We investigated the reliability of GFT during the last decade (2003 to 2013), and compared weekly public health surveillance with search query data to characterize the timing and intensity of seasonal and pandemic influenza at the national (United States), regional (Mid-Atlantic) and local (New York City) levels. We identified substantial flaws in the original and updated GFT models at all three geographic scales, including completely missing the first wave of the 2009 influenza A/H1N1 pandemic, and greatly overestimating the intensity of the A/H3N2 epidemic during the 2012/2013 season. These results were obtained for both the original (2008) and the updated (2009) GFT algorithms. The performance of both models was problematic, perhaps because of changes in internet search behavior and differences in the seasonality, geographical heterogeneity and age-distribution of the epidemics between the periods of GFT model-fitting and prospective use. We conclude that GFT data may not provide reliable surveillance for seasonal or pandemic influenza and should be interpreted with caution until the algorithm can be improved and evaluated. Current internet search query data are no substitute for timely local clinical and laboratory surveillance, or national surveillance based on local data collection. New generation surveillance systems such as GFT should incorporate the use of near-real time electronic health data and computational methods for continued model-fitting and ongoing evaluation and improvement.

Monitoring the Impact of Influenza by Age: Emergency Department Fever and Respiratory Complaint Surveillance in New York City

Background The importance of understanding age when estimating the impact of influenza on hospitalizations and deaths has been well described, yet existing surveillance systems have not made adequate use of age-specific data. Monitoring influenza-related morbidity using electronic health data may provide timely and detailed insight into the age-specific course, impact and epidemiology of seasonal drift and reassortment epidemic viruses. The purpose of this study was to evaluate the use of emergency department (ED) chief complaint data for measuring influenza-attributable morbidity by age and by predominant circulating virus. Methods and Findings We analyzed electronically reported ED fever and respiratory chief complaint and viral surveillance data in New York City (NYC) during the 2001–2002 through 2005–2006 influenza seasons, and inferred dominant circulating viruses from national surveillance reports. We estimated influenza-attributable impact as observed visits in excess of a model-predicted baseline during influenza periods, and epidemic timing by threshold and cross correlation. We found excess fever and respiratory ED visits occurred predominantly among school-aged children (8.5 excess ED visits per 1,000 children aged 5–17 y) with little or no impact on adults during the early-2002 B/Victoria-lineage epidemic; increased fever and respiratory ED visits among children younger than 5 y during respiratory syncytial virus-predominant periods preceding epidemic influenza; and excess ED visits across all ages during the 2003–2004 (9.2 excess visits per 1,000 population) and 2004–2005 (5.2 excess visits per 1,000 population) A/H3N2 Fujian-lineage epidemics, with the relative impact shifted within and between seasons from younger to older ages. During each influenza epidemic period in the study, ED visits were increased among school-aged children, and each epidemic peaked among school-aged children before other impacted age groups. Conclusions Influenza-related morbidity in NYC was highly age- and strain-specific. The impact of reemerging B/Victoria-lineage influenza was focused primarily on school-aged children born since the virus was last widespread in the US, while epidemic A/Fujian-lineage influenza affected all age groups, consistent with a novel antigenic variant. The correspondence between predominant circulating viruses and excess ED visits, hospitalizations, and deaths shows that excess fever and respiratory ED visits provide a reliable surrogate measure of incident influenza-attributable morbidity. The highly age-specific impact of influenza by subtype and strain suggests that greater age detail be incorporated into ongoing surveillance. Influenza morbidity surveillance using electronic data currently available in many jurisdictions can provide timely and representative information about the age-specific epidemiology of circulating influenza viruses.

Effectiveness of Mosquito Traps in Measuring Species Abundance and Composition

Abstract Mosquito species abundance and composition estimates provided by trapping devices are commonly used to guide control efforts, but knowledge of trap biases is necessary for accurately interpreting results. We tested the hypothesis that commercially available traps (Mosquito Magnet–Pro, the Mosquito Magnet–X) would be significant improvements over the CDC Miniature Light Trap with respect to abundance, species diversity, and measures of recruitment in a wooded area of the Bronx Zoo in New York City, NY. The Mosquito Magnet–Pro collected significantly more mosquitoes (n = 1,117; mean per night, 124 ± 28.3) than the CDC Miniature Light Trap (n = 167; mean per night, 19 ± 5.5). The Simpson’s diversity index was greatest for the Mosquito Magnet–Pro. A CDC light trap from a simultaneous surveillance project was located 15 m away and used as a control trap to test for significant differences in mosquito counts on nights with or without the experimental traps. There were no significant differences between nights, indicating the test traps did not recruit beyond 15 m. The traps differed significantly in abundance, but they had similarly limited sampling areas. Measured differences in abundance were independent of differences in diversity. This study highlights how differences between traps might affect species abundance and composition estimates.

Developing syndrome definitions based on consensus and current use.

OBJECTIVE Standardized surveillance syndromes do not exist but would facilitate sharing data among surveillance systems and comparing the accuracy of existing systems. The objective of this study was to create reference syndrome definitions from a consensus of investigators who currently have or are building syndromic surveillance systems. DESIGN Clinical condition-syndrome pairs were catalogued for 10 surveillance systems across the United States and the representatives of these systems were brought together for a workshop to discuss consensus syndrome definitions. RESULTS Consensus syndrome definitions were generated for the four syndromes monitored by the majority of the 10 participating surveillance systems: Respiratory, gastrointestinal, constitutional, and influenza-like illness (ILI). An important element in coming to consensus quickly was the development of a sensitive and specific definition for respiratory and gastrointestinal syndromes. After the workshop, the definitions were refined and supplemented with keywords and regular expressions, the keywords were mapped to standard vocabularies, and a web ontology language (OWL) ontology was created. LIMITATIONS The consensus definitions have not yet been validated through implementation. CONCLUSION The consensus definitions provide an explicit description of the current state-of-the-art syndromes used in automated surveillance, which can subsequently be systematically evaluated against real data to improve the definitions. The method for creating consensus definitions could be applied to other domains that have diverse existing definitions.

Applying a New Model for Sharing Population Health Data to National Syndromic Influenza Surveillance: DiSTRIBuTE Project Proof of Concept, 2006 to 2009.

The Distributed Surveillance Taskforce for Real-time Influenza Burden Tracking and Evaluation (DiSTRIBuTE) project began as a pilot effort initiated by the International Society for Disease Surveillance (ISDS) in autumn 2006 to create a collaborative electronic emergency department (ED) syndromic influenza-like illness (ILI) surveillance network based on existing state and local systems and expertise. DiSTRIBuTE brought together health departments that were interested in: 1) sharing aggregate level data; 2) maintaining jurisdictional control; 3) minimizing barriers to participation; and 4) leveraging the flexibility of local systems to create a dynamic and collaborative surveillance network. This approach was in contrast to the prevailing paradigm for surveillance where record level information was collected, stored and analyzed centrally. The DiSTRIBuTE project was created with a distributed design, where individual level data remained local and only summarized, stratified counts were reported centrally, thus minimizing privacy risks. The project was responsive to federal mandates to improve integration of federal, state, and local biosurveillance capabilities. During the proof of concept phase, 2006 to 2009, ten jurisdictions from across North America sent ISDS on a daily to weekly basis year-round, aggregated data by day, stratified by local ILI syndrome, age-group and region. During this period, data from participating U.S. state or local health departments captured over 13% of all ED visits nationwide. The initiative focused on state and local health department trust, expertise, and control. Morbidity trends observed in DiSTRIBuTE were highly correlated with other influenza surveillance measures. With the emergence of novel A/H1N1 influenza in the spring of 2009, the project was used to support information sharing and ad hoc querying at the state and local level. In the fall of 2009, through a broadly collaborative effort, the project was expanded to enhance electronic ED surveillance nationwide.

Child Care-associated Outbreak Of escherichia Coli O157:h7 And Hemolytic Uremic Syndrome

We present an outbreak of E. coli O157:H7 diarrhea in an urban child care center. Eleven of 45 attendees with diarrhea had positive tests (stool culture or shiga-like toxin assay) for E. coli O157:H7. Two of these 11 (18%) progressed to hemolytic uremic syndrome. Diarrheal illness in child care centers should be considered a public health risk. Staff education, hand washing, and cohorting or exclusion of attendees with diarrhea must be performed to help control infectious outbreaks.

Syndromic Surveillance during Pandemic (H1N1) 2009 Outbreak, New York, New York, USA

We compared emergency department and ambulatory care syndromic surveillance systems during the pandemic (H1N1) 2009 outbreak in New York City. Emergency departments likely experienced increases in influenza-like-illness significantly earlier than ambulatory care facilities because more patients sought care at emergency departments, differences in case definitions existed, or a combination thereof.

Evaluating the New York City Emergency Department Syndromic Surveillance for Monitoring Influenza Activity during the 2009-10 Influenza Season.

Objective: To use laboratory data to assess the specificity of syndromes used by the New York City emergency department (ED) syndromic surveillance system to monitor influenza activity. Design: For the period from October 1, 2009 through March 31, 2010, we examined the correlation between citywide ED syndrome assignment and laboratory-confirmed influenza and respiratory syncytial virus (RSV). In addition, ED syndromic data from five select NYC hospitals were matched at the patient and visit level to corresponding laboratory reports of influenza and RSV. The matched dataset was used to evaluate syndrome assignment by disease and to calculate the sensitivity and specificity of the influenza-like illness (ILI) syndrome. Results: Citywide ED visits for ILI correlated well with influenza laboratory diagnoses (R=0.92). From October 1, 2009, through March 31, 2010, there were 264,532 ED visits at the five select hospitals, from which the NYC Department of Health and Mental Hygiene (DOHMH) received confirmatory laboratory reports of 655 unique cases of influenza and 1348 cases of RSV. The ED visit of most (56%) influenza cases had been categorized in the fever/flu syndrome; only 15% were labeled ILI. Compared to other influenza-related syndromes, ILI had the lowest sensitivity (15%) but the highest specificity (90%) for laboratory-confirmed influenza. Sensitivity and specificity varied by age group and influenza activity level. Conclusions: The ILI syndrome in the NYC ED syndromic surveillance system served as a specific but not sensitive indicator for influenza during the 2009-2010 influenza season. Despite its limited sensitivity, the ILI syndrome can be more informative for tracking influenza trends than the fever/flu or respiratory syndromes because it is less likely to capture cases of other respiratory viruses. However, ED ILI among specific age groups should be interpreted alongside laboratory surveillance data. ILI remains a valuable tool for monitoring influenza activity and trends as it facilitates comparisons nationally and across jurisdictions and is easily communicated to the public.

Description of a School Nurse Visit Syndromic Surveillance System and Comparison to Emergency Department Visits, New York City

OBJECTIVES We compared school nurse visit syndromic surveillance system data to emergency department (ED) visit data for monitoring illness in New York City schoolchildren. METHODS School nurse visit data recorded in an electronic health record system are used to conduct daily surveillance of influenza-like illness, fever-flu, allergy, asthma, diarrhea, and vomiting syndromes. We calculated correlation coefficients to compare the percentage of syndrome visits to the school nurse and ED for children aged 5 to 14 years, from September 2006 to June 2011. RESULTS Trends in influenza-like illness correlated significantly (correlation coefficient = 0.89; P < .001) and 72% of school signals occurred on days that ED signaled. Trends in allergy (correlation coefficient = 0.73; P < .001) and asthma (correlation coefficient = 0.56; P < .001) also correlated and school signals overlapped with ED signals on 95% and 51% of days, respectively. Substantial daily variation in diarrhea and vomiting visits limited our ability to make comparisons. CONCLUSIONS Compared with ED syndromic surveillance, the school nurse system identified similar trends in influenza-like illness, allergy, and asthma syndromes. Public health practitioners without school-based surveillance may be able to use age-specific analyses of ED syndromic surveillance data to monitor illness in schoolchildren.

Evaluating and implementing temporal, spatial, and spatio-temporal methods for outbreak detection in a local syndromic surveillance system:

The New York City Department of Health and Mental Hygiene has operated an emergency department syndromic surveillance system since 2001, using temporal and spatial scan statistics run on a daily basis for cluster detection. Since the system was originally implemented, a number of new methods have been proposed for use in cluster detection. We evaluated six temporal and four spatial/spatio-temporal detection methods using syndromic surveillance data spiked with simulated injections. The algorithms were compared on several metrics, including sensitivity, specificity, positive predictive value, coherence, and timeliness. We also evaluated each method’s implementation, programming time, run time, and the ease of use. Among the temporal methods, at a set specificity of 95%, a Holt-Winters exponential smoother performed the best, detecting 19% of the simulated injects across all shapes and sizes, followed by an autoregressive moving average model (16%), a generalized linear model (15%), a modified version of the Early Aberration Reporting System’s C2 algorithm (13%), a temporal scan statistic (11%), and a cumulative sum control chart (<2%). Of the spatial/spatio-temporal methods we tested, a spatial scan statistic detected 3% of all injects, a Bayes regression found 2%, and a generalized linear mixed model and a space-time permutation scan statistic detected none at a specificity of 95%. Positive predictive value was low (<7%) for all methods. Overall, the detection methods we tested did not perform well in identifying the temporal and spatial clusters of cases in the inject dataset. The spatial scan statistic, our current method for spatial cluster detection, performed slightly better than the other tested methods across different inject magnitudes and types. Furthermore, we found the scan statistics, as applied in the SaTScan software package, to be the easiest to program and implement for daily data analysis.