Abraham D. Flaxman

Neonatal, postneonatal, childhood, and under-5 mortality for 187 countries, 1970–2010: a systematic analysis of progress towards Millennium Development Goal 4

BACKGROUNDnPrevious assessments have highlighted that less than a quarter of countries are on track to achieve Millennium Development Goal 4 (MDG 4), which calls for a two-thirds reduction in mortality in children younger than 5 years between 1990 and 2015. In view of policy initiatives and investments made since 2000, it is important to see if there is acceleration towards the MDG 4 target. We assessed levels and trends in child mortality for 187 countries from 1970 to 2010.nnnMETHODSnWe compiled a database of 16 174 measurements of mortality in children younger than 5 years for 187 countries from 1970 to 2009, by use of data from all available sources, including vital registration systems, summary birth histories in censuses and surveys, and complete birth histories. We used Gaussian process regression to generate estimates of the probability of death between birth and age 5 years. This is the first study that uses Gaussian process regression to estimate child mortality, and this technique has better out-of-sample predictive validity than do previous methods and captures uncertainty caused by sampling and non-sampling error across data types. Neonatal, postneonatal, and childhood mortality was estimated from mortality in children younger than 5 years by use of the 1760 measurements from vital registration systems and complete birth histories that contained specific information about neonatal and postneonatal mortality.nnnFINDINGSnWorldwide mortality in children younger than 5 years has dropped from 11.9 million deaths in 1990 to 7.7 million deaths in 2010, consisting of 3.1 million neonatal deaths, 2.3 million postneonatal deaths, and 2.3 million childhood deaths (deaths in children aged 1-4 years). 33.0% of deaths in children younger than 5 years occur in south Asia and 49.6% occur in sub-Saharan Africa, with less than 1% of deaths occurring in high-income countries. Across 21 regions of the world, rates of neonatal, postneonatal, and childhood mortality are declining. The global decline from 1990 to 2010 is 2.1% per year for neonatal mortality, 2.3% for postneonatal mortality, and 2.2% for childhood mortality. In 13 regions of the world, including all regions in sub-Saharan Africa, there is evidence of accelerating declines from 2000 to 2010 compared with 1990 to 2000. Within sub-Saharan Africa, rates of decline have increased by more than 1% in Angola, Botswana, Cameroon, Congo, Democratic Republic of the Congo, Kenya, Lesotho, Liberia, Rwanda, Senegal, Sierra Leone, Swaziland, and The Gambia.nnnINTERPRETATIONnRobust measurement of mortality in children younger than 5 years shows that accelerating declines are occurring in several low-income countries. These positive developments deserve attention and might need enhanced policy attention and resources.nnnFUNDINGnBill & Melinda Gates Foundation.

Population Health Metrics Research Consortium gold standard verbal autopsy validation study: design, implementation, and development of analysis datasets

BackgroundVerbal autopsy methods are critically important for evaluating the leading causes of death in populations without adequate vital registration systems. With a myriad of analytical and data collection approaches, it is essential to create a high quality validation dataset from different populations to evaluate comparative method performance and make recommendations for future verbal autopsy implementation. This study was undertaken to compile a set of strictly defined gold standard deaths for which verbal autopsies were collected to validate the accuracy of different methods of verbal autopsy cause of death assignment.MethodsData collection was implemented in six sites in four countries: Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India. The Population Health Metrics Research Consortium (PHMRC) developed stringent diagnostic criteria including laboratory, pathology, and medical imaging findings to identify gold standard deaths in health facilities as well as an enhanced verbal autopsy instrument based on World Health Organization (WHO) standards. A cause list was constructed based on the WHO Global Burden of Disease estimates of the leading causes of death, potential to identify unique signs and symptoms, and the likely existence of sufficient medical technology to ascertain gold standard cases. Blinded verbal autopsies were collected on all gold standard deaths.ResultsOver 12,000 verbal autopsies on deaths with gold standard diagnoses were collected (7,836 adults, 2,075 children, 1,629 neonates, and 1,002 stillbirths). Difficulties in finding sufficient cases to meet gold standard criteria as well as problems with misclassification for certain causes meant that the target list of causes for analysis was reduced to 34 for adults, 21 for children, and 10 for neonates, excluding stillbirths. To ensure strict independence for the validation of methods and assessment of comparative performance, 500 test-train datasets were created from the universe of cases, covering a range of cause-specific compositions.ConclusionsThis unique, robust validation dataset will allow scholars to evaluate the performance of different verbal autopsy analytic methods as well as instrument design. This dataset can be used to inform the implementation of verbal autopsies to more reliably ascertain cause of death in national health information systems.

Rapid Scaling Up of Insecticide-Treated Bed Net Coverage in Africa and Its Relationship with Development Assistance for Health: A Systematic Synthesis of Supply, Distribution, and Household Survey Data

BACKGROUNDnDevelopment assistance for health (DAH) targeted at malaria has risen exponentially over the last 10 years, with a large fraction of these resources directed toward the distribution of insecticide-treated bed nets (ITNs). Identifying countries that have been successful in scaling up ITN coverage and understanding the role of DAH is critical for making progress in countries where coverage remains low. Sparse and inconsistent sources of data have prevented robust estimates of the coverage of ITNs over time.nnnMETHODS AND PRINCIPAL FINDINGSnWe combined data from manufacturer reports of ITN deliveries to countries, National Malaria Control Program (NMCP) reports of ITNs distributed to health facilities and operational partners, and household survey data using Bayesian inference on a deterministic compartmental model of ITN distribution. For 44 countries in Africa, we calculated (1) ITN ownership coverage, defined as the proportion of households that own at least one ITN, and (2) ITN use in children under 5 coverage, defined as the proportion of children under the age of 5 years who slept under an ITN. Using regression, we examined the relationship between cumulative DAH targeted at malaria between 2000 and 2008 and the change in national-level ITN coverage over the same time period. In 1999, assuming that all ITNs are owned and used in populations at risk of malaria, mean coverage of ITN ownership and use in children under 5 among populations at risk of malaria were 2.2% and 1.5%, respectively, and were uniformly low across all 44 countries. In 2003, coverage of ITN ownership and use in children under 5 was 5.1% (95% uncertainty interval 4.6% to 5.7%) and 3.7% (2.9% to 4.9%); in 2006 it was 17.5% (16.4% to 18.8%) and 12.9% (10.8% to 15.4%); and by 2008 it was 32.8% (31.4% to 34.4%) and 26.6% (22.3% to 30.9%), respectively. In 2008, four countries had ITN ownership coverage of 80% or greater; six countries were between 60% and 80%; nine countries were between 40% and 60%; 12 countries were between 20% and 40%; and 13 countries had coverage below 20%. Excluding four outlier countries, each US

Performance of physician-certified verbal autopsies: multisite validation study using clinical diagnostic gold standards

1 per capita in malaria DAH was associated with a significant increase in ITN household coverage and ITN use in children under 5 coverage of 5.3 percentage points (3.7 to 6.9) and 4.6 percentage points (2.5 to 6.7), respectively.nnnCONCLUSIONSnRapid increases in ITN coverage have occurred in some of the poorest countries, but coverage remains low in large populations at risk. DAH targeted at malaria can lead to improvements in ITN coverage; inadequate financing may be a reason for lack of progress in some countries. Please see later in the article for the Editors Summary.

Random forests for verbal autopsy analysis: multisite validation study using clinical diagnostic gold standards

BackgroundPhysician review of a verbal autopsy (VA) and completion of a death certificate remains the most widely used approach for VA analysis. This study provides new evidence about the performance of physician-certified verbal autopsy (PCVA) using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 VAs. The study was also designed to analyze issues related to PCVA, such as the impact of a second physician reader on the cause of death assigned, the variation in performance with and without household recall of health care experience (HCE), and the importance of local information for physicians reading VAs.MethodsThe certification was performed by 24 physicians. The assignment of VA was random and blinded. Each VA was certified by one physician. Half of the VAs were reviewed by a different physician with household recall of health care experience included. The completed death certificate was processed for automated ICD-10 coding of the underlying cause of death. PCVA was compared to gold standard cause of death assignment based on strictly defined clinical diagnostic criteria that are part of the Population Health Metrics Research Consortium (PHMRC) gold standard verbal autopsy study.ResultsFor individual cause assignment, the overall chance-corrected concordance for PCVA against the gold standard cause of death is less than 50%, with substantial variability by cause and physician. Physicians assign the correct cause around 30% of the time without HCE, and addition of HCE improves performance in adults to 45% and slightly higher in children to 48%. Physicians estimate cause-specific mortality fractions (CSMFs) with considerable error for adults, children, and neonates. Only for neonates for a cause list of six causes with HCE is accuracy above 0.7. In all three age groups, CSMF accuracy improves when household recall of health care experience is available.ConclusionsResults show that physician coding for cause of death assignment may not be as robust as previously thought. The time and cost required to initially collect the verbal autopsies must be considered in addition to the analysis, as well as the impact of diverting physicians from servicing immediate health needs in a population to review VAs. All of these considerations highlight the importance and urgency of developing better methods to more reliably analyze past and future verbal autopsies to obtain the highest quality mortality data from populations without reliable death certification.

Performance of the Tariff Method: validation of a simple additive algorithm for analysis of verbal autopsies

BackgroundComputer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs).MethodsThe Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE).ResultsFor all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates.ConclusionsWe found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

BackgroundVerbal autopsies provide valuable information for studying mortality patterns in populations that lack reliable vital registration data. Methods for transforming verbal autopsy results into meaningful information for health workers and policymakers, however, are often costly or complicated to use. We present a simple additive algorithm, the Tariff Method (termed Tariff), which can be used for assigning individual cause of death and for determining cause-specific mortality fractions (CSMFs) from verbal autopsy data.MethodsTariff calculates a score, or tariff, for each cause, for each sign/symptom, across a pool of validated verbal autopsy data. The tariffs are summed for a given response pattern in a verbal autopsy, and this sum (score) provides the basis for predicting the cause of death in a dataset. We implemented this algorithm and evaluated the methods predictive ability, both in terms of chance-corrected concordance at the individual cause assignment level and in terms of CSMF accuracy at the population level. The analysis was conducted separately for adult, child, and neonatal verbal autopsies across 500 pairs of train-test validation verbal autopsy data.ResultsTariff is capable of outperforming physician-certified verbal autopsy in most cases. In terms of chance-corrected concordance, the method achieves 44.5% in adults, 39% in children, and 23.9% in neonates. CSMF accuracy was 0.745 in adults, 0.709 in children, and 0.679 in neonates.ConclusionsVerbal autopsies can be an efficient means of obtaining cause of death data, and Tariff provides an intuitive, reliable method for generating individual cause assignment and CSMFs. The method is transparent and flexible and can be readily implemented by users without training in statistics or computer science.

Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards

BackgroundVerbal autopsy (VA) is an important method for obtaining cause of death information in settings without vital registration and medical certification of causes of death. An array of methods, including physician review and computer-automated methods, have been proposed and used. Choosing the best method for VA requires the appropriate metrics for assessing performance. Currently used metrics such as sensitivity, specificity, and cause-specific mortality fraction (CSMF) errors do not provide a robust basis for comparison.MethodsWe use simple simulations of populations with three causes of death to demonstrate that most metrics used in VA validation studies are extremely sensitive to the CSMF composition of the test dataset. Simulations also demonstrate that an inferior method can appear to have better performance than an alternative due strictly to the CSMF composition of the test set.ResultsVA methods need to be evaluated across a set of test datasets with widely varying CSMF compositions. We propose two metrics for assessing the performance of a proposed VA method. For assessing how well a method does at individual cause of death assignment, we recommend the average chance-corrected concordance across causes. This metric is insensitive to the CSMF composition of the test sets and corrects for the degree to which a method will get the cause correct due strictly to chance. For the evaluation of CSMF estimation, we propose CSMF accuracy. CSMF accuracy is defined as one minus the sum of all absolute CSMF errors across causes divided by the maximum total error. It is scaled from zero to one and can generalize a methods CSMF estimation capability regardless of the number of causes. Performance of a VA method for CSMF estimation by cause can be assessed by examining the relationship across test datasets between the estimated CSMF and the true CSMF.ConclusionsWith an increasing range of VA methods available, it will be critical to objectively assess their performance in assigning cause of death. Chance-corrected concordance and CSMF accuracy assessed across a large number of test datasets with widely varying CSMF composition provide a robust strategy for this assessment.

Automated quality control for mobile data collection

BackgroundVerbal autopsy (VA) is used to estimate the causes of death in areas with incomplete vital registration systems. The King and Lu method (KL) for direct estimation of cause-specific mortality fractions (CSMFs) from VA studies is an analysis technique that estimates CSMFs in a population without predicting individual-level cause of death as an intermediate step. In previous studies, KL has shown promise as an alternative to physician-certified verbal autopsy (PCVA). However, it has previously been impossible to validate KL with a large dataset of VAs for which the underlying cause of death is known to meet rigorous clinical diagnostic criteria.MethodsWe applied the KL method to adult, child, and neonatal VA datasets from the Population Health Metrics Research Consortium gold standard verbal autopsy validation study, a multisite sample of 12,542 VAs where gold standard cause of death was established using strict clinical diagnostic criteria. To emulate real-world populations with varying CSMFs, we evaluated the KL estimations for 500 different test datasets of varying cause distribution. We assessed the quality of these estimates in terms of CSMF accuracy as well as linear regression and compared this with the results of PCVA.ResultsKL performance is similar to PCVA in terms of CSMF accuracy, attaining values of 0.669, 0.698, and 0.795 for adult, child, and neonatal age groups, respectively, when health care experience (HCE) items were included. We found that the length of the cause list has a dramatic effect on KL estimation quality, with CSMF accuracy decreasing substantially as the length of the cause list increases. We found that KL is not reliant on HCE the way PCVA is, and without HCE, KL outperforms PCVA for all age groups.ConclusionsLike all computer methods for VA analysis, KL is faster and cheaper than PCVA. Since it is a direct estimation technique, though, it does not produce individual-level predictions. KL estimates are of similar quality to PCVA and slightly better in most cases. Compared to other recently developed methods, however, KL would only be the preferred technique when the cause list is short and individual-level predictions are not needed.

Performance of InterVA for assigning causes of death to verbal autopsies: multisite validation study using clinical diagnostic gold standards

Systematic interviewer error is a potential issue in any health survey, and it can be especially pernicious in low- and middle-income countries, where survey teams may face problems of limited supervision, chaotic environments, language barriers, and low literacy. Survey teams in such environments could benefit from software that leverages mobile data collection tools to provide solutions for automated data quality control. As a first step in the creation of such software, we investigate and test several algorithms that find anomalous patterns in data. We validate the algorithms using one labeled data set and two unlabeled data sets from two community outreach programs in East Africa. In the labeled set, some of the data is known to be fabricated and some is believed to be relatively accurate. The unlabeled sets are from actual field operations. We demonstrate the feasibility of tools for automated data quality control by showing that the algorithms detect the fake data in the labeled set with a high sensitivity and specificity, and that they detect compelling anomalies in the unlabeled sets.