Hyak Mortality Monitoring System: Innovative Sampling and Estimation Methods - Proof of Concept by Simulation
Samuel J. Clark, Jon Wakefield, Tyler McCormick, Michelle Ross
HHyak
Mortality Monitoring System
Innovative Sampling and Estimation Methods
Proof of Concept by SimulationSamuel J. Clark , Jon Wakefield , Tyler McCormick , and Michelle Ross Department of Sociology, The Ohio State University MRC/Wits Rural Public Health and Health Transitions Research Unit (Agincourt),School of Public Health, Faculty of Health Sciences, University of the Witwatersrand INDEPTH Network, Accra ALPHA Network, London Department of Statistics, University of Washington Department of Biostatistics, University of Washington Department of Sociology, University of Washington Department of Biostatistics and Epidemiology, University of Pennsylvania * Correspondence to: [email protected]
November 5, 2018
Abstract
Traditionally health statistics are derived from civil and/or vital registration. Civil registration in low-to middle-income countries varies from partial coverage to essentially nothing at all. Consequently thestate of the art for public health information in low- to middle-income countries is efforts to combine ortriangulate data from different sources to produce a more complete picture across both time and space – data amalgamation . Data sources amenable to this approach include sample surveys, sample registrationsystems, health and demographic surveillance systems, administrative records, census records, health facilityrecords and others.We propose a new statistical framework for gathering health and population data –
Hyak – that leveragesthe benefits of sampling and longitudinal, prospective surveillance to create a cheap, accurate, sustainablemonitoring platform.
Hyak has three fundamental components: • Data Amalgamation : a sampling and surveillance component that organizes two or more datacollection systems to work together: (1) data from HDSS with frequent, intense, linked, prospectivefollow-up and (2) data from sample surveys conducted in large areas surrounding the Health andDemographic Surveillance System (HDSS) sites using informed sampling so as to capture as manyevents as possible; • Cause of Death : verbal autopsy to characterize the distribution of deaths by cause at the populationlevel; and • SES : measurement of socioeconomic status in order to characterize poverty and wealth.We conduct a simulation study of the informed sampling component of
Hyak based on the Agincourt HDSSsite in South Africa. Compared to traditional cluster sampling,
Hyak ’s informed sampling captures moredeaths, and when combined with an estimation model that includes spatial smoothing, produces estimatesof both mortality counts and mortality rates that have lower variance and small bias. i a r X i v : . [ s t a t . O T ] M a y CKNOWLEDGMENTSPreparation of this manuscript was supported by the Bill and Melinda Gates Foundation and grantsK01 HD057246 and K01 HD078452 from the Eunice Kennedy Shriver National Institute of ChildHealth and Human Development (NICHD). JW was supported by R01-CA095994. The authors aregrateful to Peter Byass, Basia Zaba, Stephen Tollman, Adrian Raftery, Philip Setel, Osman Sankohand two very constructive anonymous reviewers for helpful discussions or other inputs.ii ontents
Hyak . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Hyak
Informed-Sampling via Simulation 7
A Appendix 25
A.1 Optimum allocation sampling strategy details . . . . . . . . . . . . . . . . . . . . . . 25A.2 Village-level characteristics for the current and historic cohorts . . . . . . . . . . . . 26A.3 Additional simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27iii
New Directions for Health and Population Statistics in Low- toMiddle-Income Countries
In most of the developed world the traditional source of basic public health information is civilregistration and vital statistics. Civil registration is a system that records births and deaths withina government jurisdiction. The purpose is two-fold: (1) to create a legal record for each person,and (2) to provide vital statistics. Optimally a civil register includes everyone in the jurisdiction,provides the basis to ensure their civil rights and creates a steady stream of vital statistics (UnitedNations. Statistical Division, 2014).The vital statistics obtained from many well-functioning civil registration systems include birthrates by age of mother, mortality rates by sex, age and other characteristics, and causes of deathfor each death. These basic indicators are the foundation of public health information systems, andwhen they are taken from a near-full-coverage civil registration system, they relate to the wholepopulation.Although the idea is inherently simple, implementing full-coverage civil registration is not, and onlythe world’s richest countries are able to maintain ongoing civil registration systems that cover amajority of the population. Civil registration in the rest of the world varies from partial coverage toessentially nothing at all (Mathers et al., 2005). A four-article series titled “Who Counts?” in the
Lancet in 2007 reviews the current state of civil registration (AbouZahr et al., 2007; Boerma andStansfield, 2007; Hill et al., 2007; Horton, 2007; Mahapatra et al., 2007; Setel et al., 2007). Thiswas followed eight years later with another four-article series presenting a similar but slightly morehopeful picture (AbouZahr et al., 2015b,a; Mikkelsen et al., 2015; Phillips et al., 2015). The authorslament that there has been a half a century of neglect in civil registration in low- to middle-incomecountries, and critically, that it is not possible to obtain useful vital statistics from those countries(Mahapatra et al., 2007; Setel et al., 2007; Mikkelsen et al., 2015).The
Lancet authors argue that in the long term all countries need complete civil registration toensure the civil rights of each one of their citizens and to provide useful, timely public healthinformation (AbouZahr et al., 2007, 2015a), and they explore a number of interim options thatwould allow countries to move from where they are today to full civil registration (Hill et al., 2007).Echoing the
Lancet special series are additional urgent pleas for better health statistics in low-and middle-income countries (for example: Abouzahr et al., 2010; Bchir et al., 2006; Matherset al., 2005, 2009; Rudan et al., 2000). The WHO and its partners and supporters have activelysupported improvements in civil registration and vital statistics (CRVS) over the recent past (WorldHealth Organization, 2013a,b, 2014). These workers clearly identify a need for representativedata describing sex-, age-, and cause-specific mortality through time in small enough areas to bemeaningful for local governance and health institutions. These critiques are for the most partdiscussed in the framework of civil registration as the ‘primary’ source of data.Recently, various United Nations agencies, including the office of the Secretary General, havearticulated strong, specific support for rapid improvement in the evidence base for the SustainableDevelopment Goals (SDG) (United Nations, 2014b) – the international target framework thatfollows from the MDGs (e.g., Data Revolution Group: The UN Secretary General’s IndependentExpert Advisory Group on a Data Revolution for Sustainable Development, 2014; Commissionon Population and Development, 2016; United Nations, 2016). The appropriately named
Data evolution (United Nations, 2014a) is the flagship program organized by the UN to address thesystematic lack of data to measure progress toward the SDG targets.We agree that in order to ensure civil rights and provide each unique citizen with a legal identity,full-coverage civil registration is the long-term goal. Acknowledging that, we propose decouplingthe discussion of civil registration from vital statistics. In particular, we can obtain accurate andrepresentative vital statistics measurements by making inferences from carefully adjusted sam-ples.The sample-based approach drives the production of population statistics in many other fieldsincluding economics, sociology and political science. Borrowing from these fields public healthworkers have developed sample-driven approaches to health statistics that partially substitute forvital statistics derived from civil registration. India has conducted a sample registration system(SRS) for several decades (Office of the Registrar General & Census Commissioner, India, 2012)that has produced good basic vital statistics, and more recently Jha and colleagues (2006) haveadded verbal autopsy (Lopez et al., 2011) to this system to create the Indian Million Death Study(MDS). In a similar vein, USAID’s Sample Vital Registration with Verbal Autopsy (SAVVY) isa program that combines sample registration with verbal autopsy and provides general-purposetools to collect data (MEASURE Evaluation, 2012). USAID’s Demographic and Health Surveys(DHS) (Measure DHS, 2012) and UNICEF’s Multiple Indicator Cluster Surveys (MICS) (UNICEF- Statistics and Monitoring, 2012) are good examples of traditional household surveys that describea select subset of indicators for national populations at multiple points in time. There are manymore similar sample surveys conducted by smaller organizations and aimed at specific diseases orthe evaluation of specific interventions.These approaches generally utilize sampling designs developed to provide cross-sectional snapshotsof the current state of the population with respect to an indicator. With the exception of India’sSRS and SAVVY, they lack the ongoing, prospective, longitudinal structure of a traditional vitalregistration system. They also often lack the spatial resolution to distinguish differences in indicatorvalues across short distances. Finally, they often miss or undercount rare events because theytypically take one measurement and rely on recall to fill in recent history.The current state of the art for public health information in low- to middle-income countries is effortsto combine or triangulate data from multiple sources to produce a more complete picture across bothtime and space. The usual sources of data include: non-representative, low-coverage, poor qualityvital registration data; roughly once-per-decade census data; snapshot or repeated snapshot datafrom (sometimes nationally representative) household surveys; one-off sample surveys conductedfor a variety of specific reasons by a diverse array of organizations; sample registration systems;and finally, a hodgepodge of miscellaneous data sources that may include health and demographicsurveillance systems (HDSS), sentinel surveillance systems, administrative records, clinic/hospitalrecords and others.Combining data from different sources with multiple sampling schemes presents a myriad of statis-tical challenges. We use data pooling as a broad term that describes methods that adjust for biasdue to differences in representativeness across data from different sources. The global burden ofdisease study by the Institute for Health Metrics and Evaluation (Naghavi et al., 2015) is a highlyvisible example of data pooling. As another example, Gething et al. (2011) pool survey data toproduce fine geographical scale Plasmodium falciparum malaria endemicity. Data amalgamation ,also uses data from multiple sources, but is differentiated by active engagement in the data collec-tion process. Data amalgamation uses proactive (e.g.
Hyak ) or adaptive mechanisms that actively2djust the data collection process to optimize a set of metrics – minimize bias, minimize variance,minimize cost, etc. A recent study of malaria prevalence by Kabaghe et al. (2017) is an example of data amalgamation in which survey locations are adaptively chosen to minimize the variance of atarget, see Chipeta et al. (2016) for statistical details. In the survey sampling literature, adaptivecluster sampling has a relatively long history (Thompson and Seber, 1996), and has been usedextensively in surveys of rare animal and plant species; we are not aware of any applications in thecontext considered here. In short, we use data pooling for situations where researchers combineseveral datasets not necessarily collected to measure the indicator of interest, whereas data amal-gamation is an intentional strategy that incorporates multiple heterogeneous data sources into thedesign process.Rowe (2009) and Bryce and Steketee (2010) describe a system of ‘integrated, continuous surveys’that would produce ongoing, longitudinal monitoring of a variety of outcomes – a proactive en-gagement with the data collection process in keeping with our definition of amalgamation . Datafrom such a system could be representative with respect to population, time and space and therebysubstitute for and improve on traditional vital statistics data. The idea is to systematize the na-tionally representative household surveys already implemented in a country, conduct them on aregular schedule with a permanent team and institute rigorous quality controls. The innovationis to turn traditional cross sectional surveys into something quasi longitudinal and to ensure alevel of consistency and quality. This concept appears to still be in the idea stage without anyreal methodological development or real-world testing. More in the spirit of data amalgamation,Bryce and colleagues (2004) use a variety of data sources to conduct a multi-country evaluation ofIntegrated Management of Childhood Illness (IMCI) interventions. This evaluation does developsome ad-hoc methods for combining and interpreting data from diverse sources.Victora et al. (2011) articulate a similar vision for a national platform for evaluating the effective-ness of public health interventions, specifically those targeting the Millennium Development Goals(MDG). The authors argue that national coverage with district-level granularity is necessary, andlike Rowe and Bryce, that continuous monitoring is required to assess changes and thereby inter-vention impacts. That article contains significant discussion of general survey methods, sample sizeconsiderations and other methodological requirements that would be necessary to evaluate MDGinterventions. Again however, there are no methodological details that would allow someone todesign and implement a national, prospective survey system of the type described.Several authors who work at HDSS sites have described an idea for carefully distributing HDSSsites throughout a country in way that could lead to a pseudo representative description of healthindicators in the country through time (Ye et al., 2012). Although these authors do not providedetails for how this could be done or evidence that it works, the basic idea is supported by work fromByass and his colleagues (2011) who examine the national representativeness of health indicatorsgenerated in individual Swedish counties in 1925. Byass and colleagues discover that any of thenot-obviously-unusual counties produced indicator values that were broadly representative of thenational population – the counties being roughly equivalent to an HDSS site, and Sweden in 1925being roughly equivalent to low- and middle-income countries today.Prabhat Jha (2012) summarizes all of this in his description of five ideas for improving mortalitymonitoring with cause of death. His five ideas include SRS systems with verbal autopsy, improvingthe representativeness of HDSS (similar to Ye and colleagues (2012)), coordinating, representativeretrospective surveys (similar to Rowe and Bryce) and finally using whatever decent-quality civilregistration data might be available. 3e find only two fully implemented and demonstrated examples of data amalgamation in the publichealth sphere. Alkema and colleagues (2007; 2008) working with the UNAIDS Reference Group onEstimates, Modelling and Projections develop a Bayesian statistical method that simultaneouslyestimates the parameters of an epidemiological model that represents the time-evolving dynamicsof HIV epidemics and calibrates the results of that model to match population-wide estimatesof HIV prevalence. The epidemiological model is fit to sentinel surveillance data describing HIVprevalence among pregnant women who attend antenatal clinics, and the population-wide measuresof prevalence come from DHS surveys. Interestingly the second example relates to a similar problem.Lanjouw and Ivaschenko at the World Bank (2010) describe a method to amalgamate population-level data from DHS surveys and HIV prevalence data from a sentinel surveillance system. TheDHS contains representative information on a variety of items but not HIV prevalence, and thesentinel surveillance system describes the HIV prevalence of a select (non-representative) subgroup,again pregnant women who attend antenatal clinics. Building on ideas in small-area estimation,they develop and demonstrate a method to adjust the sentinel surveillance data and then predictthe HIV prevalence of the whole population.Although these are two specific applications of data amalgamation, it is this level of conceptualand methodological detail that are necessary in order to amalgamate data from different sources toproduce representative, probabilistically meaningful results.
The population, public health and eval-uation literatures are full of urgent requests for better data and more useful methods to amalgamatedata from different sources to answer questions about cause and effect and change at national andsubnational levels, but there is very little in any of those literatures that actually develops the newconcepts and methods that are necessary to deliver the required new capabilities. Chipeta et al.(2016) describe an adaptive design whose aim is to estimate disease prevalence.
Taking account of the situation described in the literature and firmly in the spirit of ‘data amalgama-tion’, we aim to develop a system that provides high quality, continuously generated, representativevital statistics and other population and health indicators using a system that is cheap and logisti-cally tractable. We are confident that such a system can provide highly useful health informationat all important geographical (and other) scales: nation, province, district, and perphaps evensubdistrict.As we argue above, we strongly believe that a sample-based approach is both appropriate andsufficient to produce meaningful, useful public health information, and we do not believe it isfiscally responsible to attempt to cover the entire population with a public health informationsystem. That argument must be made on the basis of guaranteeing human rights alone . What we want is a cheap, sustainable , continuously operated monitoring system that combinesthe benefits of both sample surveys (representativity, sparse sampling, logistically tractable) andsurveillance systems (detailed, linked, longitudinal, prospective with potentially intense monitoring– e.g. of pregnancy outcomes and neonatal deaths) to provide useful indicators for large popula-tions over prolonged periods of time, so that we can monitor change and relate changes to possible4eterminants, including interventions. More specifically, ‘useful’ in this context means an informa-tive balance of accuracy (bias) and precision (variance) – i.e. minimal but probably not zero biasaccompanied by moderate variance.
We want indicators that are close to the truth most of thetime , and we want an ability to study causality properly. Critically, we want the whole system tobe cheaper and more sustainable than existing systems, and perhaps offer additional advantages aswell.
Hyak
We propose an integrated data collection and statistical analysis framework for improved populationand public health monitoring in areas without comprehensive civil registration and/or vital statisticssystems. We call this platform
Hyak – a word meaning ‘fast’ in the Chinook Jargon of theNorthwestern United States.
Hyak is conceived as having three fundamental components: • Data Amalgamation : a sampling and surveillance component that organizes two data col-lection systems to work together to provide the desired functionality: (1) data from HDSSwith frequent, intense, linked, prospective follow-up and (2) data from sample surveys con-ducted in large areas around the HDSS sites using informed sampling so as to capture asmany events as possible. • Verbal Autopsy (Lopez et al., 2011) to estimate the distribution of deaths by cause at thepopulation level, and • SES : measurement of socioeconomic status (SES) at household, and perhaps other levels, inorder to characterize poverty and wealth.Hyak uses relatively small, intensive, longitudinal HDSS sites to understand what types of individ-uals (or households) are likely to be the most informative if they were to be included in a sample.With this knowledge the areas around the HDSS sites are sampled with preference given to themore informative individuals (households), thus increasing the efficiency of sampling and ensuringthat sufficient data are collected to describe rare populations and/or rare events. This fully utilizesthe information generated on an on-going basis by the HDSS and produces indicator values thatare representative of a potentially very large area around the HDSS site(s). Further, the informa-tion collected from the sample around the HDSS site can be used to calibrate the more detaileddata from the HDSS, effectively allowing the detail in the HDSS data to be extrapolated to thelarger population. For an example of how this has been done in the context of antenatal clinicHIV prevalence surveillance and DHS surveys, see Alkema et al. (2008). Another way to do thisis to build a hierarchical Bayesian model of the indicator of interest, say mortality, with the HDSSbeing the first (informative) level and the surrounding areas being at the second level. Thus thesurrounding area can borrow information from the HDSS but is not required to match or mirrorthe HDSS.In the remainder of this work we focus on the informed sampling component of
Hyak . Informedsampling seeks to capture as many events as possible. This is critical for the measurement ofmortality, and especially for the measurement of cause-specific mortality fractions (CSMF) at thepopulation level. In order to adequately characterize the epidemiology of a population, it is nec-essary to measure the CSMF with some precision, and to do this a large number of death events5ith verbal autopsy are required, especially for rare causes. Informed sampling aims to make themeasurement of mortality rates and CSMFs as efficient as possible.Below we present a detailed example of the informed sampling idea and a pilot study based on in-formation from the Agincourt HDSS site in South Africa (Kahn et al., 2007, 2012). The AgincourtHDSS is situated in the rural northeast of South Africa and covers an area of 420km comprisinga sub-district of 27 villages. The site monitors roughly 90,000 people in 16,000 households. Thevillages and households are dispersed widely across this area, and there is a functional road networklinking them all. The epidemiology of the site is typical for South Africa with generally low mor-tality except for the effect of HIV at very young and middle ages, and in terms of wealth/poverty,the population is typical of a middle-income country (e.g. Kabudula et al., 2016; Clark et al., 2015;Houle et al., 2014; Clark et al., 2013; G´omez-Oliv´e et al., 2013; Houle et al., 2013). The AgincourtHDSS is the canonical HDSS, not extreme along any dimension, and generally representative ofwhat an HDSS site is.We generate virtual populations based on information from the Agincourt site, and then we simulateapplications of traditional two-stage cluster and Hyak sampling designs. We estimate sex-age-specific mortality rates for children ages 0 − Hyak system and the ‘demographic feasibility’ of
Hyak .We are thinking about existing data collection methods and these objectives in a unified framework,and we are starting by experimenting with sampling and analytical frameworks that work togetherto provide the basis for a measurement system that is representative, accurate and efficient interms of information gained per dollar spent (not the same as cheap in an absolute sense becauseestimation of a binary outcome like death is still bound by the fundamental constraints of thebinomial model; i.e. relatively large numbers of deaths are needed for useful measurements).A measurement system like this would be among the cheapest and most informative ways to monitorthe mortality of children affected by interventions that cover large areas and exist for prolongedperiods of time. With this in mind, the pilot project we present below focuses on childhood ages0 − From Kahn et al. (2012): The Agincourt health and socio-demographic surveillance system (HDSS), located inrural northeast South Africa close to the Mozambique border, was established in 1992 to support district healthsystems development led by the post-apartheid ministry of health. At baseline in 1992, 57,600 people were recordedin 8,900 households in 20 villages; by 2006, the population had increased to 70,000 people in 11,700 households.This increase is partly due to Mozambican in-migrants overlooked in the baseline survey and to a new settlementestablished as part of the post-apartheid governments Reconstruction and Development Program. In 2007, the studyarea was extended to include the catchment area of a new privately supported community health centre establishedto provide HIV treatment before public sector roll-out of HAART. By mid-2011, the population under surveillancecomprised 90,000 people residing in 16,000 households in 27 villages. Households are self-defined as people who eatfrom the same pot of food. Given sustained high levels of temporary labour migration in southern Africa, we includedtemporary migrants residing for less than 6 months per year who retain close ties with their rural homes in the HDSS.There have been 17 census and vital event update rounds conducted strictly annually since 2000. Participation isvirtually complete, with only two households refusing to participate in 2011. Pilot Study of
Hyak
Informed-Sampling via Simulation
In this section we describe our approach to sampling and analysis. To be concrete, we suppose thatthe outcome of interest is alive or dead for children age 0 −
4. There are two novel aspects to ourapproach: • Informed Sampling:
Using existing information from a HDSS site we construct a mortalitymodel based on village-level characteristics. On the basis of this model we subsequentlypredict the number of outcomes of interest in each village of the study region. We then setsample sizes in each village in proportion to these predictions. • Analysis:
We model the sampled deaths as a function of known demographic factors andvillage-level characteristics, and then we employ spatial smoothing to tune the model to eachvillage and exploit similarities of risk in neighboring villages.
Given our interest in the binary status alive or dead , our modeling framework is logistic regressionwith random effects. Specifically, let i = 1 , . . . , I represent villages within the study region and j = 1 , . . . , ,
1) years,old: [1 ,
5) years). Households within areas will be represented by k = 1 , . . . , K i , for i = 1 , . . . , I .The quantity of interest is Y ij , the unobserved true number of deaths in village i and in sex/agestratum j . We assume that the populations N ij are known in all villages. Also assumed known arevillage-specific covariates X i (for example, the average SES in village i , a measure of water quality,or proximity to health care facilities).The probability of dying in village i and stratum j is denoted by p ij , which is the hypotheticalproportion of children dying in a hypothetical infinite population in area i and strata j . Westress that we are carrying out a small-area estimation problem so the target of interest is Y ij andthe probability is just an intermediary which allows us to set up a model. If the full data wereobserved, we would take the probability to be the observed frequency ˜ p ij = Y ij /N ij . The surveydesign problem corresponds to choosing n ij , the number of children in stratum j that we samplein village i . Of these, y ij are recorded as dying.In the next section, we describe models that will be used to analyze the data; once we have estimatedprobabilities from a generic model, (cid:98) p ij , we use the estimator: (cid:98) Y ij = y ij + ( N ij − n ij ) × (cid:98) p ij , (1)where y ij is the observed number of deaths and ( N ij − n ij ) is the number of unsampled individualsin village i and stratum j . In this section, we describe models that may be fit to the sampled data.7
Na¨ıve Model:
This baseline model simply estimates (cid:98) p = y/n , i.e., a single probability isapplied to the unsampled individuals in each village. The predicted number of deaths in eachvillage is then (1) with (cid:98) p ij = (cid:98) p .II Strata Model:
This model estimates (cid:98) p j = y j /n j , so that estimates of four stratum-specificprobabilities are calculated. The predicted number of deaths in each village is then (1) with (cid:98) p ij = (cid:98) p j .III Covariate Model:
This approach fits a model to data from all villages where samplingwas carried out and estimates stratum effects along with the association between risk andvillage-level covariates x i . We assume a logistic form,logit p ij = x i β + γ j , (2)where j = 1 , . . . ,
4. Hence, we have a model with a separate baseline for each stratum andwith the covariates having a common effect across stratum and village, so there no interactionbetween covariates and stratum, and covariates and area. We use the maximum likelihoodestimates (cid:98) γ j and (cid:98) β to obtain fitted probabilities: (cid:98) p ij = expit( x i (cid:98) β + (cid:98) γ j ) = exp( x i (cid:98) β + (cid:98) γ j )1 + exp( x i (cid:98) β + (cid:98) γ j ) , which may be used in (1).IV Spatial Covariate Model:
This approach requires sufficient villages to have sampled dataso that spatial random effects can be estimated. Specifically, we assume a Bayesian imple-mentation of the model: logit p ijk = x i β + γ j + (cid:15) i + S i + h k , (3)where j = 1 , . . . ,
4. We have three random effects in this model. The unstructured village-and household-level error terms (cid:15) i ∼ iid N (0 , σ (cid:15) ) and h k ∼ iid N (0 , σ h ), respectively, are in-dependent and allow for excess-binomial variability. The household-level random effects alsoallow for dependence within households. The S i error terms are village-level spatial randomeffects that allow the smoothing of rates across space. There are many different forms thatthese random effects could take. A model-based geostatistical approach (Diggle et al., 1998)would assume the collection [ S , . . . , S n ] arise from a multivariate normal distribution, withcovariances a function of the distance between villages. We go a different route and use anintrinsic conditional auto-regressive (ICAR) model (Besag et al., 1991) in which: S i | S j , j ∈ ne( i ) ∼ N ( S i , σ s /n i ) , where ne( i ) is the set of neighbors of village i and n i is the number of such neighbors. Thismodel assumes that the prior distribution for the spatial effect in area i , given its neighbors,is centered on the mean of the neighbors, with a variance that depends on the numberof neighbors (with more neighbors reducing the prior variance). We describe our ‘sharedboundary’ neighborhood scheme in the next section. We use the posterior means (cid:98) β , (cid:98) γ j , (cid:98) (cid:15) i and (cid:98) S i to obtain fitted probabilities: (cid:98) p ij = exp( x i (cid:98) β + (cid:98) γ j + (cid:98) (cid:15) i + (cid:98) S i )1 + exp( x i (cid:98) β + (cid:98) γ j + (cid:98) (cid:15) i + (cid:98) S i ) , INLA R package implements the INLA method. A Bayesianimplementation requires specification of priors for all of the unknown parameters, which formodel (3) consist of β , γ , σ (cid:15) , σ s and σ h . We choose flat priors for β , γ , and Gamma( a, b )priors for σ − (cid:15) , σ − s and σ − h . Figure 1:
The 20 villages of the Agincourt region with Voronoi tesselations defining neighborhoodstructure. Grey lines indicate neighboring villages.
We describe the study region that we create for the simulation study, in order to provide a contextwithin which the different sampling strategies can be described. The study region is based onthe Agincourt HDSS site in South Africa (Kahn et al., 2007, 2012). We assume N individualsreside in one of 20 villages and that there are between 1,400 and 14,000 children in each village, N i ∼ Unif(1400 , − − • P (household with 1 child) = 75 /
470 = 0 . • P (household with 2 children) = 100 /
470 = 0 . • P (household with 3 children) = 125 /
470 = 0 . • P (household with 4 children) = 100 /
470 = 0 . P (household with 5 children) = 70 /
470 = 0 . . We sample a single population of N children and then take S = 100 repeated draws from thispopulation under the four sampling schemes described below. Beginning with the denominators N ij , we sample the observed deaths y ij using a binomial with probabilities given by (2).We sample a second population of N children and treat this population as a historical cohort. It isfrom this population that we treat three of these villages as HDSS sites for which we have extensiveand complete information.We form a Voronoi tessellation of the village boundaries based on the 20 coordinate pairs thatdescribe the centroids of the villages. This operation forms a set of tiles, each associated with acentroid and is the set of points nearest to that point. This is a standard operation in spatialstatistics (e.g. Denison and Holmes, 2001). We can then define neighbors (for the spatial model)as those villages whose tiles share an edge. Figure 1 shows the study region along with villagecentroids and associated village polygons (as defined by the Voronoi tesselations), along with edgesshowing the neighborhood structure. In this section we describe the sampling strategies that we compare. In each strategy, we considerfour different sample sizes, n , for the total number of children sampled: 1,300, 2,600, 3,900 and5,200. • Two-stage Cluster Sampling:
Randomly select 5 villages and randomly sample ( n/ / n/ • Stratified Sampling:
Randomly sample n/
20 children’s outcomes from each of the 20villages. This strategy lies between the cluster sampling and informed sampling designs. • Hyak – HDSS with Informative Sampling:
The number of children sampled from eachvillage is proportional to the predicted number of deaths based on the HDSS data. In particu-lar, we select all children from the three HDSS villages in the historical cohort and we fit model(2). On the basis of the estimated β , γ , we obtain predicted counts of deaths for all villages,using the village-level covariates x i , i = 1 , . . . , I . Let β (cid:63) , γ (cid:63) be the estimated parametersbased on the historic HDSS data only and p (cid:63)ij be the associated village and stratum-specificprobabilities. We estimate p (cid:63)i via p (cid:63)i = J (cid:88) j =1 N ij N i p (cid:63)ij . Then, the predicted number of deaths are (cid:101) Y i = N i × p (cid:63)i . We then select sample sizes as the(rounded versions of) n i ∝ (cid:101) Y i so that villages with more predicted deaths are sampled moreheavily. Specifically, we take n i = n × (cid:101) Y i / (cid:101) Y + where (cid:101) Y + is the total predicted number ofdeaths. The observed number of deaths from n i is y i . • Optimum Allocation:
As in the
Hyak sampling design, we obtain the village-level esti-mates of the probability of death, p (cid:63)i , based on the historic HDSS data only. We then select10ample sizes as the (rounded versions of) n i = n × N i (cid:112)(cid:98) p i (1 − (cid:98) p i ) (cid:80) i (cid:48) N i (cid:48) (cid:112)(cid:98) p i (cid:48) (1 − (cid:98) p i (cid:48) ) . Details are provided in the Appendix.
Given N total children, broken into the four stratum, we can set risks p ij (details of which appearin Section 2.2) for each village/stratum and then simulate counts Y ij . We take this set of { Y ij : i = 1 , . . . , j = 1 , . . . , } as fixed, and then subsample from these counts, under each of the fourdesigns and repeat s = 1 , . . . , S times.The estimated number of deaths in survey villages in simulation s is (cid:98) Y ( s ) ij = y ( s ) ij + ( N ij − n ij ) × (cid:98) p ( s ) ij where the (cid:98) p ( s ) ij are obtained from one of the models we described in Section 2.1.2.To estimate the frequentist properties of the simulation procedure, we summarize the results byexamining various summary measures. An obvious measure of accuracy is the mean squared error(MSE) associated with the predicted number of deaths. The MSE of an estimator of the numberof deaths in area i and strata j , (cid:98) Y ij averaged over villages and strata isMSE( (cid:98) Y ij ) = E (cid:20)(cid:16) y ij − (cid:98) Y ij (cid:17) (cid:21) , where y ij is the true number of deaths (which recall, is fixed), and the expectation is over all possiblesamples that can be taken (for whichever design we are considering). This MSE is estimated basedon S simulations: MSE( (cid:98) Y ) = 1 S S (cid:88) s =1 20 (cid:88) i =1 4 (cid:88) j =1 ( (cid:98) Y ( s ) ij − Y ij ) = (cid:88) i =1 4 (cid:88) j =1 ( (cid:98) Y ij − Y ij ) + 1 S S (cid:88) s =1 20 (cid:88) i =1 4 (cid:88) j =1 ( (cid:98) Y ( s ) ij − (cid:98) Y ij ) = (cid:88) i =1 4 (cid:88) j =1 Bias( (cid:98) Y ij ) + (cid:88) i =1 4 (cid:88) j =1 Var( (cid:98) Y ij ) . (4)where Y ij is the true number of deaths in village i and stratum j and (cid:98) Y ij = 1 S S (cid:88) s =1 (cid:98) Y ( s ) ij is the average of the predicted counts over simulations in village i and stratum j . The decompo-sition in terms of bias and variance is useful since it makes apparent the trade-off involved inmodeling. 11 -0.22,0.08) [0.08,0.38) [0.38,0.67) [0.67,0.97) [0.97,1.27] Figure 2:
The simulated spatial random effects for the Agincourt region.
We assume there are two village-level covariates so that the length of the β vector is 2. Both ofthe village-level covariates x i and x i are generated independently from uniform distributions on0 to 1, i = 1 , . . . ,
20. Based loosely on the real values from the Agincourt HDSS in South Africa,the parameter values we use in the simulation are: • The risk of death in young girls is expit( γ ) = 0 . • The risk of death in young boys is expit( γ ) = 0 . • The risk of death in older girls is expit( γ ) = 0 . • The risk of death in older boys is expit( γ ) = 0 . • The first village-level covariate has exp( β ) = exp( − .
2) = 0 .
111 so that a unit increase in x leads to the odds of death dropping by one ninth. • The second village-level covariate has exp( β ) = exp(1 .
4) = 4 .
05 so that a unit increase in x leads to the odds of death quadrupling. • We set σ (cid:15) = 0 .
22 to determine the level of unstructured variability at the village level.This leads to a 95% range for the residual unstructured village-level odds being exp( ± . ×√ .
22) = [0 . , . • We set σ s = 0 .
48 to determine the level of structured variability at the village level. Thisoperation requires some care because the ICAR model does not define a proper probabilitydistribution. The ICAR variance is not interpretable as a marginal variance (and so is notcomparable to the other random effects variances, σ (cid:15) and σ h ) and so instead Figure 2 showsa simulated set of S i , i = 1 , . . . ,
20 values, with darker values indicating higher risk. Thespatial dependence is apparent, with this realization producing high risk to the West of theregion and low risk in the East. 12
We set σ h = 0 .
08 to determine the level of unstructured variability at the household level. Thisleads to a 95% range for the residual unstructured household-level odds being exp( ± . ×√ .
08) = [0 . , . [0.01,0.08)[0.08,0.14)[0.14,0.21)[0.21,0.27)[0.27,0.34] (a) [0.01,0.08)[0.08,0.14)[0.14,0.21)[0.21,0.27)[0.27,0.34] (b) [0.01,0.08)[0.08,0.14)[0.14,0.21)[0.21,0.27)[0.27,0.34] (c) [0.01,0.08)[0.08,0.14)[0.14,0.21)[0.21,0.27)[0.27,0.34] (d) Figure 3:
The predicted probabilities of dying for the Agincourt region: (a) young girls, (b) youngboys, (c) older girls, (d) older boys.Combining all of the elements of the model, we generate deaths Y ij for village i and stratum j byrandomly drawing from a Binomial distribution with probabilities given by (3). This yields thepredicted probabilities for all 20 villages and for each of the four stratum displayed in Figure 3.13he historic cohort is generated in the same fashion. Details of the village-level characteristics forboth cohorts are provided in Appendix A.2.The HDSS villages are selected by taking the villages with both large x and large x , small x andsmall x , followed by a randomly sampled third village.A Ga (5 ,
1) prior is used for the spatial and non-spatial random effects in the spatial models (ModelIV).
Table 1 summarizes the results of the simulation study for n = 5 , Hyak sampling strategy captures more deaths and is generally more accurate. Acrosssampling schemes and sample sizes,
Hyak generally has the smallest MSEs. Further examinationof the components of the MSE reveals that: (i)
Hyak yields smaller bias, and (ii) pays for this bysacrificing some variance. The overall comparison between the sampling strategies clearly favors
Hyak . This partly reflects the careful choice of HDSS villages so that they contain substantialvariation in terms of village-level covariates. 14 able 1:
Deaths, Bias, Variance, MSE for cluster sampling, stratified sampling,
Hyak andoptimum sampling for n = 5 , S = 100 simulations. Therewere 11,299 deaths in the simulated population from which samples were taken.‘Cluster’ is shorthand for Two-stage Cluster Sample ; ‘
Hyak ’ for
HDSS with Infor-mative Sampling ; ‘Strata/Covariates’ for
Logistic Regression Covariate Model and‘Strata/Covariates/Space’ for
Logistic Regression Random Effects Covariate Model .It is not possible to fit the spatial model (IV) to the two-stage cluster samplingscheme since there are data from 5 villages only.Design Model Deaths Bias Variance ( × ) MSE ( × )Cluster I. Na¨ıve 459 1,067 174 1,312II. Strata 459 874 188 951III. Strata/Covariates 459 651 386 810IV. Strata/Covariates/Space 459 — — —Stratified I. Na¨ıve 460 1,058 5 1,124II. Strata 460 866 15 765III. Strata/Covariates 460 651 16 439IV. Strata/Covariates/Space 460 183 80 113Hyak I. Na¨ıve 538 1,162 7 1,357II. Strata 538 969 18 956III. Strata/Covariates 538 635 16 419IV. Strata/Covariates/Space 538 182 66 100Optimum I. Na¨ıve 477 1,072 5 1,154II. Strata 477 880 18 792III. Strata/Covariates 477 632 17 416IV. Strata/Covariates/Space 477 167 74 102Comparing the analytical models also produces an encouraging result. Within each samplingstrategy, the logistic regression random effects covariate model (model IV) performs best overall(smaller MSEs). Within Hyak , this outperforms the others. Similar patterns are observed acrossall sample sizes. This suggests that accounting for unmeasured factors and taking advantage of thespatial structure of mortality risk is significantly worthwhile.The trade-off between bias and variance is clearly revealed by a closer look at the distributions ofthe estimated probability of dying produced by each model. Figure 4 displays these distributionsfor models I, III & IV –
Na¨ıve, Covariates and
Covariates & Space under the
Hyak samplingstrategy for n = 5 , n = 3 , , n = 2 ,
600 and n = 1 , Na¨ıve model estimates are very condensed,always miss the truth and have clear bias; estimates from the
Covariates model also have very littlespread, almost always miss the truth and have some bias; and finally, estimates from the
Covariates& Space model have large spread, however the distributions nearly always include the truth, andhave much less bias. Clearly the
Covariates & Space model displays the balance we are seeking:small bias and manageable spread, and importantly, distributions that include the truth. Thiscombination of sampling strategy and analytical approach provides our key objective: an indicatorthat is close to (and around) the truth most of the time.15igure 5 displays the average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models under the
Hyak sampling scheme for n = 5 , y = x line quite closely, indicating we are estimating the true number of deaths in each village quitewell. Estimates tend to be closer under the Hyak sampling strategy and for larger sample sizes,thus confirming (visually) our previous results. . . . . . P ( dea t h ) Young Females Young Males Old Females Old Males l l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll
Model IModel IIIModel IVTruth
Figure 4:
The distributions of the estimated probability of dying from models I, III and IV underthe
Hyak sampling strategy for n = 5 , The key conclusion of this pilot study is that the statistical sampling and analysis ideas supportingthe
Hyak monitoring system are sound: a combination of highly informative data such as areproduced by a HDSS site can be used to judiciously inform sampling of a large surrounding area to16
Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400 500
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure 5:
The average village- and strata-specific estimates for the (unobserved) population countsof death plotted against the true values across each of the four models under the
Hyak sampling scheme for n = 5 , y = x line of equality.17ield estimated counts of deaths that are far more useful than those produced by a traditional clustersample design. Further, Hyak combined with an analytical model that includes unstructuredrandom effects and spatial smoothing produces the most accurate and well-behaved estimates. Theimprovements are dramatic and clearly justify additional work on these ideas.Another crucial idea underlying
Hyak is the notion that very detailed information generated byan HDSS site can be extrapolated to the much larger surrounding population by calibrating thatinformation with carefully chosen and much less detailed data from the surrounding population.This idea has already been demonstrated convincingly by Alkema et al. [2008] and is currently beingapplied by UNAIDS to produce global estimates of HIV prevalence. This relies on the assumptionthat the population monitored by the HDSS is similar enough to the population surrounding theHDSS that the relationships between covariates and the outcomes of interest are the same or verysimilar. The degree to which this is true will vary among specific settings. In particular, whenHDSS sites also serve as research and intervention testing sites, it is possible that there will be
Hawthorne Effect issues – i.e. the intensively studied HDSS population will be different from thesurrounding population that has not participated in studies and trials. This may affect the keycovariate-outcome relationships that drive
Hyak . This is something that must be studied, initiallywith a real-world pilot study of
Hyak , and then in an ongoing way by occasionally verifyingthese relationships through an oversample of the surrounding population, or through small add-onstudies conducted whenever a census is done in the surrounding areas to update the sampling frame.Although this is a concern, it is unlikely to make
Hyak infeasible or invalidate
Hyak results. Anexplicit goal of a pilot study will be to characterize the uncertainty created by possible HawthorneEffect issues and build them into
Hyak estimates.A key advantage of
Hyak sampling strategy is that it captures significantly more deaths . Verbalautopsy methods (Lopez et al., 2011) can be applied to all or a fraction of these deaths to as-sign causes (immediate, contributing, etc.). This cause of death information can then be used toconstruct distributions of deaths by cause – CSMFs – which illuminate the epidemiological regimeaffecting the population, and if this is monitored through time, how the epidemiology of the pop-ulation is changing. Critically, this provides a means of measuring the impact of interventionson specific causes of death and the distribution of deaths over time. The increased number ofdeaths captured with informed sampling increases the accuracy and precision of measurements ofCSMFs.A final benefit of the
Hyak system is that it provides two types of infrastructure: the HDSS andthe sample survey. In addition to providing information with which to sample, the HDSS providesa platform on which a wide variety of longitudinal studies can be undertaken – linked observationalstudies; randomized, controlled trials, all kinds of combinations of these, etc. Moreover, the per-manent HDSS infrastructure also provides a training platform that can support a wide variety ofhealth and behavioral science training, mentoring and apprenticing/interning and experience foryoung scientists or health professionals. Having the sample survey infrastructure provides a meansof quickly validating/calibrating studies conducted by the HDSS and provides another learningdimension for the educational and training activities that the system can support.A potential limitation of any mortality monitoring system is ‘demographic feasibility’, that is theability to capture enough deaths in a given population to measure levels and/or changes in mortality,potentially by cause, through time. Death is a binomial process defined by a probability of dying,and as such, is governed by the characteristics of the binomial model. That model specifies insimple terms the number of deaths necessary to estimate the probability of dying within a givenmargin of error with a given level of confidence. No amount of sophistication will release us from18hat basic set of facts. The
Hyak system addresses this challenge by providing a means throughwhich to choose the best possible sample given what we know about the population, and this in turnmaximizes our ability to capture deaths. The fundamentals of the binomial model require that onemust observe relatively large numbers of deaths to measure mortality precisely and especially tomeasure changes in mortality with both precision and confidence. So in light of those inescapablerealities, the
Hyak system produces the most information per dollar spent, because it capturesmore deaths per dollar spent.Finally and perhaps most importantly, the
Hyak monitoring system is cheaper to run over a periodof years compared to traditional cluster sample-based survey methods. Combined with the factthat
Hyak also produces more useful information, this makes
Hyak highly cost effective – morebang for less buck.
Importantly, there are implementation considerations that must be addressed before
Hyak canbe used at provincial or national scale to provide population-representative estimates. These willneed to be resolved through additional theoretical work, simulation, and ultimately through a pilotstudy that conducts
Hyak on a large population dispersed over a large physical space. Amongmany, these critical questions need to be answered: • How big do HDSS sites need to be to provide enough information for effective informativesampling? • How many HDSS sites are necessary for effective informed sampling with respect to keydemographic and epidemiological indicators? • How should HDSS sites be dispersed geographically? • How well does
Hyak work to provide disaggregated (fine-grained) estimates of key indicatorsby sex, age, wealth/poverty, space, time, etc? • How much does the sampling frame affect
Hyak results, and what cheap, feasible solutionsare there to obtaining frequently updated sampling frames? • A detailed costing and cost comparison needs to be done comparing the costs of the the HDSSsite; the additional census, sampling, and interviewing needed for
Hyak ; and a traditionalhousehold multi-stage cluster sample survey (like DHS) conducted in the same area. • How the method can be scaled up to a larger geographical area. We envisage that only asubset of villages will be sampled, and then a geostatistical model (Wakefield et al., 2016)can be used for spatial prediction to unobserved villages (a critical question is the numberof villages needed to train the spatial model). Another important issue is to deal with thepotential problem of preferential sampling (Diggle et al., 2010) in which sampling locationsare selected based on the expected size of the response. In order to inform sampling historicaldata (for example, DHS surveys) may be used to model to create a predictive surface, uponwhich sampling may be based. Investigating this idea will be the subject of a future paper.19 eferences
AbouZahr, C., Cleland, J., Coullare, F., Macfarlane, S. B., Notzon, F. C., Setel, P., Szreter, S.,Anderson, R. N., Bawah, A. a., Betr´an, A. P., Binka, F., Bundhamcharoen, K., Castro, R.,Evans, T., Figueroa, X. C., George, C. K., Gollogly, L., Gonzalez, R., Grzebien, D. R., Hill, K.,Huang, Z., Hull, T. H., Inoue, M., Jakob, R., Jha, P., Jiang, Y., Laurenti, R., Li, X., Lievesley,D., Lopez, A. D., Fat, D. M., Merialdi, M., Mikkelsen, L., Nien, J. K., Rao, C., Rao, K., Sankoh,O., Shibuya, K., Soleman, N., Stout, S., Tangcharoensathien, V., van der Maas, P. J., Wu, F.,Yang, G., and Zhang, S. (2007). The way forward.
Lancet , 370(9601):1791–9.AbouZahr, C., De Savigny, D., Mikkelsen, L., Setel, P. W., Lozano, R., and Lopez, A. D. (2015a).Towards universal civil registration and vital statistics systems: the time is now.
The Lancet ,386(10001):1407–1418.AbouZahr, C., De Savigny, D., Mikkelsen, L., Setel, P. W., Lozano, R., Nichols, E., Notzon, F.,and Lopez, A. D. (2015b). Civil registration and vital statistics: progress in the data revolutionfor counting and accountability.
The Lancet , 386(10001):1373–1385.Abouzahr, C., Gollogly, L., and Stevens, G. (2010). Better data needed: everyone agrees, but noone wants to pay.
Lancet , 375(9715):619–21.Alkema, L., Raftery, A., and Brown, T. (2008). Bayesian melding for estimating uncertainty innational hiv prevalence estimates.
Sexually transmitted infections , 84(Suppl 1):i11–i16.Alkema, L., Raftery, A., and Clark, S. (2007). Probabilistic projections of hiv prevalence usingbayesian melding.
The Annals of Applied Statistics , 1(1):229–248.Bchir, A., Bhutta, Z., Binka, F., Black, R., Bradshaw, D., Garnett, G., Hayashi, K., Jha, P., Peto,R., Sawyer, C., Schwartl¨ander, B., Walker, N., Wolfson, M., Yach, D., and Zaba, B. (2006).Better health statistics are possible.
Lancet , 367(9506):190–3.Besag, J., York, J., and Molli`e, A. (1991). Bayesian image restoration, with two applications inspatial statistics (with discussion).
Annals of the Institute of Statistical Mathematics , 43:1–59.Boerma, J. T. and Stansfield, S. K. (2007). Health Statistics 1 Health statistics now : are wemaking the right investments ?
Tuberculosis , 369(9563):779–786.Bryce, J. and Steketee, R. (2010). Continuous surveys and quality management in low-incomecountries: a good idea.
The American journal of tropical medicine and hygiene , 82(2):360;author reply 361–2.Bryce, J., Victora, C. G., Habicht, J.-P., Vaughan, J. P., and Black, R. E. (2004). The multi-country evaluation of the integrated management of childhood illness strategy: lessons for theevaluation of public health interventions.
American journal of public health , 94(3):406–15.Byass, P., Sankoh, O., Tollman, S. M., H¨ogberg, U., and Wall, S. (2011). Lessons from historyfor designing and validating epidemiological surveillance in uncounted populations.
PloS one ,6(8):e22897.Chipeta, M. G., Terlouw, D. J., Phiri, K. S., and Diggle, P. J. (2016). Adaptive geostatisticaldesign and analysis for prevalence surveys.
Spatial Statistics , 15:70–84.Clark, S. J., G´omez-Oliv´e, F. X., Houle, B., Thorogood, M., Klipstein-Grobusch, K., Angotti, N.,20abudula, C., Williams, J., Menken, J., and Tollman, S. (2015). Cardiometabolic disease riskand hiv status in rural south africa: establishing a baseline.
BMC public health , 15(1):135.Clark, S. J., Kahn, K., Houle, B., Arteche, A., Collinson, M. A., Tollman, S. M., and Stein, A.(2013). Young children’s probability of dying before and after their mother’s death: a rural southafrican population-based surveillance study.
PLoS Med , 10(3):e1001409.Commission on Population and Development (2016). Strengthening the demographic evidence basefor the post-2015 development agenda: Report of the secretary-general. Technical report, UnitedNations.Data Revolution Group: The UN Secretary General’s Independent Expert Advisory Group on aData Revolution for Sustainable Development (2014). A world that counts: Mobilising the datarevolution for sustainable development. Technical report.Denison, D. and Holmes, C. (2001). Bayesian partitioning for estimating disease risk.
Biometrics ,57(1):143–149.Diggle, P. J., Menezes, R., and Su, T.-L. (2010). Geostatistical inference under preferential sam-pling.
Journal of the Royal Statistical Society: Series C (Applied Statistics) , 59:191–232.Diggle, P. J., Tawn, J. A., and Moyeed, R. A. (1998). Model-based geostatistics (with discussion).
Applied Statistics , 47:299–350.Gething, P. W., Patil, A. P., Smith, D. L., Guerra, C. A., Elyazar, I. R., Johnston, G. L., Tatem,A. J., and Hay, S. I. (2011). A new world malaria map: Plasmodium falciparum endemicity in2010.
Malaria journal , 10(1):378.G´omez-Oliv´e, F. X., Angotti, N., Houle, B., Klipstein-Grobusch, K., Kabudula, C., Menken, J.,Williams, J., Tollman, S., and Clark, S. J. (2013). Prevalence of hiv among those 15 and olderin rural south africa.
AIDS care , 25(9):1122–1128.Hill, K., Lopez, A. D., Shibuya, K., Jha, P., AbouZahr, C., Anderson, R. N., Bawah, A. a., Betr´an,A. P., Binka, F., Bundhamcharoen, K., Castro, R., Cleland, J., Coullare, F., Evans, T., CarrascoFigueroa, X., George, C. K., Gollogly, L., Gonzalez, R., Grzebien, D. R., Huang, Z., Hull, T. H.,Inoue, M., Jakob, R., Jiang, Y., Laurenti, R., Li, X., Lievesley, D., Fat, D. M., Macfarlane,S., Mahapatra, P., Merialdi, M., Mikkelsen, L., Nien, J. K., Notzon, F. C., Rao, C., Rao, K.,Sankoh, O., Setel, P. W., Soleman, N., Stout, S., Szreter, S., Tangcharoensathien, V., van derMaas, P. J., Wu, F., Yang, G., Zhang, S., and Zhou, M. (2007). Interim measures for meetingneeds for health sector data: births, deaths, and causes of death.
Lancet , 370(9600):1726–35.Horton, R. (2007). Counting for health.
Lancet , 370(9598):1526.Houle, B., Clark, S. J., G´omez-Oliv´e, F. X., Kahn, K., and Tollman, S. M. (2014). The unfoldingcounter-transition in rural south africa: mortality and cause of death, 1994–2009.
PLoS One ,9(6):e100420.Houle, B., Stein, A., Kahn, K., Madhavan, S., Collinson, M., Tollman, S. M., and Clark, S. J.(2013). Household context and child mortality in rural south africa: the effects of birth spacing,shared mortality, household composition and socio-economic status.
International journal ofepidemiology , 42(5):1444–1454.Jha, P. (2012). Counting the dead is one of the world’s best investments to reduce prematuremortality.
Hypothesis , 10(1). 21ha, P., Gajalakshmi, V., Gupta, P. C., Kumar, R., Mony, P., Dhingra, N., and Peto, R. (2006).Prospective study of one million deaths in India: rationale, design, and validation results.
PLoSmedicine , 3(2):e18.Kabaghe, A. N., Chipeta, M. G., McCann, R. S., Phiri, K. S., Van Vugt, M., Takken, W., Diggle,P., and Terlouw, A. D. (2017). Adaptive geostatistical sampling enables efficient identification ofmalaria hotspots in repeated cross-sectional surveys in rural malawi.
PLoS One , 12(2):e0172266.Kabudula, C. W., Houle, B., Collinson, M. A., Kahn, K., Tollman, S., and Clark, S. (2016). Assess-ing changes in household socioeconomic status in rural south africa, 2001–2013: a distributionalanalysis using household asset indicators.
Social Indicators Research , pages 1–27.Kahn, K., Collinson, M., G´omez-Oliv´e, F., Mokoena, O., Twine, R., Mee, P., Afolabi, S., Clark,B., Kabudula, C., Khosa, A., et al. (2012). Profile: Agincourt health and socio-demographicsurveillance system.
International Journal of Epidemiology , 41(4):988–1001.Kahn, K., Tollman, S., Collinson, M., Clark, S., Twine, R., Clark, B., Shabangu, M., G´omez-Oliv´e,F., Mokoena, O., and Garenne, M. (2007). Research into health, population and social transitionsin rural south africa: Data and methods of the agincourt health and demographic surveillancesystem1.
Scandinavian Journal of Public Health , 35(69 suppl):8–20.Lanjouw, P. and Ivaschenko, O. (2010). A new approach to producing geographic profiles of hivprevalence. Technical Report 5207, World Bank - Policy Research Working Paper Series.Lopez, A. D., Lozano, R., Murray, C. J., and Shibuya, K. (2011). Verbal autopsy: innovations, ap-plications, opportunities - improving cause of death measurement (article collection).
PopulationHealth Metrics , 9.Mahapatra, P., Shibuya, K., Lopez, A. D., Coullare, F., Notzon, F. C., Rao, C., and Szreter, S.(2007). Civil registration systems and vital statistics: successes and missed opportunities.
TheLancet , 370(9599):1653–1663.Mathers, C. D., Boerma, T., and Ma Fat, D. (2009). Global and regional causes of death.
Britishmedical bulletin , 92:7–32.Mathers, C. D., Fat, D. M., Inoue, M., Rao, C., and Lopez, A. D. (2005). Counting the dead andwhat they died from: an assessment of the global status of cause of death data.
Bulletin of theWorld Health Organization , 83(3):171–7.Measure DHS (2012). Demographic and Health Surveys. .MEASURE Evaluation (2012). SAVVY: Sample Vital Registration with Verbal Autopsy. .Mikkelsen, L., Phillips, D. E., AbouZahr, C., Setel, P. W., De Savigny, D., Lozano, R., and Lopez,A. D. (2015). A global assessment of civil registration and vital statistics systems: monitoringdata quality and progress.
The Lancet , 386(10001):1395–1406.Naghavi, M., Wang, H., Lozano, R., Davis, A., Liang, X., Zhou, M., Vollset, S. E., Ozgoren, A. A.,Abdalla, S., Abd-Allah, F., et al. (2015). Global, regional, and national age-sex specific all-causeand cause-specific mortality for 240 causes of death, 1990-2013: a systematic analysis for theglobal burden of disease study 2013.
Lancet , 385(9963):117–171.22ffice of the Registrar General & Census Commissioner, India (2012). India’s Sample Registra-tion System. http://censusindia.gov.in/Vital_Statistics/SRS/Sample_Registration_System.aspx .Phillips, D. E., AbouZahr, C., Lopez, A. D., Mikkelsen, L., De Savigny, D., Lozano, R., Wilmoth,J., and Setel, P. W. (2015). Are well functioning civil registration and vital statistics systemsassociated with better health outcomes?
The Lancet , 386(10001):1386–1394.Rowe, A. K. (2009). Potential of integrated continuous surveys and quality management to supportmonitoring, evaluation, and the scale-up of health interventions in developing countries.
TheAmerican journal of tropical medicine and hygiene , 80(6):971–9.Rudan, I., Lawn, J., Cousens, S., Rowe, A. K., Boschi-Pinto, C., Tomaskovi´c, L., Mendoza, W.,Lanata, C. F., Roca-Feltrer, A., Carneiro, I., Schellenberg, J. a., Polasek, O., Weber, M., Bryce,J., Morris, S. S., Black, R. E., and Campbell, H. (2000). Gaps in policy-relevant information onburden of disease in children: a systematic review.
Lancet , 365(9476):2031–40.Rue, H., Martino, S., and Chopin, N. (2009). Approximate bayesian inference for latent gaussianmodels by using integrated nested laplace approximations.
Journal of the royal statistical society:Series b (statistical methodology) , 71(2):319–392.Setel, P. W., Macfarlane, S. B., Szreter, S., Mikkelsen, L., Jha, P., Stout, S., and AbouZahr,C. (2007). A scandal of invisibility: making everyone count by counting everyone.
Lancet ,370(9598):1569–77.Thompson, S. K. and Seber, G. A. (1996).
Adaptive sampling . Wiley.UNICEF - Statistics and Monitoring (2012). Multiple Indicator Cluster Surveys (MICS). .United Nations (2014a).
Data Revolution for Sustainable Development . .United Nations (2014b). Sustainable Development Goals . http://sustainabledevelopment.un.org/owg.html .United Nations (2016). Resolution 2016/1: Strengthening the demographic evidence base for the2030 agenda for sustainable development. http://undocs.org/E/CN.9/2016/1 .United Nations. Statistical Division (2014). Principles and Recommendations for a Vital StatisticsSystem . United Nations Department of Economic and Social Affairs, New York, 3 edition.Victora, C. G., Black, R. E., Boerma, J. T., and Bryce, J. (2011). Measuring impact in the Millen-nium Development Goal era and beyond: a new approach to large-scale effectiveness evaluations.
Lancet , 377(9759):85–95.Wakefield, J., Simpson, D., and Godwin, J. (2016). Comment: Getting into space with a weightproblem. discussion of, “model-based geostatistics for prevalence mapping in low-resource set-tings”, by P. J. Diggle and E. Giorgi.
Journal of the American Statistical Association , 111:1111–1119.World Health Organization (2013a). Strengthening civil registration and vital statistics for births,deaths and causes of death: resource kit. Technical report, World Health Organization.23orld Health Organization (2013b). Strengthening civil registration and vital statistics systemsthrough innovative approaches in the health sector: Guiding principles and good practices. Tech-nical report, World Health Organization.World Health Organization (2014). Improving mortality statistics through civil registration andvital statistics systems: Strategies for country and partner support. Technical report, WorldHealth Organization.Ye, Y., Wamukoya, M., Ezeh, A., Emina, J., and Sankoh, O. (2012). Health and demographicsurveillance systems: a step towards full civil registration and vital statistics system in sub-Sahara Africa?
BMC public health , 12(1):741.24
Appendix
A.1 Optimum allocation sampling strategy details
Suppose we have stratum, indexed by i = 1 , . . . , I . In our case the strata are areas. Let N i be thepopulation of area i and N = (cid:80) i N i the total population in the study region.Let Y ik = 0 / k in area i died, k = 1 , . . . , N i , i = 1 , . . . , I .Then we are interested in T = (cid:80) i (cid:80) k Y ik , the total number of deaths. The fraction of deaths is y = (cid:98) p = T /N .Let q i = N i /N and S i be the standard deviation of the response in stratum i where S i = N i N i − p i (1 − p i ) ≈ p i (1 − p i ) , which is estimated by s i = n i n i − (cid:98) p i (1 − (cid:98) p i ) ≈ (cid:98) p i (1 − (cid:98) p i ) . If we use the usual estimator of (cid:98) p i = (cid:80) n i k =1 y ik /n i then the variance is var ( y ) = I (cid:88) i =1 q i (1 − f i ) S i n i = I (cid:88) i =1 q i (1 − f i ) N i N i − p i (1 − p i ) n i , where f i = n i /N i , which leads to var ( (cid:98) T ) = N I (cid:88) i =1 q i (1 − f i ) S i n i = N I (cid:88) i =1 q i (1 − f i ) p i (1 − p i ) n i − . Substituting in (cid:98) p i gives the estimated variances.We wish to choose n i , the number of samples to take in area i .Then the optimum allocation, in the sense of minimizing var ( y ) (which is the same as minimizingthe variance of T ) is Neyman allocation in which n i = n q i S i (cid:80) i q i S i . (5)Note: we really should be minimizing MSE as our estimators are biased (since they are randomeffects models with shrinkage).In our setting, we have an estimate of p i and so we can use this in (5) which becomes n i ≈ n × q i (cid:112)(cid:98) p i (1 − (cid:98) p i ) (cid:80) i (cid:48) q i (cid:48) (cid:112)(cid:98) p i (cid:48) (1 − (cid:98) p i (cid:48) ) . . (6)We do not include the age-gender groups j in our sampling strata, but our model produces estimates (cid:98) p ij so we estimate (cid:98) p i via (cid:98) p i = J (cid:88) j =1 N ij N i (cid:98) p ij , to use in (6). 25 .2 Village-level characteristics for the current and historic cohorts Tables A.1 and A.2 display the village characteristics for both the current-day and historical cohorts.The current-day cohort is the fixed population from which we draw repeated samples, while thehistorical cohort is used by the
Hyak and optimum sampling schemes to obtain estimated village-level probabilities of death. In our simulation, we used villages 4, 7 and 8 as the HDSS sites.
Table A.1:
Village characteristics for current-day cohort. This cohort represents our fixed popu-lation from which we draw repeated samples.Village Number of Households Number of Children x x able A.2: Village characteristics for historical cohort. The HDSS villages are 4, 7 and 8.Village Number of Households Number of Children x x A.3 Additional simulation results
Tables A.3, A.4, and A.5 summarize the results of the simulation study for n = 3 , , n = 2 , n = 1 , able A.3: Deaths, Bias, Variance, MSE for cluster sampling, stratified sampling,
Hyak andoptimum sampling for n = 3 , S = 100 simulations. Therewere 11,299 deaths in the simulated population from which samples were taken.‘Cluster’ is shorthand for Two-stage Cluster Sample ; ‘
Hyak ’ for
HDSS with In-formative Sampling ; ‘Strata/Covariates’ for
Logistic Regression Covariate Model and ‘Strata/Covariates/Space’ for
Logistic Regression Random Effects CovariateModel . It is not possible to fit the spatial model (IV) to the two-stage clustersampling scheme since there are data from 5 villages only.Design Model Deaths Bias Variance ( × ) MSE ( × )Cluster I. Na¨ıve 342 1,072 192 1,342II. Strata 342 878 207 977III. Strata/Covariates 342 644 775 1,190IV. Strata/Covariates/Space 342 — — —Stratified I. Na¨ıve 344 1,066 9 1,145II. Strata 344 871 26 785III. Strata/Covariates 344 660 25 460IV. Strata/Covariates/Space 344 225 99 150Hyak I. Na¨ıve 409 1,181 8 1,402II. Strata 409 982 25 988III. Strata/Covariates 409 640 22 431IV. Strata/Covariates/Space 409 188 92 128Optimum I. Na¨ıve 356 1,079 7 1,171II. Strata 356 885 23 806III. Strata/Covariates 356 642 23 436IV. Strata/Covariates/Space 356 194 85 12328 able A.4: Deaths, Bias, Variance, MSE for cluster sampling, stratified sampling,
Hyak andoptimum sampling for n = 2 , S = 100 simulations. Therewere 11,299 deaths in the simulated population from which samples were taken.‘Cluster’ is shorthand for Two-stage Cluster Sample ; ‘
Hyak ’ for
HDSS with In-formative Sampling ; ‘Strata/Covariates’ for
Logistic Regression Covariate Model and ‘Strata/Covariates/Space’ for
Logistic Regression Random Effects CovariateModel . It is not possible to fit the spatial model (IV) to the two-stage clustersampling scheme since there are data from 5 villages only.Design Model Deaths Bias Variance ( × ) MSE ( × )Cluster I. Na¨ıve 250 1,075 170 1,326II. Strata 250 881 190 966III. Strata/Covariates 250 659 382 816IV. Strata/Covariates/Space 250 — — —Stratified I. Na¨ıve 256 1,075 11 1,166II. Strata 256 879 30 802III. Strata/Covariates 256 664 27 468IV. Strata/Covariates/Space 256 248 123 185Hyak I. Na¨ıve 302 1,193 15 1,439II. Strata 302 992 41 1,025III. Strata/Covariates 302 646 30 448IV. Strata/Covariates/Space 302 209 109 152Optimum I. Na¨ıve 264 1,090 10 1,198II. Strata 264 893 31 829III. Strata/Covariates 264 646 29 446IV. Strata/Covariates/Space 264 223 109 15929 able A.5: Deaths, Bias, Variance, MSE for cluster sampling, stratified sampling,
Hyak andoptimum sampling for n = 1 , S = 100 simulations. Therewere 11,299 deaths in the simulated population from which samples were taken.‘Cluster’ is shorthand for Two-stage Cluster Sample ; ‘
Hyak ’ for
HDSS with In-formative Sampling ; ‘Strata/Covariates’ for
Logistic Regression Covariate Model and ‘Strata/Covariates/Space’ for
Logistic Regression Random Effects CovariateModel . It is not possible to fit the spatial model (IV) to the two-stage clustersampling scheme since there are data from 5 villages only.Design Model Deaths Bias Variance ( × ) MSE ( × )Cluster I. Na¨ıve 113 1,079 193 1,358II. Strata 113 886 241 1,025III. Strata/Covariates 113 662 1,252 1,690IV. Strata/Covariates/Space 113 — — —Stratified I. Na¨ıve 119 1,088 23 1,205II. Strata 119 895 62 863III. Strata/Covariates 119 662 60 499IV. Strata/Covariates/Space 119 325 196 301Hyak I. Na¨ıve 138 1,193 24 1,447II. Strata 138 1,001 70 1,071III. Strata/Covariates 138 655 61 491IV. Strata/Covariates/Space 138 309 175 271Optimum I. Na¨ıve 122 1,100 27 1,238II. Strata 122 902 78 891III. Strata/Covariates 122 658 68 500IV. Strata/Covariates/Space 122 306 203 29730igures A.1-A.3 display the distributions of the estimated probability of dying produced by eachmodel (models I, III & IV – Na¨ıve, Covariates and
Covariates & Space ) under the
Hyak samplingstrategy for n = 3 , , n = 2 ,
600 and n = 1 , . . . . . . P ( dea t h ) Young Females Young Males Old Females Old Males l l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll
Model IModel IIIModel IVTruth
Figure A.1:
The distributions of the estimated probability of dying from models I, III and IVunder the
Hyak sampling strategy for n = 3 , . . . . . P ( dea t h ) Young Females Young Males Old Females Old Males l l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll
Model IModel IIIModel IVTruth
Figure A.2:
The distributions of the estimated probability of dying from models I, III and IVunder the
Hyak sampling strategy for n = 2 , . . . . . . . P ( dea t h ) Young Females Young Males Old Females Old Males l l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll l l ll
Model IModel IIIModel IVTruth
Figure A.3:
The distributions of the estimated probability of dying from models I, III and IVunder the
Hyak sampling strategy for n = 1 , Hyak sampling scheme for n = 3 , , n = 2 ,
600 and n = 1 ,
20 40 60 80 100 140
Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78910 111213 1415 1617 181920100 200 300 400 500
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.4:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe
Hyak sampling scheme for n = 3 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400 500
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.5:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe
Hyak sampling scheme for n = 2 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 14151617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 14151617 181920100 200 300 400 500
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 14151617 181920
Figure A.6:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe
Hyak sampling scheme for n = 1 , n = 5 , , n = 3 , , n = 2 ,
600 and n = 1 ,
20 40 60 80 100 140
Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 lll
Model IModel IIModel III 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920
Figure A.7:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe two-stage cluster sampling scheme for n = 5 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 1819 20 lll
Model IModel IIModel III 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920
Figure A.8:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe two-stage cluster sampling scheme for n = 3 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 lll
Model IModel IIModel III 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920
Figure A.9:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe two-stage cluster sampling scheme for n = 2 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 lll
Model IModel IIModel III 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920
Figure A.10:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe two-stage cluster sampling scheme for n = 1 , n = 5 , , n = 3 , , n = 2 ,
600 and n = 1 ,
20 40 60 80 100 140
Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.11:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe simple random sampling scheme for n = 5 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.12:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe simple random sampling scheme for n = 3 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.13:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe simple random sampling scheme for n = 2 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10 111213 14151617 181920 12 3 456 78910 111213 14151617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 12 3 456 78 910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.14:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe simple random sampling scheme for n = 1 , n = 5 , , n = 3 , , n = 2 ,
600 and n = 1 ,
20 40 60 80 100 140
Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.15:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 5 , Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.16:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 3 ,,
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 3 ,, Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 910 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.17:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 2 ,,
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 2 ,, Young Females
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920 llll
Model IModel IIModel IIIModel IV 50 100 150 200 250 300
Old Females
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920100 200 300 400
Young Males
Average Estimated Y ij T r u t h
12 3 45 6 789 10111213 14151617 18 1920 12 3 456 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 14151617 181920 200 400 600 800
Old Males
Average Estimated Y ij T r u t h
12 3 456 789 10111213 1415161718 1920 12 3 45 6 789 10111213 14151617 18 1920 12 3 456 78 9 10 111213 14151617 181920 12 3 456 78 910 111213 1415 1617 181920
Figure A.18:
The average village- and strata-specific estimates for the (unobserved) populationcounts of death plotted against the true values across each of the four models underthe optimum sampling scheme for n = 1 ,,