Bayesian Latent Pattern Mixture Models for Handling Attrition in Panel Studies With Refreshment Samples
SSubmitted to the Annals of Applied Statistics
BAYESIAN LATENT PATTERN MIXTURE MODELS FORHANDLING ATTRITION IN PANEL STUDIES WITHREFRESHMENT SAMPLES
By Yajuan Si ∗ , Jerome P. Reiter † and D. Sunshine Hillygus † University of Wisconsin-Madison ∗ and Duke University † Many panel studies collect refreshment samples—new, randomlysampled respondents who complete the questionnaire at the sametime as a subsequent wave of the panel. With appropriate modeling,these samples can be leveraged to correct inferences for biases causedby non-ignorable attrition. We present such a model when the panelincludes many categorical survey variables. The model relies on aBayesian latent pattern mixture model, in which an indicator forattrition and the survey variables are modeled jointly via a latentclass model. We allow the multinomial probabilities within classes todepend on the attrition indicator, which offers additional flexibilityover standard applications of latent class models. We present resultsof simulation studies that illustrate the benefits of this flexibility. Weapply the model to correct attrition bias in an analysis of data fromthe 2007-2008 Associated Press/Yahoo News election panel study.
1. Introduction.
Many longitudinal or panel surveys, in which thesame individuals are interviewed repeatedly at different points in time, suf-fer from panel attrition. For example, in the American National ElectionStudy, 47% of respondents who completed the first wave in January 2008failed to complete the follow-up wave in June 2010. Such attrition can resultin biased inferences when the attrition generates non-ignorable missing data;that is, the reasons for attrition depend on values of unobserved variables(e.g., Schluchte, 1982; Brown, 1990; Diggle and Kenward, 1994; Ibrahim,Lipsitz and Chen, 1999; Scharfstein, Rotnitzky and Robins, 1999; Olsen,2005; Behr, Bellgardt and Rendtel, 2005; Bhattacharya, 2008; Hogan andDaniels, 2008).Unfortunately, it is not possible to determine whether the attrition isignorable or non-ignorable, nor the extent to which attrition impacts infer-ences, using the collected data alone. Consequently, analysts have to relyon strong and generally unverifiable assumptions about the attrition pro-cess. Many assume that attrition is a missing at random (MAR) process;for example, MAR assumptions underlie the use of post-stratification to ad-
Keywords and phrases:
Panel attrition, Refreshment sample, Categorical, Dirichlet pro-cess, Multiple imputation, Non-ignorable a r X i v : . [ s t a t . M E ] S e p SI ET AL. just survey weights (e.g., Holt and Smith, 1979; Gelman and Carlin, 2001;Henderson, Hillygus and Tompson, 2013) and off-the-shelf multiple imputa-tion routines to create completed datasets (e.g., Pasek et al., 2009; Honakerand King, 2010). Others allow for specific not missing at random (NMAR)processes, characterizing the attrition with a selection model (Hausmanand Wise, 1979; Brehm, 1993; Kenward, 1998; Scharfstein, Rotnitzky andRobins, 1999) or pattern mixture model (Little, 1993, 1994; Daniels andHogan, 2000; Roy, 2003; Kenward, Molenberghs and Thijs, 2003; Lin, Mc-Culloch and Rosenheck, 2004; Roy and Daniels, 2008).Many panel surveys supplement the original panel with refreshment sam-ples. These are cross-sectional, random samples of new respondents giventhe questionnaire at the same time as a subsequent wave of the panel. Forexample, refreshment samples are included in the National Educational Lon-gitudinal Study of 1988, which followed a nationally representative sampleof 21,500 eighth graders in two year intervals until 2000 and refreshed withcross-sectional samples in 1990 and 1992. Overlapping or rotating panels, inwhich a new study cohort completes their first wave at the same time a pre-vious cohort completes a second or later wave, offer equivalent information.Refreshment samples offer information that can be utilized to correctinferences for non-ignorable panel attrition (Hirano et al., 1998; Bartels,1999; Hirano et al., 2001; Sekhon, 2004; Bhattacharya, 2008; Deng et al.,2013). In particular, analysts can use an additive non-ignorable (AN) model,which comprises a model for the survey variables coupled with a selectionmodel for the attrition process (Hirano et al., 1998). The selection modelmust be additive in the variables observed and missing due to attrition sothat model parameters are identifiable.Specifying the models for the survey variables and the attrition indicatorcan be challenging, even when the data include only a modest number ofvariables. Consider, for example, a multinomial survey outcome modeled asa function of ten categorical predictors. It is difficult to determine whichinteraction terms to include in the model, especially in the presence of miss-ing data due to attrition (Erosheva, Fienberg and Junker, 2002; Vermuntet al., 2008; Si and Reiter, 2013). The model specification task is even morecomplicated when the analyst seeks to model all survey variables jointly, forexample, with a log-linear model or sequence of conditional models (e.g.,specify f ( a ), then f ( b | a ), then f ( c | a, b ), and so on). Joint modeling canbe useful when the survey variables suffer from item nonresponse.Recognizing this, Si, Reiter and Hillygus (2014) propose to use a Dirichletprocess mixture of products of multinomial distributions (Dunson and Xing,2009; Si and Reiter, 2013) to model the survey variables. This offers the ana- AYESIAN LATENT PATTERN MIXTURE MODELS lyst the potential to capture complex dependencies among variables withoutselecting interaction effects, as well as to handle item nonresponse among thesurvey variables. However, for the attrition indicator model, Si, Reiter andHillygus (2014) use probit regression with only main effects for the surveyvariables, eschewing the task of selecting interaction effects. While conve-nient, using a main-effects-only specification makes assumptions about theattrition mechanism that may not be realistic in practice. Furthermore, pro-bit regressions can suffer from the effects of separability and near co-linearityamong predictors (Gelman et al., 2008), which complicates estimation of theAN model.In this article, we present an alternative approach for leveraging refresh-ment samples based on Bayesian latent pattern mixture (BLPM) models. Wefocus on models for categorical variables. The key idea is to use the Dirich-let process mixture of products of multinomial distributions for the surveyvariables and attrition indicator jointly, thus avoiding specification of anexplicit selection model. We note that several other authors (e.g., Muth´en,Jo and Brown, 2003; Roy, 2003; Lin, McCulloch and Rosenheck, 2004) haveproposed using mixture models for handling attrition outside of the con-text of refreshment samples. As we show, the refreshment sample enablesus to allow the multinomial vectors within mixture components to dependon attrition indicators, thereby encoding a flexible imputation engine thatreduces reliance on conditional independence assumptions.We were motivated by attrition in the Associated Press/Yahoo 2008 Elec-tion Panel (APYN) study, a multi-wave longitudinal survey designed to trackthe attitudes and opinions of the American public during the 2008 presiden-tial election campaign. The APYN study was the basis of dozens of newsstories during the campaign and subsequent academic analyses of the elec-tion in the years since. However, the study lost more than one third of theoriginal sample to attrition by the final wave of data collection, calling intoquestion the accuracy of analyses based on the complete cases. The APYNincluded a refreshment sample in the final pre-election wave of data collec-tion, which we leverage via the BLPM model to create attrition-adjusted,multiply imputed datasets. We use the multiply imputed data to examinedynamics of public opinion in the 2008 presidential campaign.The remainder of the article is organized as follows. In Section 2, we in-troduce the APYN data. In Section 3, we describe pattern mixture modelsfor refreshment samples, including conditions under which model parame-ters are data-identified. To our knowledge, this is the first description ofpattern mixture models in this context. In Section 4, we propose and mo-tivate the BLPM model for refreshment sample contexts. In Section 5, we SI ET AL.
Table 1
APYN variables from wave 1 (W1), wave 2 (W2) and refreshment sample (Ref), withrates of item nonresponse. Item nonresponse arises either from refusals to answer thequestion (respondent proceeded to the next question without giving a response) orselection of a “Don’t know enough to say” response. We note that 1,011 of the wave 1participants attrited from the panel by wave 2, which could result in attrition bias.
Item nonresponse counts (%)Variable Levels W1: 2735 W2: 1724 Ref: 464Obama favorability 2 550 (20.1) 95 (5.5) 20 (4.3)Party identification (Dem., Rep., Ind.) 3 13 (0.5) 9 (1.9)Ideology (Lib., Mod., Con.) 3 57 (2.1) 10 (2.2)Age (18–29, 30–44, 45–59, 60+) 4 0 0Education ( ≤ HS, Some coll., Coll.) 3 0 0Race (White, Non-white) 2 0 0Gender 2 0 0Income (Ks) ( <
30, 30–50, 50–75, ≥
75) 4 0 0Married indicator 2 0 0 illustrate properties of the BLPM model with simulation studies. Here, wedemonstrate the benefits of allowing the multinomial vectors within mix-ture components to depend on attrition indicators. In Section 6, we analyzethe American electorate in the 2008 presidential election, using the BLPMmodel to account for attrition in the APYN data. Finally, in Section 7 wesummarize and discuss future research directions.
2. Description of APYN Data.
The APYN study included elevenwaves of data collection and three refreshment samples spanning the 2008primary and general U.S. election season. The survey was sampled from theGfK Knowledge Panel, which is one of the nation’s only online, probability-based respondent pools designed to be statistically representative of the U.S.population. The respondent pool is recruited via a probability-based sam-pling method using published sampling frames that cover 96% of the U.S.population. Sampled non-internet households are provided with a laptop andfree internet service. Individuals in the respondent pool are then invited toparticipate in online surveys, such as the APYN panel survey. Surveys fromthe GfK KnowledgePanel are approved by the Office of Management andBudget for government research and have been used in hundreds of academicpublications spanning diverse disciplines, including health and medicine,psychology, social sciences, public policy, and survey and statistical method-ology. More information about the survey methodology can be found at .Wave 1 of the APYN was fielded on November 2, 2007 and was com-pleted by 2,735 respondents out of 3,548 contacted individuals. After the
AYESIAN LATENT PATTERN MIXTURE MODELS initial wave, these wave 1 respondents were invited to participate in eachfollow-up wave, even if they failed to respond to the previous one. Con-sequently, wave-to-wave attrition rates or completion rates vary across thestudy. Three external refresh cross-sections were also collected: a sample of697 new respondents in January, 576 new respondents in September, and 464new respondents in October. Each of the refreshment samples is a randomand cross-sectional sample of the GfK respondent pool. Our analysis focuseson wave 1 (November 2007) and the ninth wave with a corresponding re-freshment sample (October 2008, the final wave before the election), whichwe label wave 2 for presentational clarity. As shown in Table 1, of those whocompleted wave 1, 1,011 (37%) respondents failed to complete the Octoberwave. In previous research using the APYN data (Pasek et al., 2009; Hender-son and Hillygus, 2011; Iyengar, Sood and Lelkes, 2012; Henderson, Hilly-gus and Tompson, 2013), scholars have mostly relied on post-stratificationweights to correct for potential panel attrition bias, although Pasek et al.(2009) used standard multiple imputation via Amelia II (King et al., 2001).Deng et al. (2013) outline the limitations of such approaches—both assumethat the attrition is MAR.The primary outcome of interest in pre-election polls tends to be evalu-ations of the candidates, as analysts attempt to gauge levels of candidatesupport within the electorate. Which candidate is most likely to win theelection? Who in the electorate supports each side? Because the earliestwaves of the APYN took place before the ballot match-up was known—i.e.,before Obama and McCain had been selected as their party nominees—wefocus on Obama favorability (coded as favorable or not). This variable offersexact comparability in question wording across survey waves and is highlycorrelated with eventual vote choice (the tetrachoric correlation of the itemsin wave 2 is 0.97). In examining Obama favorability, we consider standardcovariates from the voting behavior literature. These include demographicvariables (from “Age” to “Marital status” in Table 1) previously shown to berelated to candidate evaluations and/or panel attrition (Frankel and Hilly-gus, 2013). We also consider two relevant political background variables(“Party identification” and “Ideology” in Table 1) that are typically consid-ered time invariant in the context of a single election cycle (Bartels et al.,2011). Demographic and political profile variables are collected in profile surveys when a pan-elist joins the KnowledgePanel and are updated continually; thus, they have few missingvalues for any one study.
SI ET AL.
Table 2
Structure of panel and refreshment samples. Notation for sample sizes in parentheses.The total number of individuals in both datasets is N = N p + N r . Time-Invariant Wave 1 Wave 2Panel ( N p ) X Y Y , W = 1 ( N cp ) Y =?, W = 0 ( N ip )Refreshment Sample ( N r ) Y =? Y , W =?
3. Additive Pattern Mixture Models for Refreshment Samples.
Before introducing the BLPM model and analyzing the APYN data, wereview the AN model of Hirano et al. (1998) and present a correspondingpattern mixture model formulation. Suppose the data comprise a two wavepanel of N p individuals with a refreshment sample of N r new individualsin the second wave. For all N = N p + N r individuals, the data include q time-invariant variables X = ( X , . . . , X q ), such as demographic or framevariables. Let Y = ( Y , . . . , Y q ) be the q survey variables of interestcollected in wave 1. Let Y = ( Y , . . . , Y q ) be the corresponding q surveyvariables collected in wave 2. Here, we assume that Y and Y comprise thesame variables collected at different waves, although this is not necessary.Among the N p individuals, N cp < N p provide at least some data in thesecond wave, and the remaining N ip = N p − N cp individuals drop out ofthe panel. The refreshment sample includes only ( X, Y ); by design, Y aremissing for all the individuals in the refreshment sample. In this section, wepresume that X , Y in the panel, and Y in the refreshment sample are notsubject to nonresponse, although we relax this when analyzing the APYN.For each individual i = 1 , . . . , N , let W i = 1 if individual i would remainin wave 2 if included in wave 1, and let W i = 0 if individual i would drop outof wave 2 if included in wave 1. Here, W i is an indicator of panel attritionconditional on participation in wave 1; it is not an indicator of item or unitnonresponse among individuals in the refreshment sample. We note that W i is fully observed for all individuals in the panel but is missing for theindividuals in the refreshment sample, since individuals in the refreshmentsample are not provided the chance to respond in wave 1. Putting it alltogether, the concatenated data have the structure illustrated in Table 2.The AN model requires a joint model for ( Y , Y | X ) and a selectionmodel for ( W | X, Y , Y ), that is,( Y , Y ) | X ∼ f ( Y , Y | X, Θ) W | Y , Y , X ∼ f ( W | X, Y , Y , Θ) , (3.1)where Θ generically represents the parameters for both models. To enableidentification, (3.1) must exclude interactions between Y and Y . AYESIAN LATENT PATTERN MIXTURE MODELS As an example of an AN model, suppose Y and Y are binary variablesand X is empty, as in Hirano et al. (1998). One specification of the additivenon-ignorable selection model is Y i ∼ Bern( π ) , logit( π ) = α (3.2) Y i | Y i ∼ Bern( π i ) , logit( π i ) = β + β Y i (3.3) W i | Y i , Y i ∼ Bern( π iW ) , logit( π iW ) = τ + τ Y i + τ Y i . (3.4)For a pattern mixture model representation, we require a model for ( W | X ) and for ( Y , Y | X, W ), that is, W | X ∼ f ( W | X, Θ)( Y , Y ) | X, W ∼ f ( Y , Y | X, W, Θ) . Using the basic example, one specification of the additive pattern mixture(APM) model is W i ∼ Bern( π W ) , logit( π W ) = ω Y i | W i ∼ Bern( π i ) , logit( π i ) = δ + δ W i Y i | Y i , W i ∼ Bern( π i ) , logit( π i ) = γ + γ W i + γ Y i , (3.5)which contains as many free parameters as in (3.2) – (3.4) and thus is data-identified. To enable identification, we exclude interactions between Y and W in (3.5). We note that both the AN and APM models can include inter-actions with X and readily extend to other data types.
4. Bayesian Latent Pattern Mixture Models.
We now develop anAPM model for categorical data with q = q + q + q variables. Let Z =( X, Y , Y ) = ( Z , . . . , Z q ) comprise all potentially collected variables. Weorder variables so that j = 1 , . . . , q for X variables, j = q + 1 , . . . , q + q for Y variables, and j = q + q + 1 , . . . , q for Y variables. For i = 1 , . . . , N and j = 1 , . . . , q , without loss of generality let Z ij ∈ { , . . . , d j } denote thelevel of variable j for unit i , where d j ≥ j .We specify the pattern mixture model as f ( W ) f ( Z | W ), including X in the joint distribution of the survey variables. This facilitates imputa-tion of (ignorable) item nonresponse in X , and allows us to take advantageof computationally efficient latent class representations of categorical data.Specifically, we adapt the truncated Dirichlet process mixture of products ofmultinomial distributions (DPMPM) developed by Dunson and Xing (2009),used previously for multiple imputation of missing cross-sectional data by Si SI ET AL. and Reiter (2013). The DPMPM assumes that each individual is a memberof a latent class, and that within each class the variables follow independentmultinomial distributions. Averaging the multinomial probabilities over thelatent classes induces global dependence among the variables.For i = 1 , . . . , N , let s i ∈ { , . . . , K } indicate the class of individual i , and let π h = Pr( s i = h ) where h = 1 , . . . , K . We assume that π =( π , . . . , π K ) is the same for all individuals. For j = q + 1 , . . . , q + q , let ψ hjz = Pr( Z ij = z | s i = h ) be the probability of Z ij = z for any value z giventhat individual i is in class h . For j = 1 , . . . , q and j = q + q + 1 , . . . , q , let ψ (1) hjz = Pr( Z ij = z | W i = 1 , s i = h ) and ψ (0) hjz = Pr( Z ij = z | W i = 0 , s i = h )be the probabilities of Z ij = z for any value z given that individual i is inclass h for each value of W i . The complete-data likelihood for ( s i , W i , Z i ) inthe BLPM is as follows. s i | π ∼ Multinomial( π , . . . , π K )(4.1) W i | s i ∼ Bernoulli( ρ s i ) . (4.2)When j ∈ { q +1 , . . . , q + q } , we have(4.3) Z ij | s i ∼ Multinomial( { , . . . , d j } , ψ s i j , . . . , ψ s i jd j ) . When j ∈ { , . . . , q , q + q + 1 , . . . , q } , we have Z ij | s i , W i = 1 ∼ Multinomial( { , . . . , d j } , ψ (1) s i j , . . . , ψ (1) s i jd j )(4.4) Z ij | s i , W i = 0 ∼ Multinomial( { , . . . , d j } , ψ (0) s i j , . . . , ψ (0) s i jd j ) . (4.5)The BLPM model is a mixture of pattern mixture models, where f ( Z i , W i ) = K (cid:88) h =1 Pr( s i = h ) f ( W i | s i = h ) f ( Z i | W i , s i = h ) . As in the DPMPM, we assume that ( Z q +1 , . . . , Z q + q ), that is, Y , fol-low independent, class-specific multinomial distributions that are also inde-pendent of W (and X, Y ). However, we depart from the DPMPM by let-ting ( Z , . . . , Z q , Z q + q +1 , . . . , Z q ) follow class-specific, independent multi-nomial distributions that depend on W . Relaxing the conditional indepen-dence between Y and W (that is, Y is independent of W within any latentclass) is possible because of information offered by the refreshment sam-ple. We force Y and W to be independent within latent classes to enableidentification, following the strategy outlined in Section 3. We allow X todepend on W within classes to offer additional flexibility for settings where AYESIAN LATENT PATTERN MIXTURE MODELS the distributions of X are substantially different across attriters and non-attriters. When this is not the case—the distributions of X are observed forboth W = 1 and W = 0—one can specify the model so that X does notdepend on W within classes, thereby reducing the number of parameters toestimate.For the prior distribution on π , we use the stick-breaking representationof a Dirichlet process prior distribution (Sethuraman, 1994), truncating atlarge K for computational convenience. In particular, we have π h = V h (cid:89) g 5. Simulation Studies. In this section, we present results of simula-tion studies that illustrate the potential of the BLPM model to accountfor non-ignorable attrition. We use two data generation mechanisms: one inwhich Y and W are not independent within classes, and one in which theyare independent within classes. We compare the performance of the BLPMmodel to the usual DPMPM, a model that assumes Y and W are condition-ally independent. In each scenario, we set N p = 2 , 000 and N r = 1 , q = q = 5 binary variables; for simplicity, we do not includeany X variables. Table 3 displays the values of π and the ψ parametersfor each scenario. These designs result in non-trivial dependence structures;for example, we ran Pearson’s chi-square tests in the true datasets and re-jected independence at the 0 . 05 significance level for 29 out of the 45 pairedcombinations among the 10 variables.In each replication of the simulation, we generate a dataset with valuesof ( Z, W ) for all N = 3 , 000 records; we call this the true data. We deletethe values of Y for all records in the panel with W i = 0 and the values of AYESIAN LATENT PATTERN MIXTURE MODELS Table 3 Latent class and marginal probabilities for simulations. The first five ψ parameterscorrespond to Y j variables, and the last five ψ parameters correspond to Y j variables.The columns labeled “marginal” are the weighted averages of ψ over the latent classes. Y and W not Cond. Ind. Y and W are Cond. Ind.Parameter h = 1 h = 2 h = 3 Marginal h = 1 h = 2 h = 3 Marginal π ρ h ψ h, , ψ h, , ψ h, , ψ h, , ψ h, , ψ (0) h, , , ψ (1) h, , ψ (0) h, , , ψ (1) h, , ψ (0) h, , , ψ (1) h, , ψ (0) h, , , ψ (1) h, , ψ (0) h, , , ψ (1) h, , ( Y , W ) for all N r records in the refreshment sample. The resulting datasethas the structure in Table 2 without X . We fit the BLPM and DPMPMmodels using the Gibbs sampler, imputing Y in the panel when W = 0and ( Y , W ) in the refreshment sample in each MCMC iteration. For eachscenario, we run 100 independent replications of the simulation.To evaluate the potential of the BLPM and DPMPM models to correctfor attrition, as well as to compare them with each other, we focus primarilyon the completed data estimates of Pr( Y = 1) in the panel. Let superscript r = 1 , . . . , 100 index replications of the simulation, and let superscript t =1 , . . . , T index MCMC iterations, where T is the number of MCMC iterationsused in computation. For all ( r, t ), and for i = 1 , . . . , N and j = 1 , . . . , q ,let z ( rt ) ij be the value of z ij in replication r and MCMC iteration t . Here, if j > q , z ( rt ) ij is an observed value for all panel cases with W ( r ) i = 1 and isan imputed value for all cases with W ( r ) i = 0. For any variable indexed by j > q , we compute¯ z ( rt ) j = N p (cid:88) i =1 I ( z ( rt ) ij = 1) /N p , ˜ z ( r ) j = Median (¯ z ( r j , . . . , ¯ z ( rT ) j ) . Let ¯ z ( r,true ) j be the value of Pr( Z j = 1) for the panel in the true data asso-ciated with replication r . We then compute DIF j = | (cid:88) r =1 ˜ z ( r ) j / − ¯ z ( r,true ) j | RM SE j = (cid:32) (cid:88) r =1 (˜ z ( r ) j − ¯ z ( r,true ) j ) / (cid:33) . . SI ET AL. l l l l l l l l l l l W Y Y Y Y Y Y Y Y Y Y D I F l l l l l l l l l l l W Y Y Y Y Y Y Y Y Y Y R M SE Fig 1 . Simulation results when the data are generated with Y and W dependent withinclass. Results for DPMPM displayed with triangles and for BLPM with circles. The larger DIF j and RM SE j , the more inaccurate are the completed-dataestimate in the panel. We use only the panel and not the concatenated datato magnify the impact of the models on imputation of the missing datadue to non-ignorable attrition. We also report values of DIF j and RM SE j for the BLPM and DPMPM models for the means of W and Y in therefreshment sample. These are both fully imputed in the models.For each simulation run, we run MCMC chains for both models with K =10 classes—we obtained very similar results with K = 20 and K = 30. Werun the chains for 20,000 and 30,000 iterations for the BLPM and DPMPMmodels, respectively, which exploratory runs suggest as sufficient for thechains to converge. We keep every tenth draw from the final 10,000 drawsof each chain, leaving T = 1 , 000 MCMC draws for inference. To initializethe chains, for all h we set ρ h = N cp /N p ; set all φ parameters equal to 0.5;set α = 1; and, generate all K -1 initial values of V h from (4.7) using α = 1.Figure 1 summarizes the values of DIF j and RM SE j for each quantityfor both the BLPM and DPMPM models for the simulation with conditionaldependence between Y and W within classes. We also computed the DIF j and RM SE j when estimating each Pr( Y j = 1) in the panel with only thecomplete panel cases. For this complete-cases estimator, the average valuesof DIF and RM SE across the 100 runs are shown in Table 4.Compared to the results in Table 4, the BLPM and DPMPM tend tooffer smaller differences in point estimates, correcting the bias in complete-case analysis due to attrition. When estimating Pr( Y j = 1) using the paneldata alone, the BLPM tends to be more accurate than the DPMPM. Therelative performance of the DPMPM worsens as the magnitude of the at- AYESIAN LATENT PATTERN MIXTURE MODELS Table 4 Simulation results for the complete-cases estimator when the data are generated with Y and W dependent within class. Pr( Y j = 1) j = 1 j = 2 j = 3 j = 4 j = 5 DIF j RMSE j trition bias increases, where by attrition bias we mean the difference inthe marginal probabilities of Y j for non-attriters and attriters, that is, (cid:80) h π h ψ (1) hj − (cid:80) h π h ψ (0) hj . We also tend to see better performance when pre-dicting the missing W and Y in the refreshment sample, although thegaps are not as noticeable as those for Y . For all j > q , the simulatedmatched pair standard errors are around 0 . 003 for comparing DIF j forBLPM and DPMPM, and around 0 . 005 for comparing DIF j for BLPMand the complete-case estimator.Figure 2 summarizes the values of DIF j and RM SE j for each quantityfor both the BLPM and DPMPM models for the simulation with conditionalindependence between Y and W within classes. For the complete-cases es-timator, across the 100 runs, the average values of ( DIF , . . . , DIF ) allequal approximately 0 . 016 with associated ( RM SE , . . . , RM SE ) equal toapproximately 0 . Y j = 1) using the panel data alone more accurately than thecomplete-case analysis. When estimating Pr( Y j = 1) using the panel dataalone, the DPMPM tends to be slightly more accurate than the BLPM, butthe differences are modest when compared to those in Figure 1. The differ-ences stem from estimating additional parameters in the BLPM, whereas theDPMPM has the exact specification. For all j > q , the simulated matchedpair standard errors are around 0 . 002 when comparing DIF j for BLPM andDPMPM, and 0 . 002 when comparing DIF j for BLPM and the complete-caseestimator.In summary, these simulation results suggest that both the BLPM andDPMPM can reduce attrition bias compared to using the complete cases.The BLPM is more flexible than the DPMPM in that it can protect againstfailure of the conditional independence assumption for Y and W . However,when conditional independence holds, the BLPM estimates can be similar tothose based on the DPMPM. A sensible default position with decent samplesizes is to use the BLPM, since the data do not inform whether conditionalindependence is appropriate.In our experience, in modest sample sizes both the BLPM and the DPMPMcan suffer, as the latent class models will sacrifice higher-order relationships SI ET AL. l l l l l l l l l l l W Y Y Y Y Y Y Y Y Y Y D I F l l l l l l l l l l l W Y Y Y Y Y Y Y Y Y Y R M SE Fig 2 . Simulation results when the data are generated with Y and W independent withinclass. Results for DPMPM displayed with triangles and for BLPM with circles. among the variables. Thus, it is crucial to check the fit of the models. Wesuggest methods for doing so in the analysis of the APYN data (Section 6). 6. Using the BLPM to Correct for Attrition in the APYN data. We now apply the BLPM model to account for attrition in the APYN data.To begin, we first provide some additional context on the survey designthat is relevant for our imputations and analyses. Throughout, we referto cross-sectional unit nonresponse as non-participation or refusal in thewave when an individual is initially surveyed; attrition happens when anindividual drops out after participating in a previous wave. For example,the refreshment sample is subject to cross-sectional unit nonresponse butnot attrition, as these individuals are only surveyed at wave 2.6.1. Survey weights in the APYN. The APYN data file includes sur-vey weights at each wave. The wave 1 weights are the product of design-based weights and post-stratification adjustments for cross-sectional unitnonresponse at wave 1. These post-stratification adjustments assume thecross-sectional unit nonresponse is missing at random, as is common in theliterature (e.g., Hirano et al., 1998; Bhattacharya, 2008; Das, Toepoel andvan Soest, 2011). The wave 2 weights for the 1724 panel participants includepost-stratification adjustments for attrition in the panel, for cross-sectionalunit nonresponse at wave 1, and for cross-sectional unit nonresponse amongcases in the refreshment sample; the way that weights are reported does notallow us to disentangle these adjustments. Since we use the BLPM model toaccount for non-ignorable attrition, we disregard the wave 2 weights in allanalyses. AYESIAN LATENT PATTERN MIXTURE MODELS The original panel is approximately an equal probability sample, withdeviations due primarily to (i) slight oversampling of African American andHispanic telephone exchanges and (ii) undersampling of areas where theMSN TV service network cannot be used and where there is no access to theinternet. The post-strata in wave 1 are based on gender, race, the age groupsin Table 1, the education groups in Table 1, census region, metropolitanarea, and household internet access. We include most of these variables inthe BLPM model, thereby accounting for important aspects of the designwhen making imputations. The geographic variables and internet access arenot strong predictors of Obama favorability given all the other variables inTable 1. In a logistic regression with Obama favorability in wave 1 as thedependent variable, a drop in deviance test for the models with and withoutcensus region, metropolitan area, and internet access (including all othervariables in X ) results in a p-value of 0.20. Since these variables do notsubstantially improve our ability to predict the missing Obama favorabilityvalues, and are not of substantive interest in our analyses of the Americanelectorate, we exclude them from the imputation model.We use unweighted analyses to illustrate the attrition effects and describethe behavior of the BLPM model (as in Figures 3 and 4 in Section 6.3), andwe use survey weighted analyses when computing finite population quantities(as in Figure 5 in Section 6.3). The survey-weighted estimates account forthe sampling design and cross-sectional unit nonresponse in wave 1 only. Tomake these estimates, we use the wave 1 weights for the 1724 panelists inmultiple imputation inferences (Rubin, 1987).6.2. Generating Completed Datasets. We run the BLPM with K = 30classes using the Gibbs sampler outlined in the online supplement, treat-ing Obama favorability as ( Y , Y ) and all other variables as X . As initialvalues for W in the refreshment sample, we use independent draws froma Bernoulli distribution with probability N cp /N p = 0 . 63. For missing datain ( X, Y , Y )—due to item nonresponse and attrition—and W in the re-freshment sample, we implement the initialization steps of the MCMC asfollows. • For any missing values in X , sample from the marginal distribution of X computed from the observed cases in the combined panel and therefreshment sample. • For any missing values in Y , sample from the observed marginal dis-tribution of Y . We estimated the model with wave 1 data to avoid any issues from non-ignorableattrition. SI ET AL. • For missing values in Y in the refreshment sample, sample from theobserved marginal distribution of Y in the refreshment sample. • For missing values in Y in the panel for cases with W i = 1, samplefrom the observed marginal distribution of Y in the panel. • For missing values in Y in the panel for cases with W i = 0, sample fromindependent Bernoulli distributions with probabilities Pr( Y | W = 0) , obtained by [Pr( Y ) − Pr( Y | W = 1)Pr( W = 1)] / Pr( W = 0) . Here,Pr( Y ) is estimated with the refreshment sample, Pr( Y | W = 1) isestimated with cases with W i = 1 in the panel, and Pr( W = 1) = 0 . α = 1; set each ρ h = N cp /N p ; set each ψ parameter equal to the corresponding marginal prob-ability calculated from the initial completed dataset; and set V h = 0 . h = 1 , . . . , K -1. Each record’s latent class indicator is initialized from a drawof a multinomial distribution with probability π implied by the set of initial { V h } .We run the MCMC for 150,000 iterations, treating the first 100,000 asburn-in and thinning every 50th iteration. The trace plots of each variable’smarginal probability suggest convergence. The posterior mode of the numberof distinct occupied classes is 9, and the maximum is 18. This suggests that K = 30 classes is sufficient. We collect m = 50 completed datasets by keepingevery twentieth draw from the T = 1000 thinned draws. We use only the N p records in the completed panels for multiple imputation inferences.6.3. Results. We begin by comparing the distributions of variables inwave 2 among the N cp non-attriters in the panel and the N r respondents inthe refreshment sample; these are summarized in Table 5. Among the non-attriters, 54.9% favor Obama. In the refreshment sample, however, 61.7%favor Obama. This suggests that people who liked Obama may have droppedout with higher frequency than those who did not. As a sense of the mag-nitude of these differences, the 95% confidence interval limits correspondingto these two percentages are (0.525, 0.573) and (0.572, 0.662), offering evi-dence that the difference may well be systematic. Of note, compared to therefreshment sample, the N cp non-attriters are less likely to be Democratsand to be liberals, more likely to be non-white and to have income below$30,000, and more likely to be below age 45.These differences in the marginal frequencies reflect the effects of attri-tion, as well as differential cross-sectional unit nonresponse in the refresh-ment sample and initial wave. Reassuringly, national cross-sectional pollsin October 2008 from Gallup, Fox News, and other major polling orga-nizations also put Obama favorability ratings close to 62% ( AYESIAN LATENT PATTERN MIXTURE MODELS Table 5 Unweighted percentages of respondents in each category in wave 1 and wave 2 of thepanel (W1 and W2), and in the refreshment sample (Ref). Percentages based onavailable cases only, before imputation of item nonresponse. Variable W1 W2 Ref.Favorable to Obama 0.553 0.549 0.617Democrat 0.327 0.318 0.374Independent 0.369 0.374 0.312Liberal 0.223 0.234 0.289Conservative 0.366 0.370 0.397Age 18–29 0.148 0.135 0.110Age 30–44 0.284 0.284 0.213Age 45–59 0.317 0.320 0.341HS Edu. or less 0.343 0.325 0.323College Edu. 0.298 0.333 0.308Non-white 0.230 0.220 0.177Female 0.548 0.537 0.565Income < K pollingreport.com/obama_fav.htm ), suggesting the respondents in the re-freshment sample faithfully represent Obama’s favorability ratings at thetime. In our analyses, we assume that Obama favorability values missingfor reasons other than attrition, that is, due to cross-sectional item andunit nonresponse, are MAR given the variables in the BLPM model. Previ-ous survey methodology research indicates that missingness mechanisms forattrition and cross-sectional nonresponse are distinct (e.g., Loosveldt andCarton, 1997; Groves and Couper, 1998; Lynn, 2005; Groves, 2006; Smithand Son, 2010; Olson and Witt, 2011), so that one can plausibly considerattrition as potentially non-ignorable even when assuming cross-sectionalunit nonresponse is MAR. See Schifeling et al. (2014) for further discussionof the effects on inferences of non-ignorable cross-sectional unit nonresponsein the initial wave and refreshment sample.Figure 3 displays estimated probabilities for Obama favorability for eachof the subgroups defined by the time-invariant variables. For many sub-groups, the estimates for non-attriters in the panel are noticeably differentfrom those in the refreshment sample. This finding offers an important cor-rection to the prevailing wisdom about the nature of panel attrition in polit-ical surveys. Research had previously concluded that attrition bias impactedoutcomes related to political engagement (e.g., turnout) but not those re-lated to candidate support (e.g., favorability) (Bartels, 1999; Kruse et al., SI ET AL. ll lll l lll ll ll ll lll ll ll l Party: Dem (547, 893)Party: Rep (532,832)Party: Ind (645, 1010)Edu: <= HS (560, 937)Edu: Some Coll. (590, 984)Edu: Coll. (574, 814)Ideology: Lib (404, 611)Ideology: Mod (683, 1123)Ideology: Con (637, 1001)Age: 18−29 (232, 405)Age: 30−44 (490, 777)Age: 45−59 (552, 866)Age: 60+ (450, 687)Inc: <30K (452, 757)Inc: 30−50K (465, 735)Inc: 50−75K (405, 616)Inc: >75K (402, 627)White (1344, 2106)Non−White (380, 629)Male (799, 1236)Female (925, 1499)Married (1089, 1725)Non−Married (635, 1010) l Non−attriters Refreshment Completed Panel Fig 3 . Point estimates and 95% confidence intervals for Obama favorability in varioussubgroups. Results presented for the N cp panel non-attriters, the N r refreshment samples,and the N p panel participants. Inferences based on unweighted analyses of the m = 50 completed datasets, after multiple imputation of missing values via the BLPM model. Thenumbers in parentheses are the corresponding subgroup sizes, the first being the size amongnon-attriters and the second being among the completed panel. We randomly select oneimputed dataset to obtain the sample sizes when the background variables are subject toitem nonresponse. AYESIAN LATENT PATTERN MIXTURE MODELS results suggest that panel attrition may further complicate accurate estima-tion of their political attitudes and preferences.Figure 3 also reveals how the BLPM can correct for attrition bias. Inparticular, for most subgroups, the point estimate for the N p panel par-ticipants is shrunk towards the refreshment sample estimate; that is, theBLPM model corrects the bias due to attrition. The BLPM-corrected in-tervals tend to be wider than those computed with the non-attriters. Thisresults from two sources of variability, namely the estimation of the modelparameters based on a modest-sized refreshment sample and the imputationof the N ip = 1 , 011 values of Y .Figure 4 displays inferences for several smaller subgroups of substantiveinterest. Here, the BLPM’s advantage over AN models is particularly pre-scient, as we are able to fit the BLPM model without having to specify(perhaps arbitrarily) a selection model with interaction effects. The attritionbiases do appear to differ across the groups, suggesting the importance ofusing models that can capture interaction effects. Interestingly, high-incomemales appear not to experience substantial attrition bias, whereas variouslow-income and less educated groups appear to experience sizable underes-timations of Obama favorability. As in Figure 3, for most groups the BLPMgenerally shrinks point estimates towards those in the refreshment sample.Of course, evaluating potential attrition bias is not the end goal of ouranalyses. Rather, having created attrition-adjusted imputations with theBLPM model, we now use the m completed panel datasets to better un-derstand the American electorate during the 2008 campaign. Here, we usesurvey-weighted analysis as follows. For each population percentage of inter-est and in each of the m completed panel datasets, we compute the standardratio estimate of the population percentage and the usual estimated variancebased on the formula for unequal probability sampling with replacement(Lohr, 1999). We obtain estimates with the survey package (Lumley, 2012)in R . We then combine the point and variance estimates using the multipleimputation rules (Rubin, 1987).Accounting for the wave 1 survey weights, the marginal estimate forObama favorability in the last days before Election Day (wave 2) was 0 . . , . SI ET AL. l ll lll l l ll ll Low−inc whites (327, 544)Low−inc nonwhites (125, 213)Less−edu whites (444, 736)Less−edu nonwhites (116, 201)Low−inc females (249, 433)High−inc males (224, 357)Low−inc less−edu (198, 339)Mod Inds (321, 508)Less−edu white Dems (129, 237)Con white males (266, 409)Lib white females (172, 256)Single white females (249, 413) l Non−attriters Refreshment Completed Panel Fig 4 . Point estimates and 95% confidence intervals for Obama favorability in additionalsubgroups. Results presented for the N cp panel non-attriters, the N r refreshment samples,and the N p panel participants. Inferences based on unweighted analyses of the m = 50 completed datasets, after multiple imputation of missing values via the BLPM model. Thenumbers in parentheses are the corresponding subgroup sizes, the first being the size amongnon-attriters and the second being among the completed panel. We randomly select oneimputed dataset to obtain the sample sizes when the background variables are subject toitem nonresponse. favorability levels fall below 0.5.Comparing estimates across waves also suggests that the American elec-torate grew more favorable towards Obama as the campaign unfolded—theaverage marginal favorability in wave 1 is 0 . 569 (0 . , . . AYESIAN LATENT PATTERN MIXTURE MODELS (0 . , . Y and X are conditionallyindependent of W within latent classes. Reassuringly, the conclusions fromthis version of the BLPM are similar to those presented previously.For comparison, we fit two additional models: the DPMPM model de-scribed in Section 5 that does not have Y depend on W , and a MAR im-putation model based on the DPMPM (as in Si and Reiter, 2013) thatdisregards W entirely. The results for both models, reported in Section 3of the online supplement, are similar to each other but different from theBLPM results. These two alternative models generally result in point es-timates quite similar to those from the non-attriters; in other words, theysuggest that panel attrition bias in Obama favorability is ignorable. Thisseems implausible given the differences in Obama favorability seen in thenon-attriters and the refreshment samples.We also fit the semi-parametric AN model of Si, Reiter and Hillygus(2014), which assumes a probit regression for W conditional on ( X, Y , Y )and a DPMPM model for ( X, Y , Y ). Results are reported in Section 4 ofthe online supplement. Both the semi-parametric AN and BLPM modelssuggest that the attrition is non-ignorable. Point estimates for the quanti-ties in Figure 3 and 4 differ slightly; however, the differences are modestrelative to the multiple imputation variances. We prefer the BLPM results,as the model diagnostics of Section 6.4 suggest that the BLPM fits the datamore effectively than the semi-parametric AN model. We further note thatthe semi-parametric AN model is computationally more intensive than theBLPM, as the probit regression for W requires auxiliary data augmentationand Metropolis steps that are not necessary in the BLPM.6.4. Model Diagnostics. To check the fit of the models, we follow theadvice in Deng et al. (2013) and use posterior predictive checks (Meng,1994b; Gelman et al., 2005; He, Zaslavsky and Landrum, 2010; Burgette SI ET AL. ll llll lll ll ll ll lll ll ll l Party: DemParty: RepParty: IndEdu: <= HSEdu: Some Coll.Edu: Coll.Ideology: LiberalIdeology: ModerateIdeology: ConservativeAge: 18−29Age: 30−44Age: 45−59Age: 60+Inc: <30KInc: 30−50KInc: 50−75KInc: >75KWhiteNon−WhiteMaleFemaleMarriedNon−Married l wave 1 wave 2 ll l lll lll llll ll ll l llll l ll l lll lll llll ll ll l llll l Party: DemParty: RepParty: IndEdu: <= HSEdu: Some Coll.Edu: Coll.Ideology: LiberalIdeology: ModerateIdeology: ConservativeAge: 18−29Age: 30−44Age: 45−59Age: 60+Inc: <30KInc: 30−50KInc: 50−75KInc: >75KWhiteNon−WhiteMaleFemaleMarriedNon−Married −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 Fig 5 . Dynamics of Obama favorability ratings between wave 1 and wave 2. Top plotcompares the marginal estimates in wave 1 and wave 2. Bottom plot presents the differencesbetween wave 2 and wave 1. Results based on the N p panel participants after multipleimputation via the BLPM model. Inference based on survey-weighted estimation. AYESIAN LATENT PATTERN MIXTURE MODELS and Reiter, 2010). We use the BLPM model to generate T = 500 data setswith no missing data in ( X, Y , Y , W ), randomly sampling from the T =1000available completed datasets. Let { D (1) , . . . , D ( T ) } be the collection of the T completed datasets. For each D ( t ) , we also use the model to generatenew values of Y for all cases in the panel, including cases with W i = 1, andin the refreshment sample. This can be done after running the MCMC toconvergence as follows. For given draws of parameter values and any itemmissing data in ( X, Y ), sample new values for the observed and imputed Y using the distributions in the online supplement. Let { R (1) , . . . , R ( T ) } bethe collection of the T replicated datasets.We then compare statistics of interest in { R (1) , . . . , R ( T ) } to those in { D (1) , . . . , D ( T ) } . Specifically, suppose that S is some statistic of inter-est, such as a marginal or conditional probability in our context. For t =1 , . . . , T , let S R ( t ) and S D ( t ) be the values of S computed from R ( t ) and D ( t ) ,respectively. We compute the two-sided posterior predictive probability, ppp = 2 T ∗ min T (cid:88) t =1 I ( S R ( t ) − S D ( t ) > , T (cid:88) t =1 I ( S D ( t ) − S R ( t ) > . When the value ppp is small, for example, less than 5%, this suggests thereplicated datasets are systematically different from the observed dataset,with respect to that statistic. When the value of ppp is not small, the im-putation model generates data that look like the completed data for thatstatistic. Recognizing the limitations of posterior predictive probabilities(Bayarri and Berger, 1998), we interpret the resulting ppp values as diag-nostic tools rather than as evidence from hypothesis tests that the model is“correct.”As statistics, we select Pr( Y = 1) in the refreshment sample, Pr( Y =1 | W = 1) in the panel, Pr( Y = 1 , Y = 1 | W = 1) in the panel, andPr( Y = 1 | X, W = 1) in the panel for all conditional probabilities involvedin the subgroup analyses in Figure 3 and 4. This results in 38 quantities ofinterest. A histogram of the 38 values of ppp is displayed in Section 4 in theonline supplement. The analysis does not reveal any serious lack of modelfit as none of the ppp values are below 0.20.We repeat the same model diagnostics on the semi-parametric AN modelof Si, Reiter and Hillygus (2014). Many of posterior predictive probabilitiesare uncomfortably small. We believe the differences in the semi-parametricAN and BLPM models result because the predictor function in the AN modelfor W used by Si, Reiter and Hillygus (2014) includes only main effects,whereas the BLPM model does not a priori enforce a model for attrition. SI ET AL. 7. Concluding Remarks. The proposed Bayesian latent pattern mix-ture model offers a flexible way to leverage the information in refresh-ment samples in categorical datasets, helping to adjust for bias due to non-ignorable attrition. We have used this approach in analyzing the APYNstudy to better understand the preferences of the American electorate dur-ing the 2008 presidential campaign. Our findings suggest that panel attri-tion biased downward estimates of Obama favorability among many sub-groups in the electorate. With a more accurate assessment of voter atti-tudes, we find that Obama had sufficiently high levels of favorability amongkey subgroups—independents and moderates—to suggest that the electionoutcomes were not really in doubt by late October.The BLPM approach has key advantages over existing applications ofadditive non-ignorable models. The BLPM avoids the difficult tasks of spec-ifying a binary regression model for the attrition process. Unlike standardlatent class models, the BLPM fully utilizes the information in the refresh-ment sample by allowing for conditional dependence within latent classesbetween wave 2 variables and the attrition indicator. We note that a widerange of existing surveys have data structure amenable to BLPM modeling,including the General Social Survey, the 2008 American National ElectionStudy, the Survey of Income and Program Participation, and the NationalEducational Longitudinal Study, to name just a few.As with other modeling strategies for refreshment samples, the validityof the BLPM depends on several overarching assumptions. First, the initialwave of the panel and the refreshment sample should be representative ofthe same population of interest. Put another way, the units in the targetpopulation should not change substantially between wave 1 and wave 2, al-though certainly the distributions of the substantive variables can do so.Second, any unit (or item) nonresponse other than that due to attrition ismissing at random. Third, to ensure identifiability, we assume conditionalindependence between wave 1 survey variables and the attrition indicatorwithin classes. When this assumption is unreasonable, the BLPM model—and any additive pattern mixture model—could fail to correct for attritionbias. Unfortunately, the data do not provide information about the plausi-bility of this assumption. Methods for assessing the sensitivity of results toviolations of this assumption, as well as to violations of the two representa-tiveness assumptions, are important areas for research. Acknowledgements. This research was supported by NSF grants SES-10-61241 and SES-11-31897. AYESIAN LATENT PATTERN MIXTURE MODELS SUPPLEMENTARY MATERIAL Supplement A: Bayesian Latent Pattern Mixture Models forHandling Attrition in Panel Studies With Refreshment Samples (doi: COMPLETED LATER BY THE TYPESETTER). The supplementincludes the MCMC algorithms for the BLPM and DPMPM models, ad-ditional analyses of the APYN data using the DPMPM model and semi-parametric AN model, and details of the BLPM model diagnostics. References. Bartels, L. M. (1999). Panel effects in the American national election studies. PoliticalAnalysis Bartels, B. L. , Box-Steffensmeier, J. M. , Smidt, C. D. and Smith, R. M. (2011).The dynamic properties of individual-level party identification in the United States. Electoral Studies Bayarri, M. J. and Berger, J. O. (1998). Quantifying surprise in the data and modelverification. In Bayesian Statistics 6 (J. M. Bernardo, J. O. Berger, A. P. Dawid andA. F. M. Smith, eds.) 53-82. Oxford University Press. Behr, A. , Bellgardt, E. and Rendtel, U. (2005). Extent and determinants of panelattrition in the European Community Household Panel. European Sociological Review Bhattacharya, D. (2008). Inference in panel data models under attrition caused byunobservables. Journal of Econometrics Brehm, J. (1993). The phantom respondents . University of Michigan Press. Brown, C. H. (1990). Protecting against nonrandomly missing data in longitudinal stud-ies. Biometrics Burden, B. C. and Hillygus, D. S. (2009). Polls and elections: Opinion formation,polarization, and presidential reelection. Presidential Studies Quarterly Burgette, L. F. and Reiter, J. P. (2010). Multiple imputation via sequential regressiontrees. American Journal of Epidemiology Daniels, M. J. and Hogan, J. W. (2000). Reparameterizing the pattern mixture modelfor sensitivity analyses under informative dropout. Biometrics Das, M. , Toepoel, V. and van Soest, A. (2011). Nonparametric Tests of Panel Con-ditioning and Attrition Bias in Panel Surveys. Sociological Methods and Research Deng, Y. , Hillygus, D. S. , Reiter, J. P. , Si, Y. and Zheng, S. (2013). Handlingattrition in longitudinal studies: The case for refreshment samples. Statistical Science 238 - 256. Diggle, P. and Kenward, M. G. (1994). Informative drop-out in longitudinal dataanalysis. Journal of the Royal Statistical Society Series C (Applied Statistics) Dunson, D. B. and Xing, C. (2009). Nonparametric Bayes modeling of multivariatecategorical data. Journal of the American Statistical Association Erosheva, E. A. , Fienberg, S. E. and Junker, B. W. (2002). Alternative statisti-cal models and representations for large sparse multi-dimensional contingency tables. Annales de la Facult´e des Sciences de Toulouse Frankel, L. and Hillygus, D. S. (2013). Looking beyond demographics: Panel attritionin the ANES and GSS. Political Analysis SI ET AL. Gelman, A. and Carlin, J. B. (2001). Poststratification and weighting adjustments. In Survey Nonresponse (R. Groves, D. Dillman, J. Eltinge and R. Little, eds.) Wiley. Gelman, A. , Van Mechelen, I. , Verbeke, G. , Heitjan, D. F. and Meulders, M. (2005). Multiple imputation for model checking: Completed-data plots with missingand latent data. Biometrics Gelman, A. , Jakulin, A. , Pittau, M. G. and Su, Y.-S. (2008). A weakly informativedefault prior distribution for logistic and other regression models. The Annals of AppliedStatistics Gilens, M. (2005). Inequality and democratic responsiveness. Public Opinion Quarterly Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly Groves, R. M. and Couper, M. P. (1998). Nonresponse in Household Interview Surveys .Wiley. Hausman, J. A. and Wise, D. A. (1979). Attrition bias in experimental and panel data:the Gary income maintenance experiment. Econometrica He, Y. , Zaslavsky, A. M. and Landrum, M. B. (2010). Multiple imputation in a large-scale complex survey: A guide. Statistical Methods in Medical Research Henderson, M. and Hillygus, D. S. (2011). The dynamics of health care opinion, 2008–2010: Partisanship, self-Interest, and racial resentment. Journal of health politics, policyand law Henderson, M. , Hillygus, D. S. and Tompson, T. (2013). “Sour grapes” or rationalvoting? Voter decision making among thwarted primary voters in 2008. Public OpinionQuarterly forthcoming . Hirano, K. , Imbens, G. W. , Ridder, G. and Rubin, D. B. (1998). Combining paneldata sets with attrition and refreshment samples Technical Report No. 230, NationalBureau of Economic Research. Hirano, K. , Imbens, G. W. , Ridder, G. and Rubin, D. B. (2001). Combining paneldata sets with attrition and refreshment samples. Econometrica Hogan, J. W. and Daniels, M. J. (2008). Missing Data in Longitudinal Studies . BocaRaton: Chapman and Hall. Holt, D. and Smith, T. M. F. (1979). Post stratification. Journal of the Royal StatisticalSociety Series A Honaker, J. and King, G. (2010). What to do about missing values in time-series cross-section data. American Journal of Political Science Ibrahim, J. G. , Lipsitz, S. R. and Chen, M. H. (1999). Missing covariates in generalizedlinear models when the missing data mechanism is non-ignorable. Journal of the RoyalStatistical Society Series B Ishwaran, H. and James, L. F. (2001). Gibbs sampling for stick-breaking priors. Journalof the American Statistical Association Iyengar, S. , Sood, G. and Lelkes, Y. (2012). Affect, not ideology: A social identityperspective on polarization. Public opinion quarterly Kenward, M. G. (1998). Selection models for repeated measurements with non-randomdropout: An illustration of sensitivity. Statistics in Medicine Kenward, M. G. , Molenberghs, G. and Thijs, H. (2003). Pattern-mixture modelswith proper time dependence. Biometrika Kim, J. K. , Brick, J. M. , Fuller, W. A. and Kalton, G. (2006). On the bias of themultiple imputation variance estimator in survey sampling. Journal of Royal StatisticalSociety, Series B King, G. , Honaker, J. , Joseph, A. and Scheve, K. (2001). Analyzing incomplete polit-AYESIAN LATENT PATTERN MIXTURE MODELS ical science data: An alternative algorithm for multiple imputation. American PoliticalScience Review Kruse, Y. , Callegaro, M. , Dennis, J. M. , Subias, S. , Lawrence, M. , DiSogra, C. and Tompson, T. (2009). Panel conditioning and attrition in the AP-Yahoo! NewsElection Panel Study In . Lin, H. , McCulloch, C. E. and Rosenheck, R. A. (2004). Latent pattern mixturemodels for informative intermittent missing data in longitudinal studies. Biometrics Little, R. J. A. (1993). Pattern-mixture models for multivariate incomplete data. Jour-nal of the American Statistical Association Little, R. J. A. (1994). A class of pattern-mixture models for normal incomplete data. Biometrika Lohr, S. L. (1999). Sampling: Design and Analysis . New York: Duxbury Press. Loosveldt, G. and Carton, A. (1997). Evaluation of Nonresponse in the Belgian Elec-tion Panel Study ’91-‘95. In Proceedings of the Section on Survey Research Methods ofthe American Statistical Association . Lumley, T. (2012). Survey: analysis of complex survey samples. R package version 3.28-2. Lynn, P. (2005). A Review of Methodological Research Pertinent to Longitudinal SurveyDesign and Data Collection Technical Report, UK: Institute for Social and EconomicResearch, University of Esses. Meng, X. L. (1994a). Multiple-imputation inferences with uncongenial sources of input(Disc: p558-573). Statistical Science Meng, X. (1994b). Posterior predictive p-values. Annals of Statistics Muth´en, B. , Jo, B. and Brown, C. H. (2003). Principal stratification approach tobroken randomized experiments: A case study of school choice vouchers in New YorkCity [with comment]. Journal of the American Statistical Association Olsen, R. J. (2005). The problem of respondent attrition: Survey methodology is key. Monthly Labor Review Olson, K. and Witt, L. (2011). Are we keeping the people who used to stay? Changes incorrelates of panel survey attrition over time. Social Science Research Pasek, J. , Tahk, A. , Lelkes, Y. , Krosnick, J. A. , Payne, B. K. , Akhtar, O. and Tompson, T. (2009). Determinants of turnout and candidate choice in the 2008 USPresidential Election illuminating the impact of racial prejudice and other considera-tions. Public Opinion Quarterly Roy, J. (2003). Modeling longitudinal data with nonignorable dropouts using a latentdropout class model. Biometrics Roy, J. and Daniels, M. J. (2008). A general class of pattern mixture models for non-ignorable dropout with many possible dropout times. Biometrics Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys . New York: JohnWiley & Sons. Scharfstein, D. O. , Rotnitzky, A. and Robins, J. M. (1999). Adjusting for nonig-norable drop-out using semiparametric nonresponse models. Journal of the AmericanStatistical Association Schifeling, T. , Cheng, C. , Reiter, J. P. and Hillygus, D. S. (2014). Accounting fornonignorable unit nonresponse and attrition in panel studies with refreshment samples. Journal of Survey Statistics and Methodology Submitted . Schluchte, M. D. (1982). Methods for the analysis of informatively censored longitudinaldata. Statistics in Medicine Sekhon, J. (2004). The varying role of voter information across Democratic societies. UC SI ET AL.Berkeley. Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica Si, Y. and Reiter, J. P. (2013). Nonparametric Bayesian multiple imputation for in-complete categorical variables in large-scale assessment surveys. Journal of Educationaland Behavioral Statistics Si, Y. , Reiter, J. P. and Hillygus, D. S. (2014). Semi-parametric selection models forpotentially non-ignorable attrition in panel study with refreshment sample. PoliticalAnalysis Smith, T. W. and Son, J. (2010). An Analysis of Panel Attrition and Panel Changeon the 2006- 2008 General Social Survey Panel Technical Report, GSS MethodologicalReport No. 118. Chicago: NORC. Vermunt, J. K. , Ginkel, J. R. V. , der Ark, L. A. V. and Sijtsma, K. (2008). Mul-tiple imputation of incomplete categorical data using latent class analysis. SociologicalMethodology Department of Biostatistics & Medical InformaticsDepartment of Population Health SciencesUniversity of Wisconsin-MadisonMadison, WI 53726E-mail: [email protected] Department of Statistical ScienceDuke UniversityDurham, NC 27708E-mail: [email protected]