Jeffrey J. Harden

BIC and Alternative Bayesian Information Criteria in the Selection of Structural Equation Models

Selecting between competing structural equation models is a common problem. Often selection is based on the chi-square test statistic or other fit indices. In other areas of statistical research Bayesian information criteria are commonly used, but they are less frequently used with structural equation models compared to other fit indices. This article examines several new and old information criteria (IC) that approximate Bayes factors. We compare these IC measures to common fit indices in a simulation that includes the true and false models. In moderate to large samples, the IC measures outperform the fit indices. In a second simulation we only consider the IC measures and do not include the true model. In moderate to large samples the IC measures favor approximate models that only differ from the true model by having extra parameters. Overall, SPBIC, a new IC measure, performs well relative to the other IC measures.

Persistent Policy Pathways: Inferring Diffusion Networks in the American States

The transmission of ideas, information, and resources forms the core of many issues studied in political science, including collective action, cooperation, and development. While these processes imply dynamic connections among political actors, researchers often cannot observe such interdependence. One example is public policy diffusion, which has long been a focus of multiple subfields. In the American state politics context, diffusion is commonly conceptualized as a dyadic process whereby states adopt policies (in part) because other states have adopted them. This implies a policy diffusion network connecting the states. Using a dataset of 187 policies, we introduce and apply an algorithm that infers this network from persistent diffusion patterns. The results contribute to knowledge on state policy diffusion in several respects. Additionally, in introducing network inference to political science, we provide scholars across the discipline with a general framework for empirically recovering the latent and dynamic interdependence among political actors.

A Comparison of Bayes Factor Approximation Methods Including Two New Methods

Bayes factors (BFs) play an important role in comparing the fit of statistical models. However, computational limitations or lack of an appropriate prior sometimes prevent researchers from using exact BFs. Instead, it is approximated, often using the Bayesian Information Criterion (BIC) or a variant of BIC. The authors provide a comparison of several BF approximations, including two new approximations, the Scaled Unit Information Prior Bayesian Information Criterion (SPBIC) and Information matrix-based Bayesian Information Criterion (IBIC). The SPBIC uses a scaled unit information prior that is more general than the BIC’s unit information prior, and the IBIC utilizes more terms of approximation than the BIC. Through simulation, the authors show that several measures perform well in large samples, that performance declines in smaller samples, and that SPBIC and IBIC provide improvement to existing measures under some conditions, including small sample sizes. The authors illustrate the use of the fit measures with the crime data of Ehrlich and then conclude with recommendations for researchers.

Affordable Homeownership: The Incidence and Effect of Down Payment Assistance

Using data from a panel study of low- and moderate-income homeowners, we assess the determinants of the use of several types of down payment assistance and the effect of using assistance on mortgage performance. Although we find differences in reliance on types of assistance, we find no difference in mortgage performance between those who used assistance and those who did not. Based on these findings, we urge caution in imposing down payment requirements that disproportionately restrict access to mortgage credit.

Can You Repeat That Please?: Using Monte Carlo Simulation in Graduate Quantitative Research Methods Classes

Graduate students in political science come to the discipline interested in exploring important political questions, such as “What causes war?” or “What policies promote economic growth?” However, they typically do not arrive prepared to address those questions using quantitative methods. Graduate methods instructors must provide the quantitative tools and statistical knowledge for answering these questions through data analysis. Often this is done with a heavy emphasis on lecture and textbook training, which comes with an array of Greek letters, formulae, and formal proofs, or by walking students through point-and-click statistical software without ever seriously engaging the underlying mathematical concepts. Neither approach is ideal because they both leave students lacking real understanding of core concepts. In this article, we discuss a useful alternative: Monte Carlo simulation. Simulations allow students to experience and visualize the meaning of core statistical concepts and to make comparisons between various statistical methods. We describe how simulation can illustrate these difficult concepts and improve comprehension of foundational material, even for students with minimal quantitative backgrounds.

Party Competition, Party Polarization, and the Changing Demand for Lobbying in the American States

Interest system density influences internal dynamics within interest organizations, how they lobby, and policy conditions. But how do political conditions influence interest system density? How does politics create demand for interest representation? We examine these questions by assessing how legislative party competition and ideological distance between parties in state legislatures affect the number of lobby groups. After stating our theoretical expectations, we examine 1997 and 2007 data on legislative competition and party polarization to assess their influence on system density. We find mixed results: Whereas politics slightly influenced the structuring of nonprofit interest communities, they seem to have not affected the structuring of for-profit interest communities or interest communities as a whole.

An Unbiased Model Comparison Test Using Cross-Validation

Social scientists often consider multiple empirical models of the same process. When these models are parametric and non-nested, the null hypothesis that two models fit the data equally well is commonly tested using methods introduced by Vuong (Econometrica 57(2):307–333, 1989) and Clarke (Am J Political Sci 45(3):724–744, 2001; J Confl Resolut 47(1):72–93, 2003; Political Anal 15(3):347–363, 2007). The objective of each is to compare the Kullback–Leibler Divergence (KLD) of the two models from the true model that generated the data. Here we show that both of these tests are based upon a biased estimator of the KLD, the individual log-likelihood contributions, and that the Clarke test is not proven to be consistent for the difference in KLDs. As a solution, we derive a test based upon cross-validated log-likelihood contributions, which represent an unbiased KLD estimate. We demonstrate the CVDM test’s superior performance via simulation, then apply it to two empirical examples from political science. We find that the test’s selection can diverge from those of the Vuong and Clarke tests and that this can ultimately lead to differences in substantive conclusions.

A Legislature or a Legislator Like Me? Citizen Demand for Collective and Dyadic Political Representation:

American politics scholars have long distinguished between legislature-level “collective” and legislator-level “dyadic” representation. However, most research on these concepts focuses on elite-level outcomes (e.g., policy output or roll-call behavior), and whether one or both forms leads to the representation of citizen interests. Less is known about the demand side of the relationship—whether constituents prefer collective or dyadic representation. Yet the citizen perspective is critical to both scholarly and normative discussions of representation. Through a novel survey experiment administered to national samples of Americans, we examine citizen preferences for collective and dyadic representation with respect to two important social identities: race and political partisanship. We posit that citizens prefer collective over dyadic representation because collective representation provides better representation of constituents’ interests via substantive and symbolic benefits. We show results that support this expectation, and then conclude by discussing the implications for scholarly and normative understandings of representation in American politics.