Kelly Rader

Randomization Tests and Multi-Level Data in U.S. State Politics

Many hypotheses in U.S. state politics research are multi-level, positing that state-level variables affect individual-level behavior. Unadjusted standard errors for state-level variables are too small, leading to overconfidence and possible false rejection of null hypotheses. Primo, Jacobsmeier, and Milyo (2007) explore this problem in their reanalysis of Wolfinger, Highton, and Mullins (2005) data on the effects of post-registration laws on voter turnout. Primo et al. advocate the use of clustered standard errors to solve the overconfidence problem, but we offer an alternative solution: randomization tests. Randomization tests are non-parametric tests that do not rely on comparisons to theoretical test statistic distributions. Instead, they use distributions tailored to the data, created by randomly scrambling the data many times to simulate what would be observed under the null hypothesis. Unlike with clustering, with the randomization test, U.S. state-level reforms generally fail to be significant both as additive effects and as interactions with individual characteristics.

Dyadic Analysis in International Relations: A Cautionary Tale

We explore problems with the use of dyadic data in international relations. We illustrate these problems by analyzing a central proposition among IR scholars that democracies seek out other democracies as trading partners. Our main contribution is to present randomization tests to infer the correct p-values associated with the trade hypotheses. Our results show that typical statistical tests for significance are severely overconfident in dyadic data. Second, we show that democratic trade can be modeled using nations as the units of analysis, testing whether the proportion of trade with other democracies increases when a country becomes more democratic. Third, we conduct a difference-in-differences analysis of change in trading partners following democratic or anti-democratic shocks. Rather than adding further layers of statistical complexity, these tests are simple and intuitive. They provide the cleanest evidence that when nations undergo democratic or nondemocratic transitions, their trade patterns change just as theory would suggest.

Have Food Stamp Program Changes Increased Participation

Food stamp caseloads increased dramatically between October 2000 and October 2003. This trend could be explained by the declining economy or by Food Stamp Program changes that increased the likelihood that families leaving welfare would keep their food stamps. This article shows that families leaving welfare between 2000 and 2002 were substantially more likely to continue to participate in the Food Stamp Program than families that left welfare between 1995 and 1997 or 1997 and 1999. Participation rates for families with no cash welfare experience did not change during the period 1997–2002. Differences in the characteristics of families do not explain these trends in participation. Food Stamp Program changes thus appear to have facilitated the greater participation of former welfare recipients.

Bargaining Power in the Supreme Court: Evidence from Opinion Assignment and Vote Switching

How can we assess relative bargaining power within the Supreme Court? Justices cast two votes in every case, one during the initial conference and one on the final merits of the case. Between these two votes, a justice is assigned to draft the majority opinion. We argue that vote switching can be used to detect the power of opinion authors over opinion content. Bargaining models make different predictions for opinion content and therefore for when other justices in the initial majority should be more or less likely to defect from initial positions. We derive hypotheses for how opinion authorship should affect vote switching and find that authorship has striking effects on switching. Authors thus have disproportionate influence and by extension so do chief justices, who make most assignments. This evidence is compatible with only the “author influence” class of bargaining models, with particular support for one model within this class.

The Federal Spending Paradox: Economic Self-Interest and Symbolic Racism in Contemporary Fiscal Politics

We show how symbolic politics condition public opinion on federal spending and how this helps to explain an important puzzle in contemporary American politics. Using multilevel regression and poststratification to estimate state-level opinion on federal spending, we show that, curiously, opposition to federal spending is higher in states receiving more federal money, per tax dollar paid. Belying the popular narrative surrounding so-called “red state socialism,” we find that simple hypocrisy does not explain this paradox—individuals who are likely to benefit from spending tend to support it. But, income is a more powerful predictor of opinion on spending in “taker” states than “giver” states, heightening state-level opposition in the former. There is also more to the story than economic self-interest. Symbolic racism is four times more powerful than income in explaining opposition to spending, and there are more people with such attitudes in states receiving more federal money.

The Three Prongs of a Jurisprudential Regimes Test: A Response to Kritzer and Richards

I n ‘‘Legal Constraints on Supreme Court Decision Making: Do Jurisprudential Regimes Exist?’’, we explored whether one can say that jurisprudential regime change occurred in Supreme Court decision making—whether key legal precedents led to changes in how justices voted. We found that the standard test, a Chow test of coefficient change, used in Kritzer and Richards’s research design, is strikingly overconfident in finding that a change has occurred in voting across cases before and after the precedent. Rather than making a Type-1 error of finding regime change when none exists 5% of the time, the test does so sometimes close to 100% of the time, even though the data has been randomly shuffled so that no systematic difference can exist between the before and after cases. We appreciate Kritzer and Richards’s openmindedness about our inquiry into their findings and are grateful for their thoughtful responses (now and while we worked on our original paper). We also appreciate the opportunity to clarify our findings, respond to their arguments (old and new), and to present supplemental results that answer, we hope, the questions they raised. Kritzer and Richards state that our paper focuses on only one of the three prongs of their argument, the significance test of regime change, whereas they now clarify that the second and third elements are actually of great (greater?) importance. They have also supplemented their findings with a more sophisticated test of regime change. We discuss each of these three components, including their new statistical evidence, below.