Dominik Hangartner

Validating vignette and conjoint survey experiments against real-world behavior

Significance Little evidence exists on whether preferences about hypothetical choices measured in a survey experiment are driven by the same structural determinants of the actual choices made in the real world. This study answers this question using a natural experiment as a behavioral benchmark. Comparing the results from conjoint and vignette experiments on which attributes of hypothetical immigrants generate support for naturalization with the outcomes of closely corresponding referendums in Switzerland, we find that the effects estimated from the surveys match the effects of the same attributes in the behavioral benchmark remarkably well. We also find that seemingly subtle differences in survey designs can produce significant differences in performance. Overall, the paired conjoint design performs the best. Survey experiments, like vignette and conjoint analyses, are widely used in the social sciences to elicit stated preferences and study how humans make multidimensional choices. However, there is a paucity of research on the external validity of these methods that examines whether the determinants that explain hypothetical choices made by survey respondents match the determinants that explain what subjects actually do when making similar choices in real-world situations. This study compares results from conjoint and vignette analyses on which immigrant attributes generate support for naturalization with closely corresponding behavioral data from a natural experiment in Switzerland, where some municipalities used referendums to decide on the citizenship applications of foreign residents. Using a representative sample from the same population and the official descriptions of applicant characteristics that voters received before each referendum as a behavioral benchmark, we find that the effects of the applicant attributes estimated from the survey experiments perform remarkably well in recovering the effects of the same attributes in the behavioral benchmark. We also find important differences in the relative performances of the different designs. Overall, the paired conjoint design, where respondents evaluate two immigrants side by side, comes closest to the behavioral benchmark; on average, its estimates are within 2% percentage points of the effects in the behavioral benchmark.

EXAMINATION OF THE SYNTHETIC CONTROL METHOD FOR EVALUATING HEALTH POLICIES WITH MULTIPLE TREATED UNITS

Abstract This paper examines the synthetic control method in contrast to commonly used difference‐in‐differences (DiD) estimation, in the context of a re‐evaluation of a pay‐for‐performance (P4P) initiative, the Advancing Quality scheme. The synthetic control method aims to estimate treatment effects by constructing a weighted combination of control units, which represents what the treated group would have experienced in the absence of receiving the treatment. While DiD estimation assumes that the effects of unobserved confounders are constant over time, the synthetic control method allows for these effects to change over time, by re‐weighting the control group so that it has similar pre‐intervention characteristics to the treated group. We extend the synthetic control approach to a setting of evaluation of a health policy where there are multiple treated units. We re‐analyse a recent study evaluating the effects of a hospital P4P scheme on risk‐adjusted hospital mortality. In contrast to the original DiD analysis, the synthetic control method reports that, for the incentivised conditions, the P4P scheme did not significantly reduce mortality and that there is a statistically significant increase in mortality for non‐incentivised conditions. This result was robust to alternative specifications of the synthetic control method.

Does Compulsory Voting Increase Support for Leftist Policy

Citizens unequally participate in elections and this may systematically bias policy in favor of those who vote. Many view compulsory voting as an important tool to alleviate this problem, but we still know very little about its policy consequences. We argue that sanctioned compulsory voting mobilizes citizens at the bottom of the income distribution and that this translates into an increase in support for redistributive policies. We empirically explore the effects of a sanctioned compulsory voting law on direct-democratic decisionmaking in the Swiss canton of Vaud from 1908 to 1947, a formative period for the welfare state. We find that compulsory voting increases turnout in referendums by 30 percentage points to over 85% on average and doubles electoral support for left policy positions. We discuss the implications of these results for our understanding of the policy consequences of electoral institutions and the evolution of the modern welfare state. We thank Simon Jackman, Peter Selb, Kenneth Scheve, and audiences at Stanford University, the University of Konstanz, and the University of St.Gallen for valuable comments. We thank Yvan Rielle for providing data, and Fabian Morgenthaler, Christian Vogt and Nils Burk for valuable research assistance. Michael M. Bechtel gratefully acknowledges support by the Swiss National Science Foundation (grant #PP00P1-139035). A full replication archive for this article will be made available online at the dataverse network (http://thedata.org/).Citizens unequally participate in referendums and this may systematically bias policy in favor of those who vote. Some view compulsory voting as an important tool to alleviate this problem while others worry about its detrimental effects on the legitimacy and quality of democratic decision-making. So far, however, we lack systematic knowledge about the causal effect of compulsory voting on public policy. We argue that sanctioned compulsory voting mobilizes citizens at the bottom of the income distribution and that this translates into an increase in support for leftist policies. We empirically explore the effects of a sanctioned compulsory voting law on direct-democratic decision-making in Switzerland. We find that compulsory voting significantly increases electoral support for leftist policy positions in referendums by up to 20 percentage points. We discuss the implications of these results for our understanding of the policy consequences of electoral institutions.

Reality Bites: The Limits of Framing Effects for Salient and Contested Policy Issues

A large literature argues that public opinion is vulnerable to various types of framing and cue effects. However, we lack evidence on whether existing findings, which are typically based on lab experiments involving low salience issues, travel to salient and contentious political issues in real-world voting situations. We examine the relative importance of issue frames, partisan cues, and their interaction for opinion formation using a survey experiment conducted around a highly politicized referendum on immigration policy in Switzerland. We find that voters responded to frames and cues, regardless of their direction, by increasing support for the position that is in line with their pre-existing partisan attachment. This reinforcement effect was most visible among low knowledge voters that identified with the party that owned the issue. These results support some of the previous findings in the political communication literature, but at the same time also point toward possible limits to framing effects in the context of salient and contested policy issues.

Naturalization fosters the long-term political integration of immigrants

Significance The political integration of immigrant minorities is one of the most pressing policy issues many countries face today. Despite heated debates, there exists little rigorous evidence about whether naturalization fosters or dampens the integration of immigrants into the political fabric of the host society. Our study provides new causal evidence on the long-term effects of naturalization on political integration. Our research design takes advantage of a natural experiment in Switzerland that allows us to separate the independent effect of naturalization from the nonrandom selection into naturalization. We find that in our sample, naturalization caused long-lasting improvements in political integration, with immigrants becoming likely to vote and attaining considerably higher levels of political efficacy and political knowledge. Does naturalization cause better political integration of immigrants into the host society? Despite heated debates about citizenship policy, there exists almost no evidence that isolates the independent effect of naturalization from the nonrandom selection into naturalization. We provide new evidence from a natural experiment in Switzerland, where some municipalities used referendums as the mechanism to decide naturalization requests. Balance checks suggest that for close naturalization referendums, which are decided by just a few votes, the naturalization decision is as good as random, so that narrowly rejected and narrowly approved immigrant applicants are similar on all confounding characteristics. This allows us to remove selection effects and obtain unbiased estimates of the long-term impacts of citizenship. Our study shows that for the immigrants who faced close referendums, naturalization considerably improved their political integration, including increases in formal political participation, political knowledge, and political efficacy.

When lives are put on hold: Lengthy asylum processes decrease employment among refugees

Refugees waiting an additional year for their asylum claim decision face a 20% reduction in their subsequent employment rate. European governments are struggling with the biggest refugee crisis since World War II, but there exists little evidence regarding how the management of the asylum process affects the subsequent integration of refugees in the host country. We provide new causal evidence about how one central policy parameter, the length of time that refugees wait in limbo for a decision on their asylum claim, affects their subsequent economic integration. Exploiting exogenous variation in wait times and registry panel data covering refugees who applied in Switzerland between 1994 and 2004, we find that one additional year of waiting reduces the subsequent employment rate by 4 to 5 percentage points, a 16 to 23% drop compared to the average rate. This deleterious effect is remarkably stable across different subgroups of refugees stratified by gender, origin, age at arrival, and assigned language region, a pattern consistent with the idea that waiting in limbo dampens refugee employment through psychological discouragement, rather than a skill atrophy mechanism. Overall, our results suggest that marginally reducing the asylum waiting period can help reduce public expenditures and unlock the economic potential of refugees by increasing employment among this vulnerable population.

Catalyst or Crown: Does Naturalization Promote the Long-Term Social Integration of Immigrants?

We study the impact of naturalization on the long-term social integration of immigrants into the host country society. Despite ongoing debates about citizenship policy, we lack reliable evidence that isolates the causal effect of naturalization from the nonrandom selection into naturalization. We exploit the quasi-random assignment of citizenship in Swiss municipalities that used referendums to decide on naturalization applications of immigrants. Comparing otherwise similar immigrants who narrowly won or lost their naturalization referendums, we find that receiving Swiss citizenship strongly improved long-term social integration. We also find that the integration returns to naturalization are larger for more marginalized immigrant groups and when naturalization occurs earlier, rather than later in the residency period. Overall, our findings support the policy paradigm arguing that naturalization is a catalyst for improving the social integration of immigrants rather than merely the crown on the completed integration process.

Does Direct Democracy Hurt Immigrant Minorities? Evidence from Naturalization Decisions in Switzerland

Do minorities fare worse under direct democracy than under representative democracy? We provide new evidence by studying naturalization requests of immigrants in Switzerland that were typically decided with referendums in each municipality. Using panel data from 1,400 municipalities for the 1991-2009 period, we exploit Federal Court rulings that forced municipalities to transfer the decisions to their elected municipality councils. We find that naturalization rates surged by 60% once politicians rather than citizens began deciding on naturalization applications. Whereas voters in referendums face no cost of arbitrarily rejecting qualified applicants based on discriminatory preferences, politicians in the council are constrained to formally justify rejections and may be held accountable by judicial review. Consistent with this mechanism, the increase in naturalization rates caused by switching from direct to representative democracy is much stronger for more marginalized immigrant groups and in areas where voters are more xenophobic or where judicial review is more salient.

Compulsory Voting, Habit Formation, and Political Participation

States, rms, and many other types of organizations rely on the internalization of norms to realize more cooperative outcomes. We examine norm internalization in the context of a severely sanctioned and long-standing compulsory voting law in the Swiss canton of Vaud (1900-1970). Compulsory voting strongly increases turnout in federal referenda, by about 30 percentage points on average. However, this eect quickly returns to zero after voting is no longer compulsory. Moreover, we nd only minor positive spillover eects on related forms of political participation. Our results challenge established theories of norm internalization in the context of political collective action.

Open/closed list and party choice: experimental evidence from the UK

Which parties benefit from open-list (as opposed to closed-list) proportional representation elections? This article shows that a move from closed-list to open-list competition is likely to be more favorable to parties with more internal disagreement on salient issues; this is because voters who might have voted for a unified party under closed lists may be drawn to specific candidates within internally divided parties under open lists. The study provides experimental evidence of this phenomenon in a hypothetical European Parliament election in the UK, in which using an open-list ballot would shift support from UKIP (the Eurosceptic party) to Eurosceptic candidates of the Conservative Party. The findings suggest that open-list ballots could restrict support for parties that primarily mobilize on a single issue.