Ingo Rohlfing

Combining QCA and Process Tracing in Set-Theoretic Multi-Method Research

Set-theoretic methods and Qualitative Comparative Analysis (QCA) in particular are case-based methods. There are, however, only few guidelines on how to combine them with qualitative case studies. Contributing to the literature on multi-method research (MMR), we offer the first comprehensive elaboration of principles for the integration of QCA and case studies with a special focus on case selection. We show that QCAs reliance on set-relational causation in terms of necessity and sufficiency has important consequences for the choice of cases. Using real world data for both crisp-set and fuzzy-set QCA, we show what typical and deviant cases are in QCA-based MMR. In addition, we demonstrate how to select cases for comparative case studies aiming to discern causal mechanisms and address the puzzles behind deviant cases. Finally, we detail the implications of modifying the set-theoretic cross-case model in the light of case-study evidence. Following the principles developed in this article should increase the inferential leverage of set-theoretic MMR.

What you see and what you get: Pitfalls and principles of nested analysis in comparative research

In a recent contribution to this journal, Munck and Snyder found that many studies suffer from a deficient application of qualitative and quantitative methods. They argue that the combination of small-n and large-n analysis represents a viable method for promoting the production of knowledge. Recently, Evan Lieberman proposed nested analysis as a rigorous approach for comparative research that builds on the complementary strengths of quantitative and qualitative analysis. In this paper, the author examines the methodological potential of nested inference to advance comparative political analysis, arguing that the specific methodological problems of nested designs have not been fully appreciated. It is shown that, under certain circumstances, nothing is gained from a nested analysis. On the contrary, one might lose more than one gains compared to single-method designs. The author suggests specific methodological principles that take these problems into account to make nested analysis fruitful for comparative studies.

Case Studies Nested in Fuzzy-set QCA on Sufficiency Formalizing Case Selection and Causal Inference

Qualitative Comparative Analysis (QCA) is a method for cross-case analyses that works best when complemented with follow-up case studies focusing on the causal quality of the solution and its constitutive terms, the underlying causal mechanisms, and potentially omitted conditions. The anchorage of QCA in set theory demands criteria for follow-up case studies that are different from those known from regression-based multimethod research (MMR). Based on the evolving research on set-theoretic MMR, we introduce principles for formalized case selection and causal inference after a fuzzy-set QCA on sufficiency. Using an empirical example for illustration, we elaborate on the principles of counterfactuals for intelligible causal inference in the analysis of three different types of cases. Furthermore, we explain how case-based counterfactual inferences on the basis of QCA solutions are related to counterfactuals in the course of processing a truth table in order to produce a solution. We then flesh out two important functions that ideal types play for QCA-based case studies: First, they inform the development of formulas for the choice of the best available cases for with-case analysis and, second, establish the boundaries of generalization of the causal inferences.

Comparative hypothesis testing via process tracing

Causal inference via process tracing has received increasing attention during recent years. A 2 × 2 typology of hypothesis tests takes a central place in this debate. A discussion of the typology demonstrates that its role for causal inference can be improved further in three respects. First, the aim of this article is to formulate case selection principles for each of the four tests. Second, in focusing on the dimension of uniqueness of the 2 × 2 typology, I show that it is important to distinguish between theoretical and empirical uniqueness when choosing cases and generating inferences via process tracing. Third, I demonstrate that the standard reading of the so-called doubly decisive test is misleading. It conflates unique implications of a hypothesis with contradictory implications between one hypothesis and another. In order to remedy the current ambiguity of the dimension of uniqueness, I propose an expanded typology of hypothesis tests that is constituted by three dimensions.

The Push and Pull of Ministerial Resignations in Germany, 1969–2005

When and why are cabinet ministers forced out of office? We argue that ministerial resignations cannot be understood as mechanistic consequences of serious personal or departmental errors as the classical responsibility hypothesis implies. Rather, they follow a systematic political logic. Cabinet ministers have to resign whenever the prime minister perceives the political costs of a minister staying in office to be higher than the benefits of keeping the status quo. We test this argument with resignation events in Germany in the period 1969 to 2005. Based on detailed data collection, we find 111 resignation events, i.e. serious public discussions about a cabinet ministers future, 14 of which ended in resignation. These data are analysed employing statistical as well as Qualitative Comparative Analysis based on Boolean algebra to detect patterns of ministerial resignations.

Integrating Cross-case Analyses and Process Tracing in Set-Theoretic Research: Strategies and Parameters of Debate

In recent years, there has been increasing interest in the combination of two methods on the basis of set theory. In our introduction and this special issue, we focus on two variants of cross-case set-theoretic methods—qualitative comparative analysis (QCA) and typological theory (TT)—and their combination with process tracing (PT). Our goal is to broaden and deepen set-theoretic empirical research and equip scholars with guidance on how to implement it in multimethod research (MMR). At first glance, set-theoretic cross-case methods and PT seem to be highly compatible when causal relationships are conceptualized in terms of set theory. However, multiple issues have not so far been thoroughly addressed. Our article builds on the emerging MMR literature and seeks to enhance it in four ways. First, we offer a comprehensive and coherent elaboration of the two sequences in which case studies can be combined with a cross-case method. Second, we expand the perspective and discuss QCA and TT as two alternative methods for the cross-case analysis. Third, based on the idea of analytical priority, we introduce the distinction between a condition-centered and a mechanism-centered variant of set-theoretic MMR. Fourth, we point attention to the challenges of theorizing and analyzing arrangements of conditions and mechanisms associated with sufficient conjunctions.

Why simulations are appropriate for evaluating Qualitative Comparative Analysis

QCA has recently been subject to massive criticism and although the substance of that criticism is not completely new, it differs from earlier critiques by invoking simulations for the evaluation of QCA. In addition to debates about the meaning of the simulation results, there is a more fundamental discussion about whether simulations promise any relevant insights in principle. Some voices in the QCA community reject simulations per se because they necessarily lack case knowledge. As a consequence, the debate is at an impasse on a metalevel because critics of QCA rely on simulations, the results of which some QCA proponents refuse to consider as insightful. This article addresses this impasse and presents six reasons why simulations must be considered appropriate for evaluating QCA. I show that if taken to its conclusion, the central argument against simulations undermines the need for running a truth table analysis in the first place. The way forward in this debate should not be about whether simulations are useful, but how to configure meaningful simulations evaluating QCA.

Analyzing multilevel data with QCA: a straightforward procedure

The social sciences are witnessing a growing body of multilevel theories and debates about the proper methodological tools for the analysis of multilevel data. In a recent contribution to this journal, multilevel Qualitative Comparative Analysis (QCA) was proposed as a new methodological tool for discerning set-relational patterns in multilevel data. I argue that the presentation of multilevel QCA is erroneous in two respects. First, multilevel QCA ignores the fact that equifinal solutions entail diversity and therefore leads one to overestimate the complexity of QCA solutions. Second, the ordinary minimization procedure of truth tables that contain multilevel data yields the same solutions as multilevel QCA, but is much easier to implement. I conclude that the established inventory of QCA does not need to be extended by a special multilevel approach.

Mind the gap: A review of simulation designs for Qualitative Comparative Analysis

In a simulation-based analysis of Qualitative Comparative Analysis (QCA), Krogslund et al. (2015) conclude that its performance is suboptimal in several settings. I review their simulation setups and discuss three errors that were made in their analysis. First, the simulations involving inclusion thresholds are overpowered based on a misunderstanding of their role in truth table analyses. Second, the fact that a truth table analysis could exhibit model ambiguity and yield more than one model is ignored. If multiple models are derived from a truth table and they are combined into one, one overestimates the complexity of the models and underestimates their number, making it impossible to retrieve the target model of the simulation. Third, the simulations on the consequences of including irrelevant conditions intermingle sensitivity to overfitting with sensitivity to varying the inclusion thresholds. A reconsideration of KCP’s simulations correcting for the errors confirms some of their findings, but also reveals that some of those errors lead to an underestimation of QCA’s robustness. On a broader level, the review underscores that simulations are useful for the evaluation of QCA, but that simulation designs need to match QCA’s mechanics and principles to produce valid conclusions about its performance.

A Unifying Framework for Causal Analysis in Set-Theoretic Multimethod Research

The combination of Qualitative Comparative Analysis (QCA) with process tracing, which we call set-theoretic multimethod research (MMR), is steadily becoming more popular in empirical research. Despite the fact that both methods have an elected affinity based on set theory, it is not obvious how a within-case method operating in a single case and a cross-case method operating on a population of cases are compatible and can be combined in empirical research. There is a need to reflect on whether and how set-theoretic MMR is internally coherent and how QCA and process tracing can be integrated in causal analysis. We develop a unifying foundation for causal analysis in set-theoretic MMR that highlights the roles and interplay of QCA and process tracing. We argue that causal inference via counterfactuals on the level of single cases integrates QCA and process tracing and assigns proper and equally valuable roles to both methods.