David Collier

Measurement Validity: A Shared Standard for Qualitative and Quantitative Research

Scholars routinely make claims that presuppose the validity of the observations and measurements that operationalize their concepts. Yet, despite recent advances in political science methods, surprisingly little attention has been devoted to measurement validity. We address this gap by exploring four themes. First, we seek to establish a shared framework that allows quantitative and qualitative scholars to assess more effectively, and communicate about, issues of valid measurement. Second, we underscore the need to draw a clear distinction between measurement issues and disputes about concepts. Third, we discuss the contextual specificity of measurement claims, exploring a variety of measurement strategies that seek to combine generality and validity by devoting greater attention to context. Fourth, we address the proliferation of terms for alternative measurement validation procedures and offer an account of the three main types of validation most relevant to political scientists.

Understanding Process Tracing

Process tracing is a fundamental method of qualitative analysis. While it is often invoked by scholars as they examine qualitative data, too frequently this tool is neither adequately understood nor rigorously applied. This deficit motivates the present article, which offers a new framework for carrying out process tracing. This reformulation integrates discussions of process tracing and causal-process observations, gives greater attention to description as a key contribution, and emphasizes the causal sequence in which process-tracing observations can be situated. In the current period of major innovation in quantitative tools for causal inference, this reformulation is part of a wider, parallel effort to achieve greater systematization of qualitative methods. A key point here is that these methods can add inferential leverage too often lacking in quantitative analysis. The presentation is accompanied by teaching exercises, which focus on three examples from international relations, one from American politics, two from comparative politics, and one from public health/epidemiology.

Prerequisites Versus Diffusion: Testing Alternative Explanations of Social Security Adoption

Social security is one of the most important means by which modern nations protect the welfare of their citizens. The first appearance of social security represents a particularly important policy juncture at which many nations broke from the anti-welfare doctrine of traditional liberalism. The bulk of scholarship on social security treats its adoption as an explanation for other aspects of the social security experience, instead of an outcome to be explained. This article analyzes two alternative explanations of the timing of adoption: the prerequisites hypothesis, involving the level of development within nations, versus diffusion among nations. Three patterns emerge. Among the earliest adopters, social security diffuses up a developmental hierarchy of nations rather than down a hierarchy; in the middle group of adopters, a pattern of spatial diffusion is present in which social security is rapidly diffused among countries at widely different levels of modernization; and among the latest adopters, a combination of hierarchical diffusion and a prerequisite explanation appear to be the most satisfactory means of accounting for the pattern of adoption.

Essentially Contested Concepts: Debates and Applications

Conceptual confusion has long been a source of difficulty in the study of politics. W. B. Gallies analysis of ‘essentially contested concepts’, published in 1956, stands as a notable effort to address this problem. He explores the normative component of these concepts and offers seven criteria for evaluating their contestedness. In the present article, we examine Gallies framework and develop two extended applications, focused on ‘democracy’ and ‘rule of law’. We underscore major contributions of Gallies approach, as well as controversies it has generated. Some important critiques argue that three of his criteria are too narrow. We suggest that these critics fail to recognize that Gallie offers both a restrictive and broader definition of these criteria, and we seek to reconcile their views with his alternative definitions. Further, some accuse Gallie of naively promoting conceptual relativism by undermining standards for evaluating concepts, and others argue more sympathetically that he is too optimistic about prospects for resolving conceptual disputes. It is of course difficult to achieve Gallies goal of promoting the reasoned discussion of these concepts, given the sharply contrasting normative and analytic perspectives that scholars bring to them. Yet his framework, augmented by the refinements explored in this article, opens promising avenues for addressing this challenge.

Putting Typologies to Work: Concept Formation, Measurement, and Analytic Rigor

Typologies are well-established analytic tools in the social sciences. They can be “put to work” in forming concepts, refining measurement, exploring dimensionality, and organizing explanatory claims. Yet some critics, basing their arguments on what they believe are relevant norms of quantitative measurement, consider typologies old-fashioned and unsophisticated. This critique is methodologically unsound, and research based on typologies can and should proceed according to high standards of rigor and careful measurement. These standards are summarized in guidelines for careful work with typologies, and an illustrative inventory of typologies, as well as a brief glossary, are included online.

Trajectory of a Concept: ‘Corporatism’ in the Study of Latin American Politics

Beginning in the 1970s, “corporatism” became a major focus of research on Latin America, a focus readily embraced by scholars who analyzed the authoritarian political relationships emerging during that period. The concept provided a valuable alternative to earlier pluralist models and offered a useful optic for examining state control of interest group politics. This article explores the trajectory of corporatism as a concept in the study of Latin America. As the literature evolved, it was marked by: (a) convergence around similar, core definitions of corporatism; and (b) a shared understanding that within the domain of group politics, corporatism was a specific form or type in relation to the overarching concept of interest intermediation. Important conceptual innovations in this literature included: (c) revision of the overarching concept to accommodate potentially overlapping or conflicting meanings; and (d) introduction of both classical subtypes, which consist of corporatism plus some further differentiating attribute, and radial subtypes, in which a core attribute of corporatism is lacking. This use of the concept to characterize group politics stood in notable contrast with a much broader usage that posited an Iberic-Latin historical tradition of hierarchal, statist authority relations. This latter perspective has been sharply criticized because it groups together such heterogeneous phenomena under a single concept, and because it yields explanatory claims so sweeping as to be implausible. Yet some caution must be exercised in evaluating these criticisms.Author(s): Smith, Peter H | Editor(s): Peter H. Smith | Abstract:

Strategic Choice Models of Political Change in Latin America

The article assesses the use of strategic choice models in the study of Latin American politics. These models explore how given actors pursue goals by shaping the context in which other actors make choices. The discussion centers on Hirschman’s analysis of “reform-mongering,” Przeworski’s “threshold” model of transitions to democracy, and O’Donnell’s model of democratic consolidation. Basic components of the models are examined, including the definition of actors, preference distributions, coalitional thresholds, perceptions of the likelihood of given outcomes, and efforts to change actual and perceived costs of these outcomes. The relationship between such models and more familiar perspectives in the Latin American field is then explored. The models have a distinctive emphasis on uncertainty and the creative use of uncertainty by political leaders; yet they also have much in common with other research traditions. The article advocates eclecticism in employing these alternative analytic approaches.

Comment QCA Should Set Aside the Algorithms

Qualitative comparative analysis (QCA) has received substantial attention from qualitative scholars seeking to systematize their research. This method has valuable goals: understanding context, interactions, and causal complexity, including asymmetric causation. These objectives are pursued with central attention to case knowledge. However, simulations suggest that QCA’s core analytic procedures— its algorithms—provide a weak foundation for pursuing its goals. What began as a refreshingly simple method has, counterproductively, become much more complicated. QCA scholars should turn their attention to more traditional qualitative tools: case studies and process tracing. With such tools, they can pursue the important methodological objectives that motivated Charles Ragin to create QCA, unencumbered by algorithms that may well be obstacles to achieving these goals.

Rival Strategies of Validation Tools for Evaluating Measures of Democracy

The challenge of finding appropriate tools for measurement validation is an abiding concern in political science. This article considers four traditions of validation, using examples from cross-national research on democracy: the levels-of-measurement approach, structural-equation modeling with latent variables, the pragmatic tradition, and the case-based method. Methodologists have sharply disputed the merits of alternative traditions. We encourage scholars—and certainly analysts of democracy—to pay more attention to these disputes and to consider strengths and weaknesses in the validation tools they adopt. An online appendix summarizes the evaluation of six democracy data sets from the perspective of alternative approaches to validation. The overall goal is to open a new discussion of alternative validation strategies.

Methods for Census 2000 and Statistical Adjustments

This article outlines procedures for taking the US census, making adjustments, and evaluating the results. The census turns out to be remarkably accurate. Statistical adjustment is unlikely to improve on the census, because adjustment can easily put in more error than it takes out. Indeed, error rates in the adjustment are comparable to if not larger than errors in the census. The data suggest a strong geographical pattern to such errors even after controlling for demographic variables, which contradicts basic premises of adjustment. In fact, the complex demographic controls built into the adjustment process seem on whole to have been counter-productive.