Paul R. Rosenbaum

Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score

Abstract Matched sampling is a method for selecting units from a large reservoir of potential controls to produce a control group of modest size that is similar to a treated group with respect to the distribution of observed covariates. We illustrate the use of multivariate matching methods in an observational study of the effects of prenatal exposure to barbiturates on subsequent psychological development. A key idea is the use of the propensity score as a distinct matching variable.

Reducing Bias in Observational Studies Using Subclassification on the Propensity Score

Abstract The propensity score is the conditional probability of assignment to a particular treatment given a vector of observed covariates. Previous theoretical arguments have shown that subclassification on the propensity score will balance all observed covariates. Subclassification on an estimated propensity score is illustrated, using observational data on treatments for coronary artery disease. Five subclasses defined by the estimated propensity score are constructed that balance 74 covariates, and thereby provide estimates of treatment effects using direct adjustment. These subclasses are applied within sub-populations, and model-based adjustments are then used to provide estimates of treatment effects within these sub-populations. Two appendixes address theoretical issues related to the application: the effectiveness of subclassification on the propensity score in removing bias, and balancing properties of propensity scores with incomplete data.

Model-Based Direct Adjustment

Abstract Direct adjustment or standardization applies population weights to subclass means in an effort to estimate population quantities from a sample that is not representative of the population. Direct adjustment has several attractive features, but when there are many subclasses it can attach large weights to small quantities of data, often in a fairly erratic manner. In the extreme, direct adjustment can attach infinite weight to nonexistent data, a noticeable inconvenience in practice. This article proposes a method of model-based direct adjustment that preserves the attractive features of conventional direct adjustment while stabilizing the weights attached to small subclasses. The sample mean and conventional direct adjustment are both special cases of model-based direct adjustment under two different extreme models for the subclass-specific selection probabilities. The discussion of this method provides some insights into the behavior of true and estimated propensity scores: the estimated scores ...

Design of Observational Studies

Beginnings.- Dilemmas and Craftsmanship.- Causal Inference in Randomized Experiments.- Two Simple Models for Observational Studies.- Competing Theories Structure Design.- Opportunities, Devices, and Instruments.- Transparency.- Matching.- A Matched Observational Study.- Basic Tools of Multivariate Matching.- Various Practical Issues in Matching.- Fine Balance.- Matching Without Groups.- Risk-Set Matching.- Matching in R.- Design Sensitivity.- The Power of a Sensitivity Analysis and Its Limit.- Heterogeneity and Causality.- Uncommon but Dramatic Responses to Treatment.- Anticipated Patterns of Response.- Planning Analysis.- After Matching, Before Analysis.- Planning the Analysis.

Comparison of Multivariate Matching Methods: Structures, Distances, and Algorithms

Abstract A comparison and evaluation is made of recent proposals for multivariate matched sampling in observational studies, where the following three questions are answered: (1) Algorithms: In current statistical practice, matched samples are formed using “nearest available” matching, a greedy algorithm. Greedy matching does not minimize the total distance within matched pairs, though good algorithms exist for optimal matching that do minimize the total distance. How much better is optimal matching than greedy matching? We find that optimal matching is sometimes noticeably better than greedy matching in the sense of producing closely matched pairs, sometimes only marginally better, but it is no better than greedy matching in the sense of producing balanced matched samples. (2) Structures: In common practice, treated units are matched to one control, called pair matching or 1–1 matching, or treated units are matched to two controls, called 1–2 matching, and so on. It is known, however, that the optimal st...

The Bias Due to Incomplete Matching

SUMMARY Observational studies comparing groups of treated and control units are often used to estimate the effects caused by treatments. Matching is a method for sampling a large reservoir of potential controls to produce a control group of modest size that is ostensibly similar to the treated group. In practice, there is a trade-off between the desires to find matches for all treated units and to obtain matched treated-control pairs that are extremely similar to each other. We derive expressions for the bias in the average matched pair difference due to (i) the failure to match all treated units-incomplete matching, and (ii) the failure to obtain exact matches-inexact matching. A practical example shows that the bias due to incomplete matching can be severe, and moreover, can be avoided entirely by using an appropriate multivariate nearest available matching algorithm, which, in the example, leaves only a small residual bias due to inexact matching.

Optimal Matching for Observational Studies

Abstract Matching is a common method of adjustment in observational studies. Currently, matched samples are constructed using greedy heuristics (or “stepwise” procedures) that produce, in general, suboptimal matchings. With respect to a particular criterion, a matched sample is suboptimal if it could be improved by changing the controls assigned to specific treated units, that is, if it could be improved with the data at hand. Here, optimal matched samples are obtained using network flow theory. In addition to providing optimal matched-pair samples, this approach yields optimal constructions for several statistical matching problems that have not been studied previously, including the construction of matched samples with multiple controls, with a variable number of controls, and the construction of balanced matched samples that combine features of pair matching and frequency matching. Computational efficiency is discussed. Extensive use is made of ideas from two essentially disjoint literatures, namely st...

From Association to Causation in Observational Studies: The Role of Tests of Strongly Ignorable Treatment Assignment

Abstract If treatment assignment is strongly ignorable, then adjustment for observed covariates is sufficient to produce consistent estimates of treatment effects in observational studies. A general approach to testing this critical assumption is developed and applied to a study of the effects of nuclear fallout on the risk of childhood leukemia. R.A. Fishers advice on the interpretation of observational studies was “Make your theories elaborate”; formally, make causal theories sufficiently detailed that, under the theory, strongly ignorable assignment has testable consequences.

Discussing hidden bias in observational studies.

In observational studies or nonrandomized experiments, treated and control groups may differ in their outcomes even if the treatment has no effect; this may happen if the groups were not comparable before the start of treatment. The groups may fail to be comparable in either of two ways: They may differ with respect to characteristics that have been measured, in which case there is an overt bias, or they may differ in ways that have not been measured, in which case there is a hidden bias. Overt biases are controlled through adjustments, such as matching. Hidden bias is more difficult to address because the relevant measurements are not available. A sensitivity analysis asks how much hidden bias would need to be present if hidden bias were to explain the differing outcomes in the treated and control groups. A sensitivity analysis provides a tangible and specific framework for discussing hidden biases.

Combining propensity score matching and group-based trajectory analysis in an observational study

In a nonrandomized or observational study, propensity scores may be used to balance observed covariates and trajectory groups may be used to control baseline or pretreatment measures of outcome. The trajectory groups also aid in characterizing classes of subjects for whom no good matches are available and to define substantively interesting groups between which treatment effects may vary. These and related methods are illustrated using data from a Montreal-based study. The effects on subsequent violence of gang joining at age 14 are studied while controlling for measured characteristics of boys prior to age 14. The boys are divided into trajectory groups based on violence from ages 11 to 13. Within trajectory group, joiners are optimally matched to a variable number of controls using propensity scores, Mahalanobis distances, and a combinatorial optimization algorithm. Use of variable ratio matching results in greater efficiency than pair matching and also greater bias reduction than matching at a fixed ratio. The possible impact of failing to adjust for an important but unmeasured covariate is examined using sensitivity analysis.