Alessandra Mattei

Estimating and using propensity score in presence of missing background data: an application to assess the impact of childbearing on wellbeing

Propensity score methods are an increasingly popular technique for causal inference. To estimate propensity scores, we must model the distribution of the treatment indicator given a vector of covariates. Much work has been done in the case where the covariates are fully observed. Unfortunately, many large scale and complex surveys, such as longitudinal surveys, suffer from missing covariate values. In this paper, we compare three different approaches and their underlying assumptions of handling missing background data in the estimation and use of propensity scores: a complete-case analysis, a pattern-mixture model based approach developed by Rosenbaum and Rubin (J Am Stat Assoc79:516–524, 1984), and a multiple imputation approach. We apply these methods to assess the impact of childbearing events on individuals’ wellbeing in Indonesia, using a sample of women from the Indonesia Family Life Survey.

Assessing the effect of the amount of financial aids to Piedmont firms using the generalized propensity score

Regional and national development policies play an important role to support local enterprises in Italy. The amount of financial aid may be a key feature for firms’ employment policies. We study the impact on employment of the amount of financial aid attributed to enterprises located in Piedmont, a region in northern Italy, analysing small-sized firms and medium- or large-sized firms separately. We apply generalized propensity score methods under the unconfoundedness assumption that adjusting for differences in a set of observed pre-treatment variables removes all biases in comparisons by different amounts of financial aid. We find that the estimated effects are increasing with amount of financial aid for both small-sized and medium- or large-sized firms, whereas the marginal effects of additional incentives are decreasing with amount of financial aid for small-sized firms, and have an inverse J-shape for medium- or large-sized firms.

A Refreshing Account of Principal Stratification

Pearl (2011) invites researchers to contribute to a discussion on the logic and utility of principal stratification in causal inference, raising some thought-provoking questions. In our commentary, we discuss the role of principal stratification in causal inference, describing why we view the principal stratification framework as useful for addressing causal inference problems where causal estimands are defined in terms of intermediate outcomes. We focus on mediation analysis and principal stratification analysis, showing that they generally involve different causal estimands and answer different questions. We argue that even when principal stratification may not answer the causal questions of primary interest, it can be a preliminary analysis of the data to assess the plausibility of identifying assumptions. We also discuss the use of principal stratification to address issues of surrogate outcomes. Our discussion stresses that a principal stratification analysis should account for all the principal strata and evaluate the distributions of potential outcomes in each of the principal strata. To this end, we view a Bayesian analysis particularly suited for drawing inference on principal strata membership and principal strata effects.

Identification of causal effects in the presence of nonignorable missing outcome values

We consider a new approach to identify the causal effects of a binary treatment when the outcome is missing on a subset of units and dependence of nonresponse on the outcome cannot be ruled out even after conditioning on observed covariates. We provide sufficient conditions under which the availability of a binary instrument for nonresponse allows us to derive tighter identification intervals for causal effects in the whole population and to partially identify causal effects in some latent subgroups of units, named Principal Strata, defined by the nonresponse behavior in all possible combinations of treatment and instrument. A simulation study is used to assess the benefits of the presence versus the absence of an instrument for nonresponse. The simulation design is based on real health data, coming from a randomized trial on breast self-examination (BSE) affected by a large proportion of missing outcome data. An instrument for nonresponse is simulated considering alternative scenarios to discuss the key role of the instrument for nonresponse in identifying average causal effects in presence of nonignorable missing outcomes. We also investigate the potential inferential gains from using an instrument for nonresponse adopting a Bayesian approach for inference. In virtue of our theoretical and empirical results, we provide some recommendations on study designs for causal inference.

Evaluating the causal effect of university grants on student dropout: Evidence from a regression discontinuity design using principal stratification

Regression discontinuity (RD) designs are often interpreted as local randomized experiments: a RD design can be considered as a randomized experiment for units with a realized value of a so-called forcing variable falling around a pre-fixed threshold. Motivated by the evaluation of Italian university grants, we consider a fuzzy RD design where the receipt of the treatment is based on both eligibility criteria and a voluntary application status. Resting on the fact that grant application and grant receipt statuses are post-assignment (post-eligibility) intermediate variables, we use the principal stratification framework to define causal estimands within the Rubin Causal Model. We propose a probabilistic formulation of the assignment mechanism underlying RD designs, by re-formulating the Stable Unit Treatment Value Assumption (SUTVA) and making an explicit local overlap assumption for a subpopulation around the threshold. A local randomization assumption is invoked instead of more standard continuity assumptions. We also develop a model-based Bayesian approach to select the target subpopulation(s) with adjustment for multiple comparisons, and to draw inference for the target causal estimands in this framework. Applying the method to the data from two Italian universities, we find evidence that university grants are effective in preventing students from low-income families from dropping out of higher education.

Exploiting Multivariate Outcomes in Bayesian Inference for Causal Effects with Noncompliance

A Bayesian approach to causal inference in the presence of noncompliance to assigned randomized treatment is considered. It exploits multivariate outcomes for improving estimation of weakly identified models, when the usually invoked exclusion restriction assumptions are relaxed. Using artificial data sets, we analyze the properties of the posterior distribution of causal estimands to evaluate the potential gains of jointly modeling more than one outcome. The approach can be used to assess robustness with respect to deviations from structural identifying assumptions. It can also be extended to the analysis of observational studies with instrumental variables where exclusion restriction assumptions are usually questionable.

Assessing the causal effect of childbearing on household income in Albania

The relationship between demographic developments and economic performance has been the subject of rather intense debate in the economics literature for nearly two centuries. Until recently limitations on both data sources and statistical techniques have prevented clear insights into the relationship between population growth and economic wellbeing (Birdsall et al. 2001), and most of the existing studies have relied on either cross sectional or aggregate level data. Cross sectional data, no matter what techniques are applied, is unlikely to provide robust causal information about the relationship between the occurrence of life events (such as a childbearing event) and economic wellbeing. Past empirical studies concerning the relationship between economic wellbeing and fertility have consequently showed mixed results, indicating that the relationship does not appear to be unidirectional (see Schoumaker and Tabutin (1999) for further details). In this paper we analyze to what extent births may lead to changes in economic wellbeing. In contrast to most previous studies on this issue we apply appropriate econometric techniques based on longitudinal micro data in order to identify the causal effects of child bearing events on poverty. Fertility is measured in terms of childbearing events, and we use monthly real equivalised income as an indicator of household living standards. Childbearing might affect economic wellbeing through different channels. The most obvious one is that an additional child in the household increases the number of adult equivalence units without increasing household income. Therefore, childbearing would decrease, ceteris paribus, (equivalised) household income. However, as the economic theory suggests, there exist many factors that might interact with both fertility and income, generating economies and/or diseconomies of scale (Cigno 1991). One of the main factors concerns the impact of fertility on the optimal time allo-

Effect of Integrated Medicine on Physical Performances of Orthopaedic and Stroke Patients: A Propensity Score-Matched Study

Background: To evaluate the clinical efficacy of treatment with integrated medicine (acupuncture and homeopathy in addition to conventional treatment) in improving measures of physical ability. Data from medical records with patient-reported information on background characteristics, diagnosis on admission (orthopaedic conditions and stroke) and movement performance indices. Methods: Comparative, observational, propensity-score matched clinical study. Participants: 383 adults admitted to the rehabilitation Centre in Manciano, stratified by stroke and orthopaedic condition. Interventions: acupuncture and homeopathy with physical rehabilitation vs. rehabilitation alone. Main outcome measures: Activities of daily living performance indices for patients with knee or hip replacements (Barthel index) and stroke patients (Barthel index and Trunk control test). Number of days of analgesic drug treatment for orthopedic patients. Results: The activities of daily living performance indices showed a significant improvement for both stroke patients (Fisher p-value: 0.008 and 0.046 for Barthel index and Trunk Control Test respectively) and orthopaedic patients (Fisher p-value: 0.032 for Barthel index) treated with integrated medicine. The treatment also reduced the duration of analgesic treatment by 2.8 days (Fisher p-value: 0.0015). Conclusion: Integrated medicine helps to improve rehabilitation performance in stroke and orthopaedic patients and considerably reduces the need for analgesic drugs.

The effects of a dropout prevention program on secondary students’ outcomes

Innovare is a teacher-based dropout prevention program, promoted by the Regional government in Tuscany (Italy), aimed at reducing dropout in the early grades of vocational high schools through the introduction of innovative teaching methods. The Innovare study is a cluster-randomized experiment. We adopt a randomization inference approach to evaluate the effect of the Innovare program at the cluster-level, using sub-classifications on the propensity score to adjust for differences in the observed background pretreatment variables between treatment groups. We also conduct individual- level analyses using multilevel models. The results at both cluster- and individual- level suggest that Innovare has positive effects. Although the estimated effects are small, there is some evidence that Innovare decreases the probability of failing and dropping out, reduces the absence rate, and increases the probability of postponing the evaluation.

Bayesian inference for causal mechanisms with application to a randomized study for postoperative pain control

Summary We conduct principal stratification and mediation analysis to investigate to what extent the positive overall effect of treatment on postoperative pain control is mediated by postoperative self administration of intra‐venous analgesia by patients in a prospective, randomized, double‐blind study. Using the Bayesian approach for inference, we estimate both associative and dissociative principal strata effects arising in principal stratification, as well as natural effects from mediation analysis. We highlight that principal stratification and mediation analysis focus on different causal estimands, answer different causal questions, and involve different sets of structural assumptions.