Winston Lin

The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study

This paper examines the benefits and costs of Job Training Partnership Act (JTPA) Title II-A programs for economically disadvantaged adults and out-of-school youths. It is based on a 21,000-person randomized experiment conducted within ongoing Title II-A programs at 16 local JTPA Service Delivery Areas (SDAs) from around the country. In the paper, we present the background and design of our study, describe the methodology used to estimate program impacts, present estimates of program impacts on earnings and educational attainment, and assess the overall success of the programs studied through a benefit-cost analysis.

Unsupervised learning of generalized names

We present an algorithm, NOMEN, for learning generalized names in text. Examples of these are names of diseases and infectious agents, such as bacteria and viruses. These names exhibit certain properties that make their identification more complex than that of regular proper names, NOMEN uses a novel form of bootstrapping to grow sets of textual instances and of their contextual patterns. The algorithm makes use of competing evidence to boost the learning of several categories of names simultaneously. We present results of the algorithm on a large corpus. We also investigate the relative merits of several evaluation strategies.

Standard Operating Procedures: A Safety Net for Pre-Analysis Plans

Across the social sciences, growing concerns about research transparency have led to calls for pre-analysis plans (PAPs) that specify in advance how researchers intend to analyze the data they are about to gather. PAPs promote transparency and credibility by helping readers distinguish between exploratory and confirmatory analyses. However, PAPs are time-consuming to write and may fail to anticipate contingencies that arise in the course of data collection. This article proposes the use of “standard operating procedures” (SOPs)—default practices to guide decisions when issues arise that were not anticipated in the PAP. We offer an example of an SOP that can be adapted by other researchers seeking a safety net to support their PAPs.

A "placement of death" approach for studies of treatment effects on ICU length of stay.

Length of stay in the intensive care unit (ICU) is a common outcome measure in randomized trials of ICU interventions. Because many patients die in the ICU, it is difficult to disentangle treatment effects on length of stay from effects on mortality; conventional analyses depend on assumptions that are often unstated and hard to interpret or check. We adapt a proposal from Rosenbaum that addresses concerns about selection bias and makes its assumptions explicit. A composite outcome is constructed that equals ICU length of stay if the patient was discharged alive and indicates death otherwise. Given any preference ordering that compares death with possible lengths of stay, we can estimate the intervention’s effects on the composite outcome distribution. Sensitivity analyses can show results for different preference orderings. We discuss methods for constructing approximate confidence intervals for treatment effects on quantiles of the outcome distribution or on proportions of patients with outcomes preferable to various cutoffs. Strengths and weaknesses of possible primary significance tests (including the Wilcoxon–Mann–Whitney rank sum test and a heteroskedasticity-robust variant due to Brunner and Munzel) are reviewed. An illustrative example reanalyzes a randomized trial of an ICU staffing intervention.

Understanding Causal Distributional and Subgroup Effects With the Instrumental Propensity Score

To address issues with measured and unmeasured confounding in observational studies, we developed a unified approach to using an instrumental variable in more flexible ways to evaluate treatment effects. The approach is based on an instrumental propensity score conditional on baseline variables, which can then be incorporated in matching, regression, subclassification, or weighting along with various parametric, semiparametric, or nonparametric methods for the assessment of treatment effects. Therefore, the application of the instrumental propensity score allows different methods for outcome effect evaluations in addition to standard 2-stage least square models while controlling for unmeasured confounders. Several properties of the instrumental propensity score are discussed. The approach is then illustrated using subclassification along with a semiparametric density ratio model and empirical likelihood. This method allows us to evaluate distributional and subgroup treatment effects in addition to the overall average treatment effect. Simulation studies showed that the method works well. We applied our method to a study of the effects of attending a Catholic school versus a public school and found that attending a Catholic school had significant beneficial effects on subsequent wages among a subgroup of subjects.

Comments on "covariance adjustments for the analysis of randomized field experiments".

Richard Berk, Emil Pitkin, Lawrence Brown, Andreas Buja, Edward George, and Linda Zhao (2014) have written a valuable Evaluation Review paper on regression adjustment in randomized experiments. I’ve long been a fan of Berk’s critical writings on regression and meta-analysis (Berk 2004, 2007; Berk and Freedman 2003), and I recently recommended Berk, Brown, et al.’s (2014) helpful Sociological Methods and Research paper to a colleague teaching a course on regression. Also, I am grateful to Berk for sending me kind and constructively critical comments on Lin (2013) after it went to press. In my reply to his e-mail, I shared an informal essay (Lin 2012a, 2012b) discussing regression adjustment in practice. We are in agreement on many points.

A Note on Close Elections and Regression Analysis of the Party Incumbency Advantage

Much research has recently been devoted to understanding the effects of party incumbency following close elections, typically using a regression discontinuity design. Researchers have demonstrated that close elections in the United States may systematically favor certain types of candidates, and that a research design that focuses on close elections may therefore be inappropriate for estimation of the incumbency advantage. We demonstrate that any issues raised with the study of close elections may be equally applicable to the ordinary least squares analysis of electoral data, even when the sample contains all elections. When vote share is included as part of a covariate control strategy, the estimate produced by an ordinary least squares regression that includes all elections either exactly reproduces or approximates the regression discontinuity estimate.