Georgine M. Pion

What's Behind the Numbers? Definitional Issues in Counting the Homeless*

Abstract Any estimate of the number of homeless persons involves several definitional issues, including the underlying conceptual definition of “homelessness,” the intended use of and rationale for the count, how the conceptual definition is translated into operational procedures, and methodological choices. These issues are at least partially responsible for the variation in existing estimates of homeless individuals. To best serve multiple constituencies and demands for information, the adoption of a broad definition is recommended. Furthermore, it is necessary to develop and report estimates for specific policy‐relevant subgroups of homeless persons, as well as to develop a family of research studies that can yield an understanding of the conditions facing the homeless and identify “markers” for those who are at risk of becoming homeless.

EVALUATION PLANNING FOR AN INNOVATIVE CHILDREN'S MENTAL HEALTH SYSTEM

Abstract We describe an evaluation of an innovative mental health system for children and adolescents. The Ft. Bragg Demonstration was developed to provide a complete continuum of care to children receiving health care benefits through CHAMPUS. The evaluation, jointly funded by the State of North Carolina and the NIMH, is designed to evaluate the implementation, service quality, clinical outcomes, and costs of this new system of care. Over a 5-year period, approximately 1,100 children and families will be studied in a four-wave longitudinal study at Ft. Bragg and two comparison sites. This project should provide important information about whether these new systems work better than the traditional array of services.

Constructing Quantitative Implementation Scales From Categorical Services Data Examples From a Multisite Evaluation

Like measures of outcome, measures of implementation are most useful and analytically powerful when measured at client-level and are quantitative. However, high-quality, individual-level, quantitative service utilization data can be expensive, intrusive, or otherwise impractical to obtain. Cruder data—for example, presence versus absence of a given service—are often more feasible to collect, as well as more likely to be available. Consequently, evaluators can benefit by finding ways to better exploit such data at the analysis phase to compensate for shortcomings at the collection phase. This article documents one such instance in which this was done. Specifically, it describes how quantitative, client-level implementation scales were derived from qualitative (categorical) data and used to support a cross-site synthesis of implementation and outcome analyses in a multisite evaluation. It also suggests additional scenarios in which quantitative implementation scales might be derived form qualitative services data.

The Burroughs Wellcome Career Award in the Biomedical Sciences: challenges to and prospects for estimating the causal effects of career development programs.

The Career Award in Biomedical Sciences (CABS) was launched by the Burroughs Wellcome Fund to assist talented scientists to cross the “bridge” that connects postdoctoral training with independent investigator status. This study examines the causal effects of CABS, using both conventional methods and a newer method (propensity score analysis) in attempting to account for selection biases (e.g., selecting the “best and the brightest”) in estimating the awards impact. The results indicated that grantees outperformed unsuccessful applicants in obtaining a faculty position, competing for an R01 grant, and publishing in prestigious journals. However, it was nearly impossible to remove all the effects of selection bias from estimates of program effects. Because selection bias is a pervasive problem in assessing these programs, recommendations for improving evaluations of similar merit-based programs are offered.