Radha Jagannathan

Modeling the reliability and predictive validity of risk assessment in child protective services

Abstract In a time of shrinking resources policy makers and administrators in Child Protective Services are increasingly turning to tools such as structured risk assessment to manage service demand. The reliability and predictive validity of risk assessment is questionable, however, and concerns continue about the validity of using lists of explicit criteria in protective services decision-making. In this research the issues of reliability and validity are addressed using an explicated confirmatory factor analysis model. A sample of 239 cases that included 432 children brought to CPS attention for allegations of physical abuse, neglect and child/family problems are evaluated for risk of abuse or neglect using the Washington State Risk Assessment Matrix (WARM). The study employed a three-wave panel design. Results show that a widely used risk assessment instrument exhibits high levels of measurement error and increasing stability over time, which limit the instruments capacity to predict new allegations of abuse and neglect. Measurement error reduces the instruments reliability while stability, in light of changes in allegation status and service intensity, reveals a consistency or stiffness that weakens predictive validity. Recommendations are offered for constructing risk assessments that are both psychometrically sound and diagnostically useful.

Risk assessment in child protective services: a canonical analysis of the case management function

This study examined the relationship between levels of risk and the patterns of service intervention in child protective services. A stratified, random sample of 239 cases from the New Jersey Division of Youth and Family Services was examined using canonical correlation techniques. Risk was measured using the Washington Risk Assessment Matrix while case management and service strategies were recorded from case records. Principal results indicate that three distinctive risk profiles operate in the data: older children with behavior problems, children from disadvantaged households, and children with an unemployed parent. Each can be linked to a distinctive service intervention pattern. Implications of these results for workflow management and workload are discussed as well as the implications for future research in the areas of risk assessment and case management.

New Jersey’s Family Cap Experiment: Do Fertility Impacts Differ by Racial Density?

Using experimental design, this research examines the impact of the nation’s first family cap policy, implemented in New Jersey, on the fertility behavior of welfare recipients. We explore whether the change in welfare parameters mandated by the policy induces differential impact among black, white, and Hispanic recipients. We examine if impacts are conditioned by racial‐ethnic group concentration. Results show that reduced welfare payments have contributed to a decline in births for black women. While we find a large response for blacks (on average), we find no response for blacks who live in geographic areas where they form a racial‐ethnic majority.

Decision making in child protective services: a risky business?

Child Protective Services (CPS) in the United States has received a torrent of criticism from politicians, the media, child advocate groups, and the general public for a perceived propensity to make decisions that are detrimental to children and families. This perception has resulted in numerous lawsuits and court takeovers of CPS in 35 states, and calls for profound restructuring in other states. A widely prescribed remedy for decision errors and faulty judgments is an improvement of risk assessment strategies that enhance hazard evaluation through an improved understanding of threat potentials and exposure likelihoods. We examine the reliability and validity problems that continue to plague current CPS risk assessment and discuss actions that can be taken in the field, including the use of receiver operating characteristic (ROC) curve technology to improve the predictive validity of risk assessment strategies.

Experimental evidence of welfare reform impact on clinical anxiety and depression levels among poor women

In this paper, we employ a classical experiment to determine if welfare reform causes poor women to experience increased levels of clinical anxiety and depression. We organize our analyses around the insights provided by lifestyle change and ecosocial theories of illness. Our data come from the New Jersey Family Development Program (FDP), one of the most highly publicized welfare experiments in the U.S. A sample of 8393 women was randomly assigned into two groups, one which stressed welfare-to-work and the other which offered traditional welfare benefits. These women were followed from 1992 through 1996 and information on clinical diagnoses was collected quarterly from physician treatment claims to the government Medicaid program. Our intention-to-treat estimates show that for short-term welfare recipients FDP decreased the prevalence of anxiety by 40% and increased depression by 8%. For black women both anxiety and depression diagnoses declined while Hispanic women experienced a 68% increase in depression. We discuss several public policy implications which arise from our work.

NEW JERSEY’S FAMILY CAP AND FAMILY SIZE DECISIONS: FINDINGS FROM A FIVE-YEAR EVALUATION

The causal relationship between the size of welfare benefits and the birth decisions of women on welfare has been explored in a number of studies using a variety of analytical approaches applied to vital statistics data, data from the Current Population Survey, the Panel Study of Income Dynamics, or similar survey data. These studies typically use non-experimental methods to relate differences in birth rates or birth decisions across states to differences in welfare benefits levels. Analyses of this type have been criticized on several grounds. Benefits across states may be correlated with unobserved interstate differences that may also be related to birth decisions. Very often, these studies measure the key independent variable, welfare benefits level, as the cash benefit guarantee under the Aid to Families with Dependent Children (AFDC) program for a household of fixed size, varying this amount by state of residence. Actual benefits paid will vary with household size, number of AFDC-eligible household members, other sources of income, and other factors.

How Family Caps Work: Evidence from a National Study

Many experts consider family caps to be the one welfare reform explicitly designed to reduce birth rates among poor unmarried women. Despite the operation of such caps in 23 states, research reports mixed findings on the policy’s effectiveness in reducing nonmarital births. This article presents results from a national study in which family caps were found to affect fertility behavior, but this effect was conditioned by a state’s willingness to use Medicaid to fund abortions deemed necessary to protect the health of the mother, as well as by the percentages of black and Hispanic women (ages 15–44 years) in a state’s general population. These factors, like family caps, serve to adjust the cost of bearing a child relative to the costs of contraception use and the use of abortions.

Beyond Intention to Treat Analysis in Welfare-to-Work Studies

Abstract Using a sample of 2072 women who participated in New Jerseys Family Development Program experiment during 1992–1996, we compare three welfare-to-work strategieslabor force attachment (LFA), human capital investment (HCI), and a mixed strategyemploying traditional effectiveness analysis within an experimental design and an efficacy approach which controls for selective participation by experimental and control subjects. Controlling for selective participation, we find that each year of participation in LFA increases the probability of employment by about 3 percent. In our earnings analyses we find that HCI participants earn about

Flying personal planes: Modeling the airport choices of general aviation pilots using stated preference methodology

163 less per year of exposure to this strategy.

New Jersey's Family Cap and Welfare Births: An Examination of Racial Differences in Fertility Within the Framework of Proximate Determinants

This study employed stated preference (SP) models to determine why general aviation pilots choose to base and operate their aircraft at some airports and not others. Thirteen decision variables identified in pilot focus groups and in the general aviation literature were incorporated into a series of hypothetical choice tasks or scenarios. The scenarios were offered within a fractional factorial design to establish orthogonality and to preclude dominance in any combination of variables. Data from 113 pilots were analyzed for individual differences across pilots using conditional logit regression with and without controls. The results demonstrate that some airport attributes (e.g., full-range hospitality services, paved parallel taxiway, and specific types of runway lighting and landing aids) increase pilot utility. Heavy airport congestion and airport landing fees, on the other hand, decrease pilot utility. The importance of SP methodology as avehicle for modeling choice behavior and as an input into the planning and prioritization process is discussed. Actual or potential applications include the development of structured decisionmaking instruments in the behavioral sciences and in human service programs.