Michael J. Camasso

Modeling the reliability and predictive validity of risk assessment in child protective services

Abstract In a time of shrinking resources policy makers and administrators in Child Protective Services are increasingly turning to tools such as structured risk assessment to manage service demand. The reliability and predictive validity of risk assessment is questionable, however, and concerns continue about the validity of using lists of explicit criteria in protective services decision-making. In this research the issues of reliability and validity are addressed using an explicated confirmatory factor analysis model. A sample of 239 cases that included 432 children brought to CPS attention for allegations of physical abuse, neglect and child/family problems are evaluated for risk of abuse or neglect using the Washington State Risk Assessment Matrix (WARM). The study employed a three-wave panel design. Results show that a widely used risk assessment instrument exhibits high levels of measurement error and increasing stability over time, which limit the instruments capacity to predict new allegations of abuse and neglect. Measurement error reduces the instruments reliability while stability, in light of changes in allegation status and service intensity, reveals a consistency or stiffness that weakens predictive validity. Recommendations are offered for constructing risk assessments that are both psychometrically sound and diagnostically useful.

Practitioner productivity and the product content of medical care in publicly supported health centers

The productivity-patient care content relationship in general practice and primary health care has never been a popular topic among medical practitioners. Yet the time a physician spends with patients or the number of patients treated in a fixed time interval would appear to be a critical factor in the content of this care. While current research has demonstrated a clear effect with respect to psychosocial care, the evidence with respect to technical care remains equivocal. The purpose of this study is to assess how physician production--measured as the number of patients seen per hour--affects the technical care performance of preventive and well care in 6 major patient management areas. The analysis uses data collected from 15 publicly supported, primary care centers in Pennsylvania, U.S.A. The care received by 1424 patients over a seven month period (4695 medical encounters) is reflected in the analysis; the work of 64 full time physicians is also represented. Using both linear and piecewise regression techniques, the analysis uncovered evidence that production levels do influence the performance of medical care procedures. The strength, direction and functional form of the relationship, however, depends upon the specific medical component under scrutiny. The impact of encounters per hour is especially telling for the provision of medical history items and preventive care directed at the female patients. The implications of these findings are discussed and inferences about the physician style dynamics which might link productivity and consultation content are presented.

Risk assessment in child protective services: a canonical analysis of the case management function

This study examined the relationship between levels of risk and the patterns of service intervention in child protective services. A stratified, random sample of 239 cases from the New Jersey Division of Youth and Family Services was examined using canonical correlation techniques. Risk was measured using the Washington Risk Assessment Matrix while case management and service strategies were recorded from case records. Principal results indicate that three distinctive risk profiles operate in the data: older children with behavior problems, children from disadvantaged households, and children with an unemployed parent. Each can be linked to a distinctive service intervention pattern. Implications of these results for workflow management and workload are discussed as well as the implications for future research in the areas of risk assessment and case management.

New Jersey’s Family Cap Experiment: Do Fertility Impacts Differ by Racial Density?

Using experimental design, this research examines the impact of the nation’s first family cap policy, implemented in New Jersey, on the fertility behavior of welfare recipients. We explore whether the change in welfare parameters mandated by the policy induces differential impact among black, white, and Hispanic recipients. We examine if impacts are conditioned by racial‐ethnic group concentration. Results show that reduced welfare payments have contributed to a decline in births for black women. While we find a large response for blacks (on average), we find no response for blacks who live in geographic areas where they form a racial‐ethnic majority.

Decision making in child protective services: a risky business?

Child Protective Services (CPS) in the United States has received a torrent of criticism from politicians, the media, child advocate groups, and the general public for a perceived propensity to make decisions that are detrimental to children and families. This perception has resulted in numerous lawsuits and court takeovers of CPS in 35 states, and calls for profound restructuring in other states. A widely prescribed remedy for decision errors and faulty judgments is an improvement of risk assessment strategies that enhance hazard evaluation through an improved understanding of threat potentials and exposure likelihoods. We examine the reliability and validity problems that continue to plague current CPS risk assessment and discuss actions that can be taken in the field, including the use of receiver operating characteristic (ROC) curve technology to improve the predictive validity of risk assessment strategies.

Experimental evidence of welfare reform impact on clinical anxiety and depression levels among poor women

In this paper, we employ a classical experiment to determine if welfare reform causes poor women to experience increased levels of clinical anxiety and depression. We organize our analyses around the insights provided by lifestyle change and ecosocial theories of illness. Our data come from the New Jersey Family Development Program (FDP), one of the most highly publicized welfare experiments in the U.S. A sample of 8393 women was randomly assigned into two groups, one which stressed welfare-to-work and the other which offered traditional welfare benefits. These women were followed from 1992 through 1996 and information on clinical diagnoses was collected quarterly from physician treatment claims to the government Medicaid program. Our intention-to-treat estimates show that for short-term welfare recipients FDP decreased the prevalence of anxiety by 40% and increased depression by 8%. For black women both anxiety and depression diagnoses declined while Hispanic women experienced a 68% increase in depression. We discuss several public policy implications which arise from our work.

NEW JERSEY’S FAMILY CAP AND FAMILY SIZE DECISIONS: FINDINGS FROM A FIVE-YEAR EVALUATION

The causal relationship between the size of welfare benefits and the birth decisions of women on welfare has been explored in a number of studies using a variety of analytical approaches applied to vital statistics data, data from the Current Population Survey, the Panel Study of Income Dynamics, or similar survey data. These studies typically use non-experimental methods to relate differences in birth rates or birth decisions across states to differences in welfare benefits levels. Analyses of this type have been criticized on several grounds. Benefits across states may be correlated with unobserved interstate differences that may also be related to birth decisions. Very often, these studies measure the key independent variable, welfare benefits level, as the cash benefit guarantee under the Aid to Families with Dependent Children (AFDC) program for a household of fixed size, varying this amount by state of residence. Actual benefits paid will vary with household size, number of AFDC-eligible household members, other sources of income, and other factors.

Social Supports, Undesirable Life Events, and Psychological Distress in a Disadvantaged Population

One of the more consistent findings in community mental health research has been the inverse relationship between socioeconomic status and psychological distress. This study examines two processes that are believed to account for this relationship: undesirable life events and social supports. The findings indicate that undesirable life events contribute positively to levels of anxiety and depression, whereas social supports protect against these illnesses. Evidence for an indirect effect of stressors on illness through social supports was also found. Implications of this stress amplification process for social work intervention strategies are discussed.

How Family Caps Work: Evidence from a National Study

Many experts consider family caps to be the one welfare reform explicitly designed to reduce birth rates among poor unmarried women. Despite the operation of such caps in 23 states, research reports mixed findings on the policy’s effectiveness in reducing nonmarital births. This article presents results from a national study in which family caps were found to affect fertility behavior, but this effect was conditioned by a state’s willingness to use Medicaid to fund abortions deemed necessary to protect the health of the mother, as well as by the percentages of black and Hispanic women (ages 15–44 years) in a state’s general population. These factors, like family caps, serve to adjust the cost of bearing a child relative to the costs of contraception use and the use of abortions.

Beyond Intention to Treat Analysis in Welfare-to-Work Studies

Abstract Using a sample of 2072 women who participated in New Jerseys Family Development Program experiment during 1992–1996, we compare three welfare-to-work strategieslabor force attachment (LFA), human capital investment (HCI), and a mixed strategyemploying traditional effectiveness analysis within an experimental design and an efficacy approach which controls for selective participation by experimental and control subjects. Controlling for selective participation, we find that each year of participation in LFA increases the probability of employment by about 3 percent. In our earnings analyses we find that HCI participants earn about