Laura G. Hill

Revisiting the Retrospective Pretest.

The purpose of the present study was to examine a common practice in some areas of program evaluation, the retrospective pretest, and to present recommendations regarding its use. The authors review literature to emphasize first, that bias is likely in both prospective and retrospective ratings, and second, that under some circumstances, retrospective pretests may introduce greater bias than traditional pretests. The authors examined data from 100 participants who attended a family program at 15 sites. Results supported the hypothesis that items on which parents were asked to endorse socially desirable parenting behaviors resulted in greater discrepancies between prospective and retrospective ratings. Effect sizes calculated from prospective ratings were comparable with published effect sizes from research trials of the same program. The authors conclude that replacing traditional with retrospective pretests does not eliminate bias. They recommend traditional pretests for examination of program effects and retrospective pretests for examination of subjective experiences of program-related change.

A Focused Approach to Assessing Program Fidelity

The primary goals of the present study were 1) to explore the types and frequencies of adaptation reported by facilitators; 2) to document their reasons for making those adaptations; and 3) to propose an efficient approach to the study of prevention program fidelity based on interview data. We interviewed 42 program facilitators involved in a large-scale dissemination about their implementation of a community-based prevention program. Interview questions addressed facilitators’ attitudes about program fidelity and the various types of changes, deletions, and additions they made. Although nearly all facilitators reported that fidelity to program curriculum was important, most also reported adapting the program. The most frequent reason facilitators gave for adaptation was deleting or changing material because they ran out of time. We employed a method used in business and health care settings (the Pareto principle, or the “law of the vital few”) to identify the minority (30%) of types of adaptation that accounted for a majority (70%) of all adaptations reported. Similarly, we identified the minority of reasons (25%) for adaptation that accounted for a majority (75%) of reasons reported. This approach helps to focus the assessment of fidelity on those adaptations that occur with high frequency, since low-frequency events are unlikely to have a significant impact on large-scale outcomes. High-frequency adaptations can then be targeted to determine their effect on outcomes; to address in training; and to assess on an ongoing basis for continuous quality improvement.

Effectiveness of Early Screening for Externalizing Problems: Issues of Screening Accuracy and Utility

Accurate, early screening is a prerequisite for indicated interventions intended to prevent development of externalizing disorders and delinquent behaviors. Using the Fast Track longitudinal sample of 396 children drawn from high-risk environments, the authors varied assumptions about base rates and examined effects of multiple-time-point and multiple-rater screening procedures. The authors also considered the practical import of various levels of screening accuracy in terms of true and false positive rates and their potential costs and benefits. Additional research is needed to determine true costs and benefits of early screening. However, the results indicate that 1st grade single- and multiple-rater screening models effectively predicted externalizing behavior and delinquent outcomes in 4th and 5th grades and that early screening is justified.

Media exposure, internalization of the thin ideal, and body dissatisfaction: Comparing Asian American and European American college females

Internalization of the thin ideal mediates the media exposure-body dissatisfaction relation in young adult European American females. There is little related research on Asian Americans. We used structural equations modeling to test: (1) whether media exposure was associated with body dissatisfaction in Asian American young adult females, (2) internalization of the thin ideal mediated any such association, and (3) whether the mediational model provided equivalent fit for European American and Asian American samples. Participants were 287 college females (154 Asian Americans, 133 European Americans). Internalization of the thin ideal explained the media exposure-body dissatisfaction association equally well for both groups. Results suggest that Asian Americans may be employing unhealthy weight control behaviors, and may be prone to developing eating disorders, at rates similar to European American young adult females. Clinicians need to screen carefully for body dissatisfaction, unhealthy weight control behaviors, and eating disorders in Asian American females.

Research Priorities for Economic Analyses of Prevention: Current Issues and Future Directions

In response to growing interest in economic analyses of prevention efforts, a diverse group of prevention researchers, economists, and policy analysts convened a scientific panel, on “Research Priorities in Economic Analysis of Prevention” at the 19th annual conference of the Society for Prevention Research. The panel articulated four priorities that, if followed in future research, would make economic analyses of prevention efforts easier to compare and more relevant to policymakers and community stakeholders. These priorities are: (1) increased standardization of evaluation methods, (2) improved economic valuation of common prevention outcomes, (3) expanded efforts to maximize evaluation generalizability and impact as well as (4) enhanced transparency and communicability of economic evaluations. In this paper, we define three types of economic analyses in prevention, provide context and rationale for these four priorities as well as related sub-priorities, and discuss the challenges inherent in meeting them.

Secrecy and Parent-Child Communication During Middle Childhood: Associations with Parental Knowledge and Child Adjustment

Objective. This study examines the reciprocal associations among child secrecy, parent-child communication, parental knowledge of childrens activities, and child social adjustment in elementary school children. Design. Third and fourth graders (N = 203) completed questionnaires about secrecy, parent-child communication, and parental knowledge; classroom teachers completed measures of child adjustment. The procedure was repeated one year later. Results. Childrens secrecy was negatively associated with parental knowledge one year later, and lower parent-child communication predicted increased secrecy over time. Children who reported greater parental knowledge were viewed by teachers as more socially competent one year later; conversely, children who reported more secrecy were later rated as less socially competent and more oppositional and aggressive. Finally, children who were initially rated as less socially competent and more relationally aggressive reported more communication one year later. Conclusions. Childrens secrecy has implications for their adjustment during the middle childhood years. Contrary to evidence from studies of parental knowledge during adolescence, in middle childhood poor adjustment may result in increased parent-child communication.

Children's Appraisal of Moderately Stressful Situations.

This study investigated 2 questions: (1) do children show consistent styles of appraisal across a range of moderately stressful events?, and (2) what are the adjustment and parenting correlates of individual differences in childrens appraisal style? Ninety-nine 3rd though 5th grade children and their mothers participated. For each of 6 vignettes involving moderately stressful situations, children responded to 10 items assessing childrens appraisal of these events. Mothers completed a self-report measure of parenting practices and children reported on their psychological adjustment. Participants showed moderate levels of consistency in their appraisal of the events across situations, and 6 appraisal styles were identified that reflected the nature of appraisal aggregated across situations. These styles showed meaningful patterns of association with child psychological symptoms and parenting practices. Children showing the victim appraisal style reported the highest levels of anxiety and school dislike, whereas children showing the inconvenience and take responsibility styles reported the lowest level of conduct problems. Parenting style was associated with the appraisal style of boys but not girls. Specifically, boys of authoritative mothers were more likely than other boys to show the inconvenience and take responsibility appraisal styles and less likely to show the victim style.

Individual differences in appraisal of minor, potentially stressful events: A cluster analytic approach

Two studies explored individual differences in appraisal of minor, potentially stressful events. Previous research on appraisal has focused on one or two appraisal dimensions within specific situations rather than on the full range of appraisals or on the stability of appraisal across situations. Goals of the present studies were: (1) to explore stability of individual differences in appraisal across situations; (2) to identify individual differences in general appraisal styles; and (3) to examine how appraisal styles are related to personality constructs. Appraisal was moderately stable across situations and individual differences in appraisal style were related to relevant personality constructs. Primary threat appraisals and secondary appraisals of blame and manageability emerged as particularly important in distinguishing the appraisal styles. Implications of the results for the development of psychopathology are considered.

Detecting selection effects in community implementations of family-based substance abuse prevention programs.

To calculate valid estimates of the costs and benefits of substance abuse prevention programs, selection effects must be identified and corrected. A supplemental comparison sample is typically used for this purpose, but in community-based program implementations, such a sample is often not available. We present an evaluation design and analytic approach that can be used in program evaluations of real-world implementations to identify selection effects, which in turn can help inform recruitment strategies, pinpoint possible selection influences on measured program outcomes, and refine estimates of program costs and benefits. We illustrate our approach with data from a multisite implementation of a popular substance abuse prevention program. Our results indicate that the programs participants differed significantly from the population at large.

Selection effects and prevention program outcomes.

A primary goal of the paper is to provide an example of an evaluation design and analytic method that can be used to strengthen causal inference in nonexperimental prevention research. We used this method in a nonexperimental multisite study to evaluate short-term outcomes of a preventive intervention, and we accounted for effects of two types of selection bias: self-selection into the program and differential dropout. To provide context for our analytic approach, we present an overview of the counterfactual model (also known as Rubins causal model or the potential outcomes model) and several methods derived from that model, including propensity score matching, the Heckman two-step approach, and full information maximum likelihood based on a bivariate probit model and its trivariate generalization. We provide an example using evaluation data from a community-based family intervention and a nonexperimental control group constructed from the Washington State biennial Healthy Youth Survey (HYS) risk behavior data (HYS n = 68,846; intervention n = 1,502). We identified significant effects of participant, program, and community attributes in self-selection into the program and program completion. Identification of specific selection effects is useful for developing recruitment and retention strategies, and failure to identify selection may lead to inaccurate estimation of outcomes and their public health impact. Counterfactual models allow us to evaluate interventions in uncontrolled settings and still maintain some confidence in the internal validity of our inferences; their application holds great promise for the field of prevention science as we scale up to community dissemination of preventive interventions.