Peter Ji

Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination

Ever increasing demands for accountability, together with the proliferation of lists of evidence-based prevention programs and policies, led the Society for Prevention Research to charge a committee with establishing standards for identifying effective prevention programs and policies. Recognizing that interventions that are effective and ready for dissemination are a subset of effective programs and policies, and that effective programs and policies are a subset of efficacious interventions, SPR’s Standards Committee developed overlapping sets of standards. We designed these Standards to assist practitioners, policy makers, and administrators to determine which interventions are efficacious, which are effective, and which are ready for dissemination. Under these Standards, an efficacious intervention will have been tested in at least two rigorous trials that (1) involved defined samples from defined populations, (2) used psychometrically sound measures and data collection procedures; (3) analyzed their data with rigorous statistical approaches; (4) showed consistent positive effects (without serious iatrogenic effects); and (5) reported at least one significant long-term follow-up. An effective intervention under these Standards will not only meet all standards for efficacious interventions, but also will have (1) manuals, appropriate training, and technical support available to allow third parties to adopt and implement the intervention; (2) been evaluated under real-world conditions in studies that included sound measurement of the level of implementation and engagement of the target audience (in both the intervention and control conditions); (3) indicated the practical importance of intervention outcome effects; and (4) clearly demonstrated to whom intervention findings can be generalized. An intervention recognized as ready for broad dissemination under these Standards will not only meet all standards for efficacious and effective interventions, but will also provide (1) evidence of the ability to “go to scale”; (2) clear cost information; and (3) monitoring and evaluation tools so that adopting agencies can monitor or evaluate how well the intervention works in their settings. Finally, the Standards Committee identified possible standards desirable for current and future areas of prevention science as the field develops. If successful, these Standards will inform efforts in the field to find prevention programs and policies that are of proven efficacy, effectiveness, or readiness for adoption and will guide prevention scientists as they seek to discover, research, and bring to the field new prevention programs and policies.

The Relations among General and Race-Related Stressors and Psychoeducational Adjustment in Black Students Attending Predominantly White Institutions

Two hundred sixty African American college students attending predominantly White institutions completed the Black Student Stress Inventory (BSSI) and a measure of psychological adjustment; academic performance indices on a subsample were obtained via archival records. Principal components analysis of the BSSI yielded a three-factor solution: race-related stress, psychological/interpersonal stress, and academic stress. As predicted, general and race-related perceived stressors as measured by the BSSI were related to psychological distress; academic stress was specifically related to overall GPA. In fact, perceived academic stress accounted for more variance in 1st-year GPAthan did ACT scores. Implications of the findings as well as suggestions for future research are discussed.

Effects of the Positive Action programme on problem behaviours in elementary school students: A matched-pair randomised control trial in Chicago

This study examined the effects of the Positive Action (PA) programme in Chicago Public Schools on problem behaviours among a cohort of elementary school students from grade three through grade five. Using a matched-pair, randomised control design with 14 elementary schools, approximately 510 fifth-graders self-reported lifetime substance use, serious violence-related behaviour, and current bullying and disruptive behaviours. Three-level (i.e. students nested within schools within school pairs) overdispersed Poisson models were used to examine programme effects on the number of items endorsed for each of the four outcomes. Findings indicated that students in the intervention endorsed 31% fewer substance use behaviours (incidence rate ratio [IRR] = 0.69), 37% fewer violence-related behaviours (IRR = 0.63) and 41% fewer bullying behaviours (IRR = 0.59), respectively, compared to students in the control schools. Reduction in reported disruptive behaviours was of a similar magnitude (27%, IRR = 0.73), but was not statistically significant. These results replicate findings of an earlier randomised trial of the PA programme and extend evidence of its effectiveness to youth attending large urban school systems.

Factors Influencing Middle and High Schools' Active Parental Consent Return Rates.

The authors examined factors influencing the return rates for attempting to collect active parental consent forms from 21,123 students in the 7th through 10th grades in 41 middle and high schools. Overall return rates from middle schools were higher than from high schools. Schools that offered high levels of staff support for collecting consent forms had higher return rates. Procedures where the consent form was attached to a school form that parents had to complete and return to the school yielded the highest return rate. Implications for how researchers can obtain a high parent consent form return rate are discussed.

Using Social-Emotional and Character Development to Improve Academic Outcomes: A Matched-Pair, Cluster-Randomized Controlled Trial in Low-Income, Urban Schools

BACKGROUND School-based social-emotional and character development (SECD) programs can influence not only SECD but also academic-related outcomes. This study evaluated the impact of one SECD program, Positive Action (PA), on educational outcomes among low-income, urban youth. METHODS The longitudinal study used a matched-pair, cluster-randomized controlled design. Student-reported disaffection with learning and academic grades, and teacher ratings of academic ability and motivation were assessed for a cohort followed from grades 3 to 8. Aggregate school records were used to assess standardized test performance (for entire school, cohort, and demographic subgroups) and absenteeism (entire school). Multilevel growth-curve analyses tested program effects. RESULTS PA significantly improved growth in academic motivation and mitigated disaffection with learning. There was a positive impact of PA on absenteeism and marginally significant impact on math performance of all students. There were favorable program effects on reading for African American boys and cohort students transitioning between grades 7 and 8, and on math for girls and low-income students. CONCLUSIONS A school-based SECD program was found to influence academic outcomes among students living in low-income, urban communities. Future research should examine mechanisms by which changes in SECD influence changes in academic outcomes.

“Congratulations, You Have Been Randomized Into the Control Group!(?)”: Issues to Consider When Recruiting Schools for Matched-Pair Randomized Control Trials of Prevention Programs

BACKGROUND Recruiting schools into a matched-pair randomized control trial (MP-RCT) to evaluate the efficacy of a school-level prevention program presents challenges for researchers. We considered which of 2 procedures would be most effective for recruiting schools into the study and assigning them to conditions. In 1 procedure (recruit and match/randomize), we would recruit schools and match them prior to randomization, and in the other (match/randomize and recruitment), we would match schools and randomize them prior to recruitment. METHOD We considered how each procedure impacted the randomization process and our ability to recruit schools into the study. After implementing the selected procedure, the equivalence of both treatment and control group schools and the participating and nonparticipating schools on school demographic variables was evaluated. RESULTS We decided on the recruit and match/randomize procedure because we thought it would provide the opportunity to build rapport with the schools and prepare them for the randomization process, thereby increasing the likelihood that they would accept their randomly assigned conditions. Neither the treatment and control group schools nor the participating and nonparticipating schools exhibited statistically significant differences from each other on any of the school demographic variables. CONCLUSIONS Recruitment of schools prior to matching and randomization in an MP-RCT may facilitate the recruitment of schools and thus enhance both the statistical power and the representativeness of study findings. Future research would benefit from the consideration of a broader range of variables (eg, readiness to implement a comprehensive prevention program) both in matching schools and in evaluating their representativeness to nonparticipating schools.

An Academic Course That Teaches Heterosexual Students to be Allies to LGBT Communities: A Qualitative Analysis

A theory of lesbian, gay, bisexual, and transgender (LGBT) ally identity development integrated the following theories—multicultural counseling theory (MCT), self-concept formation theory (SCFT), and social identity theory (SIT)—to design a course to train heterosexual students to be allies to LGBT communities. Students participated in interviews and activities with LGBT persons, presented seminars on LGBT topics, and wrote papers about these experiences. An analysis of their reactions suggested that initially, students perceived themselves as lacking credibility to be allies. After interacting with LGBT communities, students gained the knowledge, attitudes, and skills they needed to be confident in supporting and advocating for LGBT persons.

Assessing Cigarette Sales Rates to Minors

Minors were asked questions regarding their cigarette-purchasing behavior and a group of volunteer minors were sent into stores to purchase cigarettes in a large Midwestern city. Results show that cigarette sales rates for two sampling data taken one year apart were 80% and 82%. In addition, minors reported they can easily purchase cigarettes from stores and vending machines. The results indicate that minors have easy access to cigarettes in a large Midwestern city.

Teachers’ Perceptions of School Organizational Climate as Predictors of Dosage and Quality of Implementation of a Social-Emotional and Character Development Program

Organizational climate has been proposed as a factor that might influence a school’s readiness to successfully implement school-wide prevention programs. The aim of this study was to evaluate the influence of teachers’ perceptions of three dimensions of school organizational climate on the dosage and quality of teacher implementation of Positive Action, a social-emotional and character development (SECD) program. The dimensions measured were teachers’ perceptions of (a) the school’s openness to innovation, (b) the extent to which schools utilize participatory decision-making practices, and (c) the existence of supportive relationships among teachers (teacher-teacher affiliation). Data from 46 teachers in seven schools enrolled in the treatment arm of a longitudinal, cluster-randomized, controlled trial were analyzed. Teacher perceptions of a school’s tendency to be innovative was associated with a greater number of lessons taught and self-reported quality of delivery, and teacher-teacher affiliation was associated with a higher use of supplementary activities. The findings suggest that perceptions of a school’s organizational climate impact teachers’ implementation of SECD programs and have implications for school administrators and technical assistance providers as they work to implement and sustain prevention programs in schools.

Hwa-Byung Among Middle-Aged Korean Women: Family Relationships, Gender-Role Attitudes, and Self-Esteem

We surveyed 395 Korean middle-aged women and examined how their perceptions of family relationships, gender-role attitudes, and self-esteem were associated with Hwa-Byung (HB; Korean anger syndrome). Our regression analyses revealed that participants who reported worse family relationship problems experienced more HB symptoms. Having profeminist, egalitarian attitudes toward womens gender roles was also associated with more HB symptoms. Self-esteem was not significantly associated with HB. Based on the results, we suggest that what is crucial to understanding HB is not how women evaluate themselves, but rather the level of stress caused by family relationship problems and their perception of womens roles.