Larry K. Irvin

The School-Wide Evaluation Tool (SET): A Research Instrument for Assessing School-Wide Positive Behavior Support.

Schools throughout the country are now encouraged to implement school-wide positive behavior support (PBS) procedures as a way to improve their behavioral climate, safety, and social culture. Research is needed to determine (a) the extent to which schools already use school-wide PBS, (b) if training and technical assistance efforts result in change in the use of school-wide PBS procedures, and (c) if use of these procedures is related to valued change in safety, social culture, and behavior within schools. To address these questions, researchers need a metric for assessing implementation of school-wide PBS practices. The School-Wide Evaluation Tool (SET; Sugai, Lewis-Palmer, Todd, & Horner, 2001) was created to provide a rigorous measure of primary prevention practices within school-wide behavior support. In this article, the authors describe the SET and document its psychometric characteristics. The results of their study suggest that the SET is a valid, reliable measure that can be used to assess the impact of school-wide training and technical assistance efforts. The SET should also be useful in formal analyses of the relationship between use of school-wide PBS and changes in social and academic outcomes.

Validity of Office Discipline Referral Measures as Indices of School-Wide Behavioral Status and Effects of School-Wide Behavioral Interventions

Office discipline referrals (ODRs) are widely used by school personnel to evaluate student behavior and the behavioral climate of schools. In this article, the authors report the results of a review of the relevant literature to evaluate the validity of ODR data as indices of school-wide behavioral climate, the effects of school-wide behavioral interventions, and differing behavior support needs across schools. They used Messicks unified approach to validity by focusing on examples of evidence for empirical and ethical foundations of interpretations, uses, and social consequences of ODR measures at the school-wide level. The authors also discuss ongoing issues, study limitations, and related recommendations for interpretations and uses of ODR measures as school-wide indices, based on the existing literature.

Using Office Discipline Referral Data for Decision Making About Student Behavior in Elementary and Middle Schools An Empirical Evaluation of Validity

In this evaluation we used Messicks construct validity as a conceptual framework for an empirical study assessing the validity of use, utility, and impact of office discipline referral (ODR) measures for data-based decision making about student behavior in schools. The Messick approach provided a rubric for testing the fit of our theory of use of ODR measures with empirical data on reported and actual use. It also facilitated our demonstration of Messicks principle that validation is both a developmental and an ongoing collaborative process among developers of educational and psychological measures, researchers interested in theories underlying such measures, and educators who use these measures in professional practice. We used a single-group, nonexperimental evaluation design to survey users of ODR measures from the standardized School Wide Information System in 22 elementary and 10 middle schools; respondents included school staff involved exclusively with data entry and staff actively involved in data-based decision making. Results were highly consistent across 2 independent data sources—electronic database records of actual access of summaries of ODR measures and self-report survey responses regarding frequencies and types of uses of ODR measures for decision making. Results indicated that ODR measures are regularly used for a variety of types of data-based decision making and are regarded as both efficient and effective for those purposes. We discuss implications of our SWIS ODR validity evaluation results within the context of the Messick framework.

A Construct Score Approach to the Assessment of Social Competence Rationale, Technological Considerations, and Anticipated Outcomes

This article provides a selected review of the knowledge base on social competence in children. Using the existing literature on social competence as a point of departure, a case is made that (a) a social competence construct score approach to assessing social competence is needed, (b) theoretical and empirical advances in the social competence knowledge base and in microcomputer video assessment technology make such an approach feasible, and (c) direct assessment of childrens knowledge and perceptions of key social situations, tasks, and skills can now be accomplished with far greater precision and validity than heretofore. These developments now make it possible to profile and aggregate childrens social competence across four important domain areas commonly sampled in assessing social competence (i.e., sociometric procedures, direct observations in natural settings, parent and teacher ratings, and direct assessments of childrens knowledge and perceptions of social stimuli). Social competence construct scores, developed at both global and specific levels, can be used to construct such profiles.

Schoolwide Social-Behavioral Climate, Student Problem Behavior, and Related Administrative Decisions: Empirical Patterns from 1,510 Schools Nationwide.

Office discipline referral (ODR) data provide useful information about problem behavior and consequence patterns, social-behavioral climates, and effects of social-behavioral interventions in schools. The authors report patterns of ODRs and subsequent administrative decisions from 1,510 schools nationwide that used the School-Wide Information System in 2005—2006. Sample demographics differed from key demographics for schools nationwide on school enrollment, socioeconomic status, and race and ethnicity. Descriptive analyses showed over 88% of elementary grade students with zero or one ODR for the academic year, 72% for middle schools, and 67% for high schools. ODRs were most likely to be generated from classrooms and more likely to be related to peer-directed problem behavior in elementary schools, student-adult interactions in middle schools, and tardiness and skipping in high schools. Major consequences were suspensions and detentions for middle and high school students and varied for elementary students. The discussion focuses on the utility of ODR databases and implications of the reported patterns.

Building Inclusive School Cultures Using School-Wide Positive Behavior Support: Designing Effective Individual Support Systems for Students with Significant Disabilities

School-wide positive behavior support (SWPBS) is a systematic and effective approach for broadly improving student behavior across school environments. SWPBS includes data-based strategies for supporting all students along a continuum of need and intensity based on a three-tiered model of prevention. Students with the most significant behavioral challenges are provided with assessment-based, individualized supports. To date, considerable evaluation research has demonstrated the benefits of SWPBS, documenting behavioral improvements using the whole school as the unit of analysis. Notably, less evaluation research has focused on the effects of SWPBS on the behavior of individual students with the most significant disabilities. In this paper, we describe SWPBS with an emphasis on the conceptual and procedural elements that are intended to benefit the full range of students within a school, with a particular focus on those students with the most intense needs. We discuss the SWPBS process, provide case illustrations, and call for additional research on the inclusion of students with significant disabilities with all applications of SWPBS.

Social Skills in School-Age Children and Youth: Issues and Best Practices in Assessment and Intervention.

Social skills allow individuals to develop positive relationships with others; cope successfully with the behavioral demands of specific settings; and communicate desires, needs, and preferences effectively. They also provide a foundation for competent performance in a range of academic, personal, vocational, and community contexts, Effective communication is an essential component of appropriate social behavior. This article provides definitions and conceptualizations of social skills and social competence within academic contexts. It also describes social skills assessment and intervention procedures for school-age children and youth with and without disabilities.

The burden of proof: validity as improvement of instructional practice

This article examines the validity of three recent efforts to improve instructional assessment in mathematics for special education students. In our view, to claim that an assessment is valid for informing instruction, researchers must present evidence indicating that the assessment information is in fact used by teachers as intended and that it results in improved student learning. We use examples from the three research efforts to illustrate these points and suggest fruitful areas for further research. This discussion highlights crucial relationships among constructs underlying effective practice and assessment in the context of the mathematics reform movement.

Validating the Construct of Coercion in Family Routines: Expanding the Unit of Analysis in Behavioral Assessment with Families of Children with Developmental Disabilities

We conducted an observational study of parent-child interaction in home activity settings (routines) of families raising young children with developmental disabilities and problem behavior. Our aim was to empirically investigate the construct validity of coercion in typical but unsuccessful family routines. The long-term goal was to develop an expanded ecological unit of analysis that may contribute to sustainable behavioral family intervention. Ten children with autism and/or mental retardation and their families participated. Videotaped observations were conducted in typical but unsuccessful home routines. Parent-child interaction in routines was coded in real time and sequential analyses were conducted to test hypotheses about coercive processes. Following observation, families were interviewed about the social validity of the construct. Results confirmed the presence of statistically significant, attention-driven coercive processes in routines in which parents were occupied with non-child-centered tasks. Results partially confirmed the presence of escape-driven coercive processes in routines in which parent demands are common. Additional analysis revealed an alternative pattern with greater magnitude. Family perspectives suggested the social validity of the construct. Results are discussed in terms of preliminary, partial evidence for coercive processes in routines of families of children with developmental disabilities. Implications for behavioral assessment and intervention design are discussed.

Application of Economic Analysis to School-Wide Positive Behavior Support (SWPBS) Programs.

The authors discuss how to use economic techniques to evaluate educational programs and show how to apply basic cost analysis to implementation of school-wide positive behavior support (SWPBS). A description of cost analysis concepts used for economic program evaluation is provided, emphasizing the suitability of these concepts for evaluating educational programs. The authors also describe the specific data and measurement and analytic procedures that cost analysis evaluation requires. The concepts are then applied in a case study showing a cost analysis of SWPBS. Implications are provided for extending the cost analysis case study into evaluation of cost-effectiveness and/or cost—benefit economic analyses of program success.