Regina G. Hirn

Increasing Reliability of Direct Observation Measurement Approaches in Emotional and/or Behavioral Disorders Research Using Generalizability Theory.

Reliability of direct observation outcomes ensures the results are consistent, dependable, and trustworthy. Typically, reliability of direct observation measurement approaches is assessed using interobserver agreement (IOA) and the calculation of observer agreement (e.g., percentage of agreement). However, IOA does not address intraobserver agreement, or the withinrater agreement, analogues to internal consistency. One approach to assess both IOA and intraobserver agreement is generalizability theory. This paper describes generalizability theory and provides a case study example of using generalizability theory with direct observation measures in emotional and/or behavioral disorders research.

Teacher and Student Behaviors in the Contexts of Grade-Level and Instructional Grouping.

Teacher behaviors such as providing frequent opportunities to respond and feedback are known to promote student engagement. In addition, contextual variables such as grade-level and instructional grouping may affect engagement. However, less is known about teacher behavior in these instructional contexts. This study aimed to examine active instruction and engagement across elementary, middle, and high schools using a large database of direct classroom observations. Additional analyses were conducted to examine teacher and student variables within and between instructional groupings in elementary schools. Results revealed that whole-group instruction was the most frequently used format regardless of grade level, but individual opportunities to respond and active engagement were significantly higher during small-group lessons. Exploratory analyses of subsets of observations indicated that (a) group size affected frequency and duration of discrete teacher and student behaviors and (b) relationships among variables varied by context.

Descriptive Analysis of Teacher Instructional Practices and Student Engagement Among Adolescents With and Without Challenging Behavior

The purpose of this descriptive study was to examine teacher and student behavior in high school classrooms that included at least one student identified with challenging behavior. Across two school years and within the content areas of math, reading/English, social studies, and science, student/teacher dyads were directly observed in the typical classroom setting. Results are described in terms of overall teacher rates per minute and percentages of observed instructional practice. The degree to which teacher and student behaviors differed across students with and without challenging behaviors is described. Findings revealed relatively low rates of specific instructional practices, increased use of negative feedback for students with identified challenging behaviors, and variable levels of student engagement. Study limitations and areas for future research are discussed.

Addressing Challenging Behavior Considering the Logic of Probability

When dealing with children who exhibit challenging behaviors there are no known interventions that work for all students or at all times. Thus, intervention for these students is often implemented in a trial and error manner. This article provides a logic for considering probability as a factor in selecting strategies. Understanding that some interventions are more likely to work than others and the fact that adult behavior is the main impetus for change in student behavior, the key is in considering which teacher behaviors offer the highest probability of student success. Suggestions and examples are provided.

Affecting Disproportional Outcomes by Ethnicity and Grade Level: Using Discipline Data to Guide Practice in High School.

Teachers report that the behaviors that they are forced to deal with on a daily basis are not typically violent or intense but are frequent and usurp great amounts of instructional time. Office discipline referrals provide a well-established method of tracking student behavior problems across the school, allowing for deeper analysis of contextual predictors of problem behavior. In this case study, an average-size Midwestern high school created and implemented a referral system that identified disproportional referral rates for freshmen and minority students. This article describes the process by which faculty members identified contextual predictors and agreed on simple rules, routines, and physical arrangements across the school. The authors used continuous data analysis to refine prevention measures and resulting data showed steady and consistent decreases among freshmen and minority students. Implications for considering disproportionality in the context of future research are discussed.

The Relationship Between Teachers’ Implementation of Classroom Management Practices and Student Behavior in Elementary School:

Teachers’ classroom management practices have a direct impact on their students’ probability of success. Evidence-based classroom management practices include (a) active instruction and supervision of students (i.e., teaching), (b) opportunities for students to respond, and (c) feedback to students. In this study, we examined the degree to which teachers implemented evidence-based classroom management practices and whether there was a relationship between use of those teacher behaviors and students’ time engaged in instruction and rate of disruptions. Using latent class analysis, we identified four groups of teachers from 1,242 teacher–student dyads in 65 elementary schools, with one group of teachers demonstrating very low rates of classroom management practices. We then modeled the predictive relationship of being in classrooms with low rates of classroom management practices and student engagement and disruptive behavior within a multilevel framework. Results indicate that students in classrooms with low rates of classroom management practices were statistically significantly less engaged in instruction, whereas no differences in disruptions were found. The results are discussed within the context of prior research, students with or at risk for emotional and/or behavioral disorders, and study limitations.

Teacher Instruction as a Predictor for Student Engagement and Disruptive Behaviors.

Effective instruction is a critical predictor of student achievement. As students with exceptionalities such as emotional and behavioral disorders and learning disabilities, who typically struggle with academic achievement, spend increasing amounts of general education settings, the need for precise instructional behaviors becomes more imperative. The authors present the results of 1,197 systematic direct observations of teachers’ instructional behaviors and their effect on student engagement and disruption. Results indicate statistically significant correlations between teaching and student engagement and disruptions, although there is variability depending on the grade level (elementary school vs. high school). Limitations, implications, and directions for future practice are discussed.

Defining “Effectiveness” for Students with E/BD: Teacher, Instruction, and Management Variables

T he term ‘‘effectiveness’’ is an adjective denoting that a particular action or event is generally adequate to accomplish some purpose. Thus, bridge engineers are concerned with the effectiveness of specific structures and materials to handle weight, medical researchers are concerned with the effectiveness of a drug to cure a disease, and automobile manufacturers are concerned with the effectiveness of an engine technology to increase fuel efficiency. In each example, effectiveness is judged relative to the degree to which an outcome is accomplished. By definition, a clearly defined and measurable purpose is the necessary endpoint by which relative effectiveness is judged. That is, without the need to cure a disease there is no meaningful measure of the effectiveness of a drug. Still, any reasonable definition of effectiveness also must consider efficiency or the degree to which effort, expenses, and materials are required to achieve an outcome. Clearly, the pill that can be delivered inexpensively is preferred over a complex surgical procedure if both offer the same level of cure. In this sense, neither effectiveness nor efficiency can be judged without information about the other. However, maximum effectiveness comes at the cost of efficiency while maximum efficiency is worthless without effectiveness. Thus, it makes sense to (a) first ask what works (effectiveness), and (b) then ask what is simplest (efficiency). Although desired outcomes are typically obvious in the hard sciences (bridge holds X amount of weight, drug cures disease, engine more fuel efficient), the purpose for education is not always as clearly agreeable. While academic achievement, as measured by standardized tests, may be of utmost importance to one group or constituency, others will argue that outcomes such as self-esteem, freedom, or critical thinking are ultimately more important. Regardless of which of these outcomes is actually acceptable as the purpose of education, the issue of efficiency must be considered in any decisions about effectiveness. The purpose of this paper is to discuss how issues of effectiveness and efficiency are considered most logically in the field of education. More specifically, the focus is on the importance of these issues as they pertain to teaching, instruction, and management strategies for students with emotional and behavioral disorders (E/BD). The driving question behind this discussion is, What can happen in the classroom to provide the greatest probability of student success in both school and in life—in terms of both effectiveness and efficiency?

Teacher and student race as a predictor for negative feedback during instruction.

There is a long and persistent gap between the academic achievement of White and Black students in America’s schools. Further, Black students are suspended from school at a rate that is more than three times greater than White students. While there has been some suggestion that perhaps teacher-student racial matching may be part of a solution, the research does not currently provide adequate support for teacher race alone to be sufficiently effective. This study analyzed 41 Black and White teacher-student dyad mixes in elementary schools and another 41 in a high school to examine how teacher and student race interact in terms of teacher and student behaviors. While Black students were slightly more likely to be off-task and disruptive, both Black and White teachers were found to provide significantly more negative feedback to Black students regardless of their behavior. Implications for teacher practice and future study are discussed.

Effects of a Mathematics Fluency Program on Mathematics Performance of Students with Challenging Behaviors.

In the present study, we examined the effects of a fluency-building mathematics program called Great Leaps Math on fluency of basic addition mathematics facts zero to nine and word problem solving using a multiple probe design across participants. Three elementary students with challenging behaviors and mathematics difficulty participated in the study. Results showed that all three students increased their oral fluency of basic mathematics facts and word problem solving skills, with one student increasing written fluency and written generalization fluency (e.g., problems displayed vertically and horizontally).