Nathaniel P. von der Embse

Confirmation of Models for Interpretation and Use of the Social and Academic Behavior Risk Screener (SABRS)

The purpose of this investigation was to evaluate the models for interpretation and use that serve as the foundation of an interpretation/use argument for the Social and Academic Behavior Risk Screener (SABRS). The SABRS was completed by 34 teachers with regard to 488 students in a Midwestern high school during the winter portion of the academic year. Confirmatory factor analysis supported interpretation of SABRS data, suggesting the fit of a bifactor model specifying 1 broad factor (General Behavior) and 2 narrow factors (Social Behavior [SB] and Academic Behavior [AB]). The interpretive model was further supported by analyses indicative of the internal consistency and interrater reliability of scores from each factor. In addition, latent profile analyses indicated the adequate fit of the proposed 4-profile SABRS model for use. When cross-referenced with SABRS cut scores identified via previous work, results revealed students could be categorized as (a) not at-risk on both SB and AB, (b) at-risk on SB but not on AB, (c) at-risk on AB but not on SB, or (d) at-risk on both SB and AB. Taken together, results contribute to growing evidence supporting the SABRS within universal screening. Limitations, implications for practice, and future directions for research are discussed herein.

The influence of test-based accountability policies on teacher stress and instructional practices: a moderated mediation model

Abstract The present study examined the use of student test performance for merit pay and teacher evaluation as predictive of both educator stress and counterproductive teaching practices, and the moderating role of perceived test value. Structural equation modelling of data from a sample of 7281 educators in a South-eastern state in the United States supported the hypothesis that educators who perceived the test as an invalid measure of teaching effectiveness were more likely to report high levels of test stress and to use counterproductive teaching practices, including fear appeals, in an attempt to motivate students for test-taking. This study provides initial evidence for the hypothesised relationships of test-based accountability policy with teacher mental health and instructional practices. Implications for research and practice are discussed.

Technical Adequacy of the Social, Academic, and Emotional Behavior Risk Screener in an Elementary Sample

The purpose of this study was to evaluate the psychometric defensibility of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS): a quick and easy universal screener for behavioral and emotional risk. Elementary school teachers completed the SAEBRS with 346 students in Grades 3 to 5. Teachers also completed two criterion measures, including the Student Risk Screening Scale (SRSS) and the Student Internalizing Behavior Screener (SIBS). Additional extant behavioral and academic data sources were collected including office discipline referrals, suspensions, curriculum-based measurement scores, and statewide achievement test scores. Reliability analyses were indicative of the internal consistency of all four SAEBRS scales, whereas correlational analyses and Mann–Whitney–Wilcoxon tests supported the criterion-related and construct validity. Receiver operating characteristic curve analyses suggested each SAEBRS scale was associated with acceptable or optimal diagnostic accuracy. However, cut scores selected as most appropriate within each SAEBRS scale were found to differ from those identified in previous studies, potentially suggesting the influence of criterion outcome under consideration on SAEBRS diagnostic accuracy. Limitations and future directions for research are discussed, with emphasis on the need for continued examination of the extent of variability in SAEBRS cut score performance.

Initial Development and Factor Structure of the Educator Test Stress Inventory

With the proliferation of test-based accountability policies, educators and students alike are under pressure to improve test performance. However, little is known regarding the stress experienced by educators in response to these policies. The purpose of this article is to describe the initial development and validation of a new measure of stress associated with high-stakes testing. Psychometric properties were examined within a sample of 8,084 educators in a southeastern state in the United States. An exploratory and confirmatory factor analysis of the Educator Test Stress Inventory supported a bifactor model of teacher test stress, with one general factor of Total Teacher Stress and two narrow factors of Sources of Stress and Manifestations of Stress. This study is an important first step in establishing a reliable and valid measure of teacher stress and better understanding the impact of high-stakes testing and educational accountability policies. Implications for the assessment and intervention of teacher stress are discussed.

Latent Profile Analyses of Test Anxiety A Pilot Study

In an era of test-based accountability, there has been a renewed interest in understanding the relationship between test anxiety and test performance. The development and validation of test anxiety scales have grown with the rise of test anxiety research. Research is needed to critically examine the psychometric properties of these scales prior to widespread use. The purpose of this brief report is to demonstrate the use of latent profile analysis (LPA) to develop test anxiety profiles on the FRIEDBEN Test Anxiety Scale. LPA was performed using a sample of 1,133 students from five high schools in a Midwestern state. Results indicate three distinct test anxiety profiles (i.e., high, mid, and low anxiety profiles).

Measurement equivalence: A non-technical primer on categorical multi-group confirmatory factor analysis in school psychology ☆

Evidence-based interventions (EBIs) have become a central component of school psychology research and practice, but EBIs are dependent upon the availability and use of evidence-based assessments (EBAs) with diverse student populations. Multi-group confirmatory factor analysis (MG-CFA) is an analytical tool that can be used to examine the validity and measurement equivalence/invariance of scores across diverse groups. The objective of this article is to provide a conceptual and procedural overview of categorical MG-CFA, as well as an illustrated example based on data from the Social and Academic Behavior Risk Screener (SABRS) - a tool designed for use in school-based interventions. This article serves as a non-technical primer on the topic of MG-CFA with ordinal (rating scale) data and does so through the framework of examining equivalence of measures used for EBIs within multi-tiered models - an understudied topic. To go along with the illustrated example, we have provided supplementary files that include sample data, Mplus input code, and an annotated guide for understanding the input code (http://dx.doi.org/10.1016/j.jsp.2016.11.002). Data needed to reproduce analyses in this article are available as supplemental materials (online only) in the Appendix of this article.

Readying students to test: The influence of fear and efficacy appeals on anxiety and test performance

Educational accountability policies have led to a growth in the use of high-stakes examinations for a number of important educational decisions, including the evaluation of teacher effectiveness. As such, educators are under increasing pressure to raise student test performance. In an attempt to prepare students for a high-stakes exam, teachers often resort to using threat-based messages that focus on the negative consequences of test failure rather than messages highlighting students’ ability or expectation for high performance. However, the relative influence of teacher messaging (threat-based or facilitating) under different testing conditions is unknown. The present investigation examined the use of fear and efficacy appeals with 487 university students. Anxiety, motivation, and test performance data were collected during a typical, lower stakes testing situation and a higher-stakes, final course examination. A two-way mixed ANOVA and a mediation analysis were used to examine between (i.e. fear and efficacy appeals) and within (i.e. different testing conditions) subject factors. Results suggest that fear appeals significantly harm student test performance relative to efficacy appeals, even when controlling for the impact of intrinsic motivation on test anxiety. Contrary to prediction, student anxiety did not appear to explain the relationship between fear appeals and lowered test performance. The potential implications of findings are discussed, including the importance of instructional context with regards to teacher instructional practices and student success.

Identification and Validation of a Brief Test Anxiety Screening Tool

The implementation of test-based accountability policies around the world has increased the pressure placed on students to perform well on state achievement tests. Educational researchers have begun taking a closer look at the reciprocal effects of test anxiety and high-stakes testing. However, existing test anxiety assessments lack efficiency and ease of use necessary for universal screening and progress monitoring in school settings. This study investigated the psychometric properties of a brief version of the FRIEDBEN Test Anxiety Scale (B-FTAS), which has the unique strength of measuring test anxiety using a contemporary biopsychosocial model. The B-FTAS was administered to 1,463 high school students across 6 school districts 1 week before a high-stakes test. Exploratory and confirmatory factor analyses identified a 3-factor, brief, 12-item test anxiety assessment consistent with a biopsychosocial model including social, cognitive, and physiological factors. Results provide sufficient evidence for internal reliability and validity of this brief measure of test anxiety. Implications for future development and use of screening instruments in the assessment of high or debilitating anxiety are discussed.

Sensitivity to change and concurrent validity of direct behavior ratings for academic anxiety.

Multitiered frameworks of service delivery have traditionally underserved students with mental health needs. Whereas research has supported the assessment and intervention of social and academic behavior across tiers, evidence is limited with regard to mental health concerns including internalizing behaviors (e.g., anxiety and depression). In particular, there is a notable shortage of brief anxiety assessment tools to be used for progress monitoring purposes. Moreover, traditional omnibus rating scale approaches may fail to capture contextually dependent anxiety. The purpose of the present investigation is to examine the sensitivity to change and concurrent validity of Direct Behavior Ratings (DBR; Chafouleas, Riley-Tillman, & Christ, 2009; Chafouleas, Riley-Tillman, & Sugai, 2007) of anxiety and traditional rating scales in measuring academic anxiety directly before, during, and after a potentially anxiety provoking stimulus. Research was conducted with 115 undergraduate students in a Southeastern university. Results indicated significant relationships between DBRs and pre- and postmeasures of anxiety. Change metrics suggested an overall lack of correspondence between DBR and the criterion measure, with DBR scales detecting greater change both across the testing situation and participants. The use of DBR for anxiety is considered within a multitiered, problem-solving framework. Feasibility and limitations associated with implementation are discussed.

High-Stakes Accountability: Student Anxiety and Large-Scale Testing.

This study examined the relationship between student anxiety about high-stakes testing and their subsequent test performance. The FRIEDBEN Test Anxiety Scale was administered to 1,134 11th-grade students, and data were subsequently collected on their statewide assessment performance. Test anxiety was a significant predictor of test performance when controlling for other expected predictors of test performance, including school performance as measured by grade point average. The composite test anxiety score was negatively related to test performance, although one test anxiety subscale demonstrated a positive relationship with test performance. This study provides a more nuanced understanding of the relationship between test anxiety and test performance. Implications for the provision of assessment and intervention for specific aspects of test anxiety are discussed, and directions for future research on the influence of anxiety on test performance are provided.