Francis L. Huang

Alternatives to Multilevel Modeling for the Analysis of Clustered Data

Multilevel modeling has grown in use over the years as a way to deal with the nonindependent nature of observations found in clustered data. However, other alternatives to multilevel modeling are available that can account for observations nested within clusters, including the use of Taylor series linearization for variance estimation, the design effect adjusted standard errors approach, and fixed effects modeling. Using 1,000 replications of 12 conditions with varied Level 1 and Level 2 sample sizes, the author compared parameter estimates, standard errors, and statistical significance using various alternative procedures. Results indicate that several acceptable procedures can be used in lieu of or together with multilevel modeling, depending on the type of research question asked and the number of clusters under investigation. Guidelines for applied researchers are discussed.

Effects of validity screening items on adolescent survey data.

In 2 studies, we examined the use of validity screening items in adolescent survey data. In each study, adolescent respondents were asked whether they were telling the truth and paying attention in answering survey questions. In Study 1 (N = 7,801), the prevalence rates of student risk behaviors were significantly lower after inappropriate (invalid) responders were screened out of the sample. In addition, confirmatory and multigroup factor analyses demonstrated significant differences between the factor structures of school climate scales with valid versus invalid responders. In Study 2, student perceptions of school climate were correlated with teacher perceptions in 291 schools. A bootstrap resampling procedure compared the correlations obtained with valid versus invalid responding students in each school and found that valid responders had more positive views of school conditions and produced higher correlations with teacher perceptions. These findings support the value of validity screening items in improving the quality of adolescent survey data.

Multilevel Multi-Informant Structure of the Authoritative School Climate Survey.

The Authoritative School Climate Survey was designed to provide schools with a brief assessment of 2 key characteristics of school climate--disciplinary structure and student support--that are hypothesized to influence 2 important school climate outcomes--student engagement and prevalence of teasing and bullying in school. The factor structure of these 4 constructs was examined with exploratory and confirmatory factor analyses in a statewide sample of 39,364 students (Grades 7 and 8) attending 423 schools. Notably, the analyses used a multilevel structural approach to model the nesting of students in schools for purposes of evaluating factor structure, demonstrating convergent and concurrent validity and gauging the structural invariance of concurrent validity coefficients across gender. These findings provide schools with a core group of school climate measures guided by authoritative discipline theory.

Pick Your Poisson: A Tutorial on Analyzing Counts of Student Victimization Data

School violence research is often concerned with infrequently occurring events such as counts of the number of bullying incidents or fights a student may experience. Analyzing count data using ordinary least squares regression may produce improbable predicted values, and as a result of regression assumption violations, result in higher Type I errors. Count data are optimally analyzed using Poisson-based regression techniques such as Poisson or negative binomial regression. We apply these techniques to an example study of bullying in a statewide sample of 290 high schools and explain how Poisson-based analyses, although less familiar to many researchers, can produce findings that are more accurate and reliable, and are easier to interpret in real-world contexts.

The Association of Kindergarten Entry Age with Early Literacy Outcomes

ABSTRACT The authors investigated whether age at kindergarten entry was associated with early literacy achievement gaps and if these gaps persisted over time. Using the kindergarten age eligibility cutoff date, they created 2 groups of students who represented the oldest and youngest children in a cohort of students in high-poverty, low-performing schools. The authors followed 405 students from the beginning of kindergarten until the end of Grade 2. Results indicated that the youngest students scored lower than their oldest peers at the beginning of kindergarten on various early literacy measures. The early-age achievement gap, however, narrowed over time but did not close completely by the end of Grade 2. Implications for parents and educators are discussed.

Multilevel Factor Structure and Concurrent Validity of the Teacher Version of the Authoritative School Climate Survey.

BACKGROUND School climate is well recognized as an important influence on student behavior and adjustment to school, but there is a need for theory-guided measures that make use of teacher perspectives. Authoritative school climate theory hypothesizes that a positive school climate is characterized by high levels of disciplinary structure and student support. METHODS A teacher version of the Authoritative School Climate Survey (ASCS) was administered to a statewide sample of 9099 7th- and 8th-grade teachers from 366 schools. The study used exploratory and multilevel confirmatory factor analyses (MCFA) that accounted for the nested data structure and allowed for the modeling of the factor structures at 2 levels. RESULTS Multilevel confirmatory factor analyses conducted on both an exploratory (N = 4422) and a confirmatory sample (N = 4677) showed good support for the factor structures investigated. Factor correlations at 2 levels indicated that schools with greater levels of disciplinary structure and student support had higher student engagement, less teasing and bullying, and lower student aggression toward teachers. CONCLUSIONS The teacher version of the ASCS can be used to assess 2 key domains of school climate and associated measures of student engagement and aggression toward peers and teachers.

Racial/Ethnic Differences in Perceptions of School Climate and Its Association with Student Engagement and Peer Aggression

Research indicates that a positive school climate is associated with higher levels of student engagement and lower rates of peer aggression. However, less attention has been given to whether such findings are consistent across racial/ethnic groups. The current study examined whether Black, Hispanic, and White high school students differed in their perceptions of school climate, student engagement, and peer aggression as measured by the Authoritative School Climate survey. In addition, the study tested whether the associations between school climate and both student engagement and peer aggression varied as a function of racial/ethnic group. The sample consisted of 48,027 students in grades 9–12 (51.4 % female; 17.9 % Black, 10.5 % Hispanic, 56.7 % White, and 14.9 % other) attending 323 high schools. Regression models that contrasted racial/ethnic groups controlled for the nesting of students within schools and used student covariates of parent education, student gender, and percentage of schoolmates sharing the same race/ethnicity, as well as school covariates of school size and school percentage of students eligible for free- or reduced-price meals. Perceptions of school climate differed between Black and White groups, but not between Hispanic and White groups. However, race/ethnicity did not moderate the associations between school climate and either engagement or peer aggression. Although correlational and cross-sectional in nature, these results are consistent with the conclusion that a positive school climate holds similar benefits of promoting student engagement and reducing victimization experiences across Black, Hispanic, and White groups.

A Latent Variable Investigation of the Phonological Awareness Literacy Screening-Kindergarten Assessment: Construct Identification and Multigroup Comparisons between Spanish-Speaking English-Language Learners (ELLs) and Non-ELL Students.

Psychometric properties of the Phonological Awareness Literacy Screening for Kindergarten (PALS-K) instrument were investigated in a sample of 2844 first-time public school kindergarteners. PALS-K is a widely used English literacy screening assessment. Exploratory factor analysis revealed a theoretically defensible measurement structure that was found to replicate in a randomly selected hold-out sample when examined through the lens of confirmatory factor analytic methods. Multigroup latent variable comparisons between Spanish-speaking English-language learners (ELLs) and non-ELL students largely demonstrated the PALS-K to yield configural and metric invariance with respect to associations between subtests and latent dimensions. In combination, these results support the educational utility of the PALS-K as a tool for assessing important reading constructs and informing early interventions across groups of Spanish-speaking ELL and non-ELL students.

Using Multilevel Factor Analysis With Clustered Data: Investigating the Factor Structure of the Positive Values Scale

Advances in multilevel modeling techniques now make it possible to investigate the psychometric properties of instruments using clustered data. Factor models that overlook the clustering effect can lead to underestimated standard errors, incorrect parameter estimates, and model fit indices. In addition, factor structures may differ depending on the level of analysis. The current study illustrates the application of multilevel factor analytic techniques using a large statewide sample of middle school students (n = 39,364) from 423 schools. Both multilevel exploratory and confirmatory factor analyses were used to investigate the factor structure of the Positive Values Scale (PVS) as part of a school climate survey. Results showed that for the PVS, a two-correlated factor model at Level 1 and a one-factor model at Level 2 best fit the data. Implications and guidance for applied researchers are discussed.

Factors associated with lowercase alphabet naming in kindergarteners

We investigated five hypotheses related to the successful naming of lowercase letters. Participants included 5,020 first-time kindergarteners from economically disadvantaged homes who previously attended publicly funded preschools. Results analyzed using three-level logistic regression showed that children have a higher probability of correctly identifying letters that appear in their own name and letters that appear frequently in print. In addition, lowercase letter shape similarity to its uppercase counterpart and letter order were also associated with the correct identification of the letter. Finally, if a lowercase letter had a visual and phonological resemblance to other letters, students had a lower probability of identifying the letter correctly. Implications for instruction are discussed.