Brian F. French

Confirmatory Factor Analytic Procedures for the Determination of Measurement Invariance.

Confirmatory factor analytic (CFA) procedures can be used to provide evidence of measurement invariance. However, empirical evaluation has not focused on the accuracy of common CFA steps used to detect a lack of invariance across groups. This investigation examined procedures for detection of test structure differences across groups under several conditions through simulation. Specifically, sample size, number of factors, number of indicators per factor, and the distribution of the observed variables were manipulated, and 3 criteria for assessing measurement invariance were evaluated. Power and Type I error were examined to evaluate the accuracy of detecting a lack of invariance. Results suggest that the chi-square difference test adequately controls the Type I error rate in nearly all conditions, and provides relatively high power when used with maximum likelihood (ML) estimation and normally distributed observed variables. In addition, the power of the test to detect group differences for dichotomous observed variables with robust weighted least squares estimation was generally very low.

Effects of Monetary Incentives on Engagement in the PACE Parenting Program

This study evaluated parental engagement in an 8-week parenting program offered through daycare centers that were randomly assigned to a monetary incentive or nonincentive condition. Of an initial sample of 1,050 parents who rated their intent to enroll in the program, 610 went on to enroll—319 in the incentive and 291 in the nonincentive condition. Results showed that intent to enroll predicted enrollment irrespective of condition. Further, parents did not enroll in greater numbers, attend more sessions, or participate more actively in the incentive condition. Incentives encouraged some parents, often younger and socioeconomically disadvantaged, to enroll but had no effect on their attendance. Of importance, these results could not be accounted for by between-condition differences in child and family or in daycare characteristics.

Lexical effects on spoken-word recognition in children with normal hearing.

Objectives: This study is the first in a series designed to develop and norm new theoretically motivated sentence tests for children. The purpose was to examine the independent contributions of word frequency (i.e., how often words occur in language) and lexical density (the number of similar sounding words or “neighbors” to a target word) to the perception of key words in the new sentence set. Design: Twenty-four children with normal hearing aged 5 to 12 yrs served as participants; they were divided into four equal age-matched groups. The stimuli consisted of 100 semantically neutral sentences that were 5 to 7 words in length. Each sentence contained 3 key words that were controlled for word frequency and lexical density. Words with few neighbors come from sparse neighborhoods, whereas words with many neighbors come from dense neighborhoods. The key words within a sentence belonged to one of the four lexical categories: (1) high-frequency sparse, (2) low-frequency dense, (3) high-frequency dense, and (4) low-frequency sparse. Participants were administered the sentence list and the 300 key words in isolation at 65 dB SPL. Each participant group was tested in spectrally matched noise at one of the four signal-to-noise ratios (SNRs −2, 0, 2, and 4 dB). The percent of words correctly identified was calculated as a function of SNR, key word context (sentences vs. words), and key word lexical category. Results: SNR had a significant effect on the recognition of key words in sentences and in isolation; performance improved at higher SNRs. There were significant main effects of word frequency and lexical density as well as a significant interaction between the two lexical factors. In isolation, high-frequency words were recognized more accurately than low-frequency words. In both word and sentence contexts, sparse words yielded greater accuracy than dense words, irrespective of word frequency. There was a modest but significant negative correlation between lexical density and the recognition of words in isolation and in sentences. Conclusions: Word frequency and lexical density seem to influence word recognition independently in children with normal hearing. This is similar to earlier results in adults with normal hearing. In addition, there seems to be an interaction between the two factors, with lexical density being more heavily weighted than word frequency. These results give us further insight into the way children organize and access words from long-term lexical memory in a relational way. Our results showed that lexical effects were most evident at poorer SNRs. This may have important implications for assessing spoken-word recognition performance in children with sensory aids because they typically receive a degraded auditory signal.

Differential Effects of Divergent Thinking, Domain Knowledge, and Interest on Creative Performance in Art and Math

The purpose of this study was to investigate the main effects of divergent thinking, domain knowledge, and two types of interest (i.e., individual and situational interest) on creative performance in art and math, as well as moderating and mediating effects of the two types of interest. A series of hierarchical multiple regression analyses were conducted on data collected from 221 Korean 8th graders. Both divergent thinking and domain knowledge contributed to creative performance in art and math. However, the relative importance of these two factors was different in the two domains. In art, divergent thinking explained more of the variance in creative performance than domain knowledge did; in math, domain knowledge explained more of the variance than divergent thinking. Individual interest had no statistically significant main effect either in art or math. Situational interest had a statistically significant main effect on creative performance in math, but not in art. None of the hypothesized moderating and mediating effects of the two types of interest was statistically significant. The theoretical implications of the study are discussed, especially with respect to linking the relative importance of the variables in this study to the different domain structures of art and math.

Estimation of MIMIC Model Parameters with Multilevel Data.

The purpose of this simulation study was to assess the performance of latent variable models that take into account the complex sampling mechanism that often underlies data used in educational, psychological, and other social science research. Analyses were conducted using the multiple indicator multiple cause (MIMIC) model, which is a flexible and effective tool for relating observed and latent variables. The data were simulated in a hierarchical framework (e.g., individuals nested in schools) so that a multilevel modeling approach would be appropriate. Analyses were conducted accounting for and not accounting for the nested data to determine the impact of ignoring such multilevel data structures in full structural equation models. Results highlight the differences in modeling results when the analytic strategy is congruent with the data structure and what occurs when this congruency is absent. Type I error rates and power for the standard and multilevel methods were similar for within-cluster variables and for the multilevel model with between-cluster variables. However, Type I error rates were inflated for the standard approach when modeling between-cluster variables.

Extensions of Mantel–Haenszel for Multilevel DIF Detection

Multilevel data structures are ubiquitous in the assessment of differential item functioning (DIF), particularly in large-scale testing programs. There are a handful of DIF procures for researchers to select from that appropriately account for multilevel data structures. However, little, if any, work has been completed to extend a popular DIF method to this case. Thus, the primary goal of this study was to introduce and investigate the effectiveness of several new options for DIF assessment in the presence of multilevel data with the Mantel–Haenszel (MH) procedure, a popular, flexible, and effective tool for DIF detection. The performance of these new methods was compared with the standard MH technique through a simulation study, where data were simulated in a multilevel framework, corresponding to examinees nested in schools, for example. The standard MH test for DIF detection was employed, along with several multilevel extensions of MH. Results demonstrated that these multilevel tests proved to be preferable to standard MH in a wide variety of cases where multilevel data were present, particularly when the intraclass correlation was relatively large. Implications of this study for practice and future research are discussed.

Multilevel Latent Class Analysis: Parametric and Nonparametric Models

Latent class analysis is an analytic technique often used in educational and psychological research to identify meaningful groups of individuals within a larger heterogeneous population based on a set of variables. This technique is flexible, encompassing not only a static set of variables but also longitudinal data in the form of growth mixture modeling, as well as the application to complex multilevel sampling designs. The goal of this study was to investigate—through a Monte Carlo simulation study—the performance of several methods for parameterizing multilevel latent class analysis. Of particular interest was the comparison of several such models to adequately fit Level 1 (individual) data, given a correct specification of the number of latent classes at both levels (Level 1 and Level 2). Results include the parameter estimation accuracy as well as the quality of classification at Level 1.

Elementary Teachers' Knowledge and Self-Efficacy for Measurement Concepts

Educational standardized testing impacts millions of children and educational professionals each year. In the current accountability climate, an effective educational system depends on professionals who are literate in assessment and can take the appropriate actions in response to test results. Measurement researchers should begin to focus more attention on how teachers use assessment results, what skills teachers possess, and what teachers believe they can do in working with test results. This study examined elementary teacher knowledge and self-efficacy in measurement concepts through a random sample of teachers in the state of Washington. Teachers had greater success with skills related to basic measurement concepts compared to using test scores for informed decisions. No relationship was found between years of teaching and measurement knowledge or self-efficacy. However, teachers showing interest in resources for communicating test results to parents had lower self-efficacy compared to teachers not interested in resources.

Model Misspecification and Invariance Testing Using Confirmatory Factor Analytic Procedures.

Confirmatory factor analytic procedures are routinely implemented to provide evidence of measurement invariance. Current lines of research focus on the accuracy of common analytic steps used in confirmatory factor analysis for invariance testing. However, the few studies that have examined this procedure have done so with perfectly or near perfectly fitting models. In the present study, the authors examined procedures for detecting simulated test structure differences across groups under model misspecification conditions. In particular, they manipulated sample size, number of factors, number of indicators per factor, percentage of a lack of invariance, and model misspecification. Model misspecification was introduced at the factor loading level. They evaluated three criteria for detection of invariance, including the chi-square difference test, the difference in comparative fit index values, and the combination of the two. Results indicate that misspecification was associated with elevated Type I error rates in measurement invariance testing.

Validity Evidence for the State Mindfulness Scale for Physical Activity

ABSTRACT Being attentive to and aware of one’s experiences in the present moment with qualities of acceptance and openness reflects the state of mindfulness. Positive associations exist between state mindfulness and state autonomous motivation for everyday activities. Though this suggests that state mindfulness links with adaptive motivational experiences, no suitable measure of state mindfulness exists that would facilitate the examination of these relationships in a physical activity context. Thus, we revised the State Mindfulness Scale (Tanay & Bernstein, 2013) and provided score validity evidence for the measure in a physical activity context. A bi-factor model reflecting mindfulness of the mind and body as specific factors and a general mindfulness factor was supported. Validity evidence, such as positive relationships with intrinsic motivation, and a negative relationship with body surveillance support score use. The revised scale can facilitate investigations of the role of mindfulness in physical activity settings.