Roser Bono

Analyzing Small Samples of Repeated Measures Data with the Mixed-Model Adjusted F Test

This research examines the Type I error rates obtained when using the mixed model with the Kenward-Roger correction and compares them with the Between–Within and Satterthwaite approaches in split-plot designs. A simulated study was conducted to generate repeated measures data with small samples under normal distribution conditions. The data were obtained via three covariance matrices (unstructured, heterogeneous first-order auto-regressive, and random coefficients), the one with the best fit being selected according to the Akaike criterion. The results of the simulation study showed the Kenward-Roger test to be more robust, particularly when the population covariance matrices were unstructured or heterogeneous first-order auto-regressive. The main contribution of this study lies in showing that the Kenward–Roger method corrects the liberal Type I error rates obtained with the Between–Within and Satterthwaite approaches, especially with positive pairings between group sizes and covariance matrices.

Using the linear mixed model to analyze nonnormal data distributions in longitudinal designs

Using a Monte Carlo simulation and the Kenward–Roger (KR) correction for degrees of freedom, in this article we analyzed the application of the linear mixed model (LMM) to a mixed repeated measures design. The LMM was first used to select the covariance structure with three types of data distribution: normal, exponential, and log-normal. This showed that, with homogeneous between-groups covariance and when the distribution was normal, the covariance structure with the best fit was the unstructured population matrix. However, with heterogeneous between-groups covariance and when the pairing between covariance matrices and group sizes was null, the best fit was shown by the between-subjects heterogeneous unstructured population matrix, which was the case for all of the distributions analyzed. By contrast, with positive or negative pairings, the within-subjects and between-subjects heterogeneous first-order autoregressive structure produced the best fit. In the second stage of the study, the robustness of the LMM was tested. This showed that the KR method provided adequate control of Type I error rates for the time effect with normally distributed data. However, as skewness increased—as occurs, for example, in the log-normal distribution—the robustness of KR was null, especially when the assumption of sphericity was violated. As regards the influence of kurtosis, the analysis showed that the degree of robustness increased in line with the amount of kurtosis.

General Linear Mixed Model for Analysing Longitudinal Data in Developmental Research

Many areas of psychological, social, and health research are characterised by hierarchically structured data. Growth curves are usually represented by means of a two-level hierarchical structure in which observations are the first-level units nested within subjects, the second-level units. With data such as these, the best option for analysis is the general linear mixed model, which can be used even with longitudinal data series in which intervals are not constant or for which over the passage of time there is loss of data. In this paper an overview is given of the general linear mixed model approach to the analysis of longitudinal data in developmental research. The advantages of this model in comparison with the traditional approaches for analysing longitudinal data are shown, emphasising the usefulness of modelling the covariance structure properly to achieve a precise estimation of the parameters of the model.

The effect of skewness and kurtosis on the robustness of linear mixed models.

This study analyzes the robustness of the linear mixed model (LMM) with the Kenward–Roger (KR) procedure to violations of normality and sphericity when used in split-plot designs with small sample sizes. Specifically, it explores the independent effect of skewness and kurtosis on KR robustness for the values of skewness and kurtosis coefficients that are most frequently found in psychological and educational research data. To this end, a Monte Carlo simulation study was designed, considering a split-plot design with three levels of the between-subjects grouping factor and four levels of the within-subjects factor. Robustness is assessed in terms of the probability of type I error. The results showed that (1) the robustness of the KR procedure does not differ as a function of the violation or satisfaction of the sphericity assumption when small samples are used; (2) the LMM with KR can be a good option for analyzing total sample sizes of 45 or larger when their distributions are normal, slightly or moderately skewed, and with different degrees of kurtosis violation; (3) the effect of skewness on the robustness of the LMM with KR is greater than the corresponding effect of kurtosis for common values; and (4) when data are not normal and the total sample size is 30, the procedure is not robust. Alternative analyses should be performed when the total sample size is 30.

Short Time Series Analysis: C Statistic vs Edgington Model

Youngs C statistic (1941) makes it possible to compare the randomization of a set of sequentially organized data and constitutes an alternative of appropriate analysis in short time series designs. On the other hand, models based on the randomization of stimuli are also very important within the behavioral content applied. For this reason, a comparison is established between the C statistic and the Edgington model. The data analyzed in the comparative study have been obtained from graphs in studies published in behavioral journals. According to the results obtained, it is concluded that the Edgington model in experimental designs AB involves many measurements while the C statistic requires fewer observations to reach the conventional significance level.

Should we rely on the Kenward–Roger approximation when using linear mixed models if the groups have different distributions?

The study explores the robustness to violations of normality and sphericity of linear mixed models when they are used with the Kenward-Roger procedure (KR) in split-plot designs in which the groups have different distributions and sample sizes are small. The focus is on examining the effect of skewness and kurtosis. To this end, a Monte Carlo simulation study was carried out, involving a split-plot design with three levels of the between-subjects grouping factor and four levels of the within-subjects factor. The results show that: (1) the violation of the sphericity assumption did not affect KR robustness when the assumption of normality was not fulfilled; (2) the robustness of the KR procedure decreased as skewness in the distributions increased, there being no strong effect of kurtosis; and (3) the type of pairing between kurtosis and group size was shown to be a relevant variable to consider when using this procedure, especially when pairing is positive (i.e., when the largest group is associated with the largest value of the kurtosis coefficient and the smallest group with its smallest value). The KR procedure can be a good option for analysing repeated-measures data when the groups have different distributions, provided the total sample sizes are 45 or larger and the data are not highly or extremely skewed.

The effect of skewness and kurtosis on the Kenward-Roger approximation when group distributions differ

BACKGROUND This study examined the independent effect of skewness and kurtosis on the robustness of the linear mixed model (LMM), with the Kenward-Roger (KR) procedure, when group distributions are different, sample sizes are small, and sphericity cannot be assumed. METHODS A Monte Carlo simulation study considering a split-plot design involving three groups and four repeated measures was performed. RESULTS The results showed that when group distributions are different, the effect of skewness on KR robustness is greater than that of kurtosis for the corresponding values. Furthermore, the pairings of skewness and kurtosis with group size were found to be relevant variables when applying this procedure. CONCLUSIONS With sample sizes of 45 and 60, KR is a suitable option for analyzing data when the distributions are: (a) mesokurtic and not highly or extremely skewed, and (b) symmetric with different degrees of kurtosis. With total sample sizes of 30, it is adequate when group sizes are equal and the distributions are: (a) mesokurtic and slightly or moderately skewed, and sphericity is assumed; and (b) symmetric with a moderate or high/extreme violation of kurtosis. Alternative analyses should be considered when the distributions are highly or extremely skewed and samples sizes are small.

Evaluating Effects in Short Time Series: Alternative Models of Analysis

In the applied context, short time-series designs are suitable to evaluate a treatment effect. These designs present serious problems given autocorrelation among data and the small number of observations involved. This paper describes analytic procedures that have been applied to data from short time series, and an alternative which is a new version of the generalized least squares method to simplify estimation of the error covariance matrix. Using the results of a simulation study and assuming a stationary first-order autoregressive model, it is proposed that the original observations and the design matrix be transformed by means of the square root or Cholesky factor of the inverse of the covariance matrix. This provides a solution to the problem of estimating the parameters of the error covariance matrix. Finally, the results of the simulation study obtained using the proposed generalized least squares method are compared with those obtained by the ordinary least squares approach. The probability of Type I error associated with the proposed method is close to the nominal value for all values of ρ1 and n investigated, especially for positive values of ρ1. The proposed generalized least squares method corrects the effect of autocorrelation on the tests power.

AUTOCORRELATION PROBLEMS IN SHORT TIME SERIES

The work of Huitema (1985) on autocorrelation in behavioral data suggests that the use of conventional statistical methods is justified. The present study restates the problem of autocorrelation by analyzing 100 baselines of small samples designs published in the Journal of Applied Behavior Analysis during 1992. The results show a negative bias in the autocorrelations, especially with very small samples. The autocorrelation values are normally distributed, and the method of Davies, Trigg, and Newbold (1977) is the most accurate in calculating the standard deviation.

Manipulating the alpha level cannot cure significance testing

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.