David Giofrè

High Impact = High Statistical Standards? Not Necessarily So

What are the statistical practices of articles published in journals with a high impact factor? Are there differences compared with articles published in journals with a somewhat lower impact factor that have adopted editorial policies to reduce the impact of limitations of Null Hypothesis Significance Testing? To investigate these questions, the current study analyzed all articles related to psychological, neuropsychological and medical issues, published in 2011 in four journals with high impact factors: Science, Nature, The New England Journal of Medicine and The Lancet, and three journals with relatively lower impact factors: Neuropsychology, Journal of Experimental Psychology-Applied and the American Journal of Public Health. Results show that Null Hypothesis Significance Testing without any use of confidence intervals, effect size, prospective power and model estimation, is the prevalent statistical practice used in articles published in Nature, 89%, followed by articles published in Science, 42%. By contrast, in all other journals, both with high and lower impact factors, most articles report confidence intervals and/or effect size measures. We interpreted these differences as consequences of the editorial policies adopted by the journal editors, which are probably the most effective means to improve the statistical practices in journals with high or low impact factors.

Differences in the intellectual profile of children with intellectual vs. learning disability.

The WISC-IV was used to compare the intellectual profile of two groups of children, one with specific learning disorders (SLDs), the other with intellectual disabilities (ID), with a view to identifying which of the four main factor indexes and two additional indexes can distinguish between the groups. We collected information on WISC-IV scores for 267 children (Mage=10.61 [SD=2.51], range 6-16 years, females=99) with a diagnosis of either SLD or ID. Children with SLD performed better than those with ID in all measures. Only the SLD children, not the ID children, revealed significant differences in the four main factor indexes, and their scores for the additional General Ability Index (GAI) were higher than for the Cognitive Proficiency Index (CPI). Children with a diagnosis of SLD whose Full-Scale Intelligence Quotient (FSIQ) was <85 showed a similar pattern. Our findings confirm the hypothesis that children with SLD generally obtain high GAI scores, but have specific deficiencies relating to working memory and processing speed, whereas children with ID have a general intellectual impairment. These findings have important diagnostic and clinical implications and should be considered when making diagnostic decisions in borderline cognitive cases.

Intuitive geometry and visuospatial working memory in children showing symptoms of nonverbal learning disabilities.

Visuospatial working memory (VSWM) and intuitive geometry were examined in two groups aged 11–13, one with children displaying symptoms of nonverbal learning disability (NLD; n = 16), and the other, a control group without learning disabilities (n = 16). The two groups were matched for general verbal abilities, age, gender, and socioeconomic level. The children were presented with simple storage and complex-span tasks involving VSWM and with the intuitive geometry task devised by Dehaene, Izard, Pica, and Spelke (2006). Results revealed that the two groups differed in the intuitive geometry task. Differences were particularly evident in Euclidean geometry and in geometrical transformations. Moreover, the performance of NLD children was worse than controls to a larger extent in complex-span than in simple storage tasks, and VSWM differences were able to account for group differences in geometry. Finally, a discriminant function analysis confirmed the crucial role of complex-span tasks involving VSWM in distinguishing between the two groups. Results are discussed with reference to the relationship between VSWM and mathematics difficulties in nonverbal learning disabilities.

The relationship among geometry, working memory, and intelligence in children.

Although geometry is one of the main areas of mathematical learning, the cognitive processes underlying geometry-related academic achievement have not been studied in detail. This study explored the relationship among working memory (WM), intelligence (g factor), and geometry in 176 typically developing children attending school in their fourth and fifth grades. Structural equation modeling showed that approximately 40% of the variance in academic achievement and in intuitive geometry (which is assumed to be independent of a persons cultural background) was explained by WM and the g factor. After taking intelligence and WM into account, intuitive geometry was no longer significantly related to academic achievement in geometry. We also found intuitive geometry to be closely related to fluid intelligence (as measured by Ravens colored progressive matrices) and reasoning ability, whereas academic achievement in geometry depended largely on WM. These results were confirmed by a series of regressions in which we estimated the contributions of WM, intelligence, and intuitive geometry to the unique and shared variance explaining academic achievement in geometry. Theoretical and educational implications of the relationship among WM, intelligence, and academic achievement in geometry are discussed.

Strengths and Weaknesses in the Intellectual Profile of Different Subtypes of Specific Learning Disorder: A Study on 1,049 Diagnosed Children

The present study analyzes whether and how the most common diagnoses within the specific learning disorder (SLD) category are characterized by different intellectual profiles. The issue is relevant to the current debate on the unitary versus decomposable nature of the SLD category and may help define specific interventions. Intellectual profiles were obtained using the Wechsler Intelligence Scale for Children–IV (WISC-IV) on 1,049 children diagnosed with SLD using the ICD-10 codes. Four major subsamples were compared: reading disorder, spelling disorder, disorder of arithmetical skills, and mixed disorder of scholastic skills. The four main WISC-IV indexes (verbal comprehension, perceptual reasoning, working memory, and processing speed) were considered. Results showed that all SLD subgroups share similar weaknesses in working memory and processing speed, but they also showed that the subgroups are characterized by partly different intellectual profiles. These specificities should be considered in the definition of SLD.

Problem Solving and Working Memory Updating Difficulties in a Group of Poor Comprehenders

Problem-solving abilities, text comprehension, and working memory updating were investigated in 25 8–year-old children with a specific difficulty at comprehending written texts (i.e., poor comprehenders) and 25 control children matched for gender, age, schooling, and reading decoding ability. Poor comprehenders revealed to be less competent not only in the tasks that required a good text comprehension but also in problem-solving and updating tasks. Results are discussed in terms of their implications for educational settings.

The Influence of Journal Submission Guidelines on Authors' Reporting of Statistics and Use of Open Research Practices

From January 2014, Psychological Science introduced new submission guidelines that encouraged the use of effect sizes, estimation, and meta-analysis (the “new statistics”), required extra detail of methods, and offered badges for use of open science practices. We investigated the use of these practices in empirical articles published by Psychological Science and, for comparison, by the Journal of Experimental Psychology: General, during the period of January 2013 to December 2015. The use of null hypothesis significance testing (NHST) was extremely high at all times and in both journals. In Psychological Science, the use of confidence intervals increased markedly overall, from 28% of articles in 2013 to 70% in 2015, as did the availability of open data (3 to 39%) and open materials (7 to 31%). The other journal showed smaller or much smaller changes. Our findings suggest that journal-specific submission guidelines may encourage desirable changes in authors’ practices.

Forward and backward digit span difficulties in children with specific learning disorder

ABSTRACT This study examined performance in the forward and backward digit span task of the Wechsler Intelligence Scale for Children–Fourth Edition (WISC–IV) in a large group of children with specific learning disorder (SLD) as compared with a group of typically developing children matched for age and sex. Our results further support the hypothesis that the intellectual difficulties of children with SLD involve working memory in the forward digit span task to a greater extent than in the backward digit span task. The correlation of the two spans with a General Ability Index (GAI) was similar in SLD, and smaller in magnitude than in typically developing children. Despite a GAI within normal range, children with SLD had difficulty with both digit span tasks, but more so for forward span. This pattern was similar for different SLD profiles with clinical diagnoses of dyslexia and mixed disorder, but the impairments were more severe in the latter. Age differences were also investigated, demonstrating larger span impairment in older children with SLD than in younger.

Visuospatial working memory in children with autism: The effect of a semantic global organization

It has been reported that individuals with Autism Spectrum Disorders (ASD) perceive visual scenes as a sparse set of details rather than as a congruent and meaningful unit, failing in the extraction of the global configuration of the scene. In the present study, children with ASD were compared with typically developing (TD) children, in a visuospatial working memory task, the Visual Patterns Test (VPT). The VPT array was manipulated to vary the semantic affordance of the pattern, high semantic (global) vs. low semantic; temporal parameters were also manipulated within the change detection protocol. Overall, there was no main effect associated with Group, however there was a significant effect associated with Semantics, which was further qualified by an interaction between the Group and Semantic factors; there was only a significant effect of semantics in the TD group. The findings are discussed in light of the weak central coherence theory where the ASD group are unable to make use of long term memory semantics in order to construct global representations of the array.

The pervasive avoidance of prospective statistical power: major consequences and practical solutions.

The estimation of the prospective statistical power (PSP) is mandatory when using a classical Neyman-Pearson statistical method that together with the one by Fisher, represents one of the pillar of the so-called frequentist statistical approach. Unfortunately in the course of the history of their application, these two methods have been mixed, adopting the Fisher approach for hypotheses or model comparisons and their differences ignored in the classical Null Hypothesis Significant Testing (NHST) approach, which at present, represents the most used statistical approach in many research fields, from psychology to medicine, from neuroscience to ecology.The uncritical application of the NHST statistical approach in ignoring its assumptions, strengths and weakness, has been considered one if not the principal cause of the “crisis of confidence” in scientific evidence.In this paper we offer practical solutions on how to overcome this situation.