Kenneth D. Hopkins

The Unit of Analysis: Group Means Versus Individual Observations

This paper shows that the common recommendation to use group means when there may be nonindependence among observational units is unnecessary, unduly restrictive, impoverishes the analysis, and limits the questions that can be addressed in a study. When random factors are properly identified and included in the analysis, the results (Fs and critical Fs) are identical in balanced ANOVA designs, irrespective of whether group means or individual observations are employed. The use of individual observations also allows the exploration of other interesting questions pertaining to interaction and generalizability. In addition, the pooling strategy can be considered. Thus, the question of the proper experimental unit or unit of analysis for treatment effects is answered directly, correctly, and implicitly when the proper statistical model is employed.

Tests for normality and measures of skewness and kurtosis: their place in research reporting

Even though research interest is typically greatest for questions pertaining to central tendency and, to a lesser degree, variability, knowledge about the nature of a measure or variable is impoverished when information about the shape of the frequency distribution is ignored. This paper makes the point that descriptive and inferential measures of non-normality should be a routine part of research reporting, along with graphic displays of the frequency distribution of important variables. This point is especially true for research involving measures with non-arbitrary metrics where the shape of the distribution is not affected by measurement artifacts.

Response Rates in Survey Research: A Meta-Analysis of the Effects of Monetary Gratuities.

Abstract A meta-analysis was used to compare the response rate to mailed surveys with (E) and without (C) a monetary gratuity. The average response rate increased 19% when a gratuity was enclosed. When a gratuity was promised (contingent on the return of a questionnaire), the average increase was 7%. Larger gratuities had a greater effect than lesser amounts—an enclosed

The relationship between academic achievement and metacognitive comprehension monitoring ability of spanish secondary school students

1 gratuity increased the response rate by 20%. This pattern was consistent regardless of the salience of the topic or the nature of the population (general vs. professional) surveyed. The impact of the gratuity remained substantial even when the survey design included two mailings—the use of follow-ups did not wash out the effect of the gratuity. The impact of the gratuity appeared to be attenuated by poor survey design and instrumentation, such as a cover letter that failed to present the incentive as a gratuity (rather than compensation). The findings indicate that the external validity of mail surveys can be substantially increased by ...

The Experimental Unit in Statistical Analysis

A measure of metacognitive comprehension monitoring ability (CMA) was developed to determine whether this ability was related to academic achievement, as measured by marks (GPA) in several high school courses in Spain. CMA was found to be significantly related to GPA, although the correlations were not high, and decreased with grade level. CMA was highly related to grade level; an effect size of 1.0 was observed between mean scores at Grade 10 and Grade 12. CMA appears to have no relationship with gender.

Ten-Year Stability of Verbal and Nonverbal IQ Scores

healthy skepticism toward their own work, because the validity of any statistical analysis for any experiment depends upon how nearly the data and components of the experiment approximate the conditions of the mathematical model employed. The mathematical statistician is largely concerned with the development of &dquo;theoretical&dquo; experiments; the researcher employs models developed by the theoretician to assist him in making an objective and, hopefully, correct interpretation. Some situations closely parallel the models but other situations are only rough approximations, at best. The &dquo;success&dquo; of gambling casinos in Las Vegas rests on the premise that the assumptions of their mathematical models are almost perfectly met. Conversely, the probability-calculated assuming fair dice-of 15 straight &dquo;passes&dquo; is irrelevant in a crap game with loaded dice. Perfect isomorphism between models and reality is rarely possible in the behavioral sciences, yet the accuracy of our conclusions and inferences is contingent, at least in part, upon the degree of correspondence between the model and the experimental processes and data. A researcher needs

Comments: A Schema for Proper Utilization of Multiple Comparisons in Research and a Case Study1

Intelligence tests continue to be the most widely used measures of cognitive aptitudes. Performance on such measures is usually expressed as an IQ score. Popular opinion to the contrary, relatively little is known about the long term measuring of IQ scores from group verbal and non-verbal intelligence tests, especially the latter. This study shows that, below ten years of age, stability in IQ scores from group verbal tests is considerably below that for the Stanford-Binet. Non-verbal IQ scores were found to have substantially less stability than Verbal IQs.

The Effect of Character Sex on Story Interest and Comprehension in Children

The recently published study by Rothkopf (1966) illustrates a common, albeit undesirable, phenomenon in behavioral research: The design and analysis are not adequately congruent. Rothkopfs study reflects unusual care and insight into the matter of design and internal validity (with one major deficiency; he does not indicate that the subjects were randomly assigned to treatments, although there is good reason to expect that this was the case). The authors are in complete sympathy with the position that design is of primary importance, and that statistical analysis is secondary, that statistics should be the researchers slave, not his master. Such a position, however, is not equivalent to, nor an excuse for, selecting inappropriate statistical techniques and models which lead to crude probability estimates of type-I errors, these being, of course, the basis for inference. Rothkopfs study examines the effects of experimental test-like questions (EQs) on a general achievement test (GT) for various reading selections varying the temporal position of EQs [before (B) and after (A)], with and without given answers. He also compared the effects of interspersed EQs versus EQs given in a block prior to reading. He also employed two control groups (no EQs), one of which was given no specialized instructions (C) while the other was directed to read carefully and slowly (called Direction Reference Group DRG by Rothkopf). Rothkopfs (1966) seven-level one-way ANOVA design is illustrated below, showing the combinations of the above-mentioned variables which he included. EQs No EQs (controls)

The Effects of Books in Spanish and Free Reading Time on Hispanic Students' Reading Abilities and Attitudes.

The effect of the sex of the main character on boys’ and girls’ reading interest and comprehension was studied for 540 students in grade 5. Each student was randomly assigned one of 18 representative stories from children’s periodicals written at an appropriate readability. There were three stories in each of the three types of stories (mystery, adventure, and humor); each of the nine stories appeared in two versions that were identical except for the sex of the main character. Following the reading of the story, the students completed a measure of interest and two comprehension measures (multiple-choice and cloze tests). Students were classified by sex into three levels of reading, and a balanced five-factor (reader-sex, sex-of-protagonist, ability-level, type-of-story, and story [nested within type-of-story]) ANOVA was performed for each dependent measure. A strong sex-of-reader by sex-of-main-character interaction was observed on the interest measure. Boys rated stories much less interesting when the main character was female; girls found stories with male protagonists less interesting, although their preferences were much less pronounced. The interaction was evident at each of three levels of reading ability for each story type. The interest pattern did not translate into comprehension differences. On the comprehension measures, the sex of the main character had no significant effects—girls outscored boys, but this pattern was not significantly influenced by the sex of the protagonist.

Validity and Reliability Consequences of Confidence Weighting

The Spanish and English reading abilities, reading attitudes and academic self-concepts of two comparable groups of elementary Hispanic students were investigated. The effects of providing a great variety of books in Spanish and sixty minutes a week of free reading time were studied by analyzing the results of the Tests of Reading: Inter-American Series and by having students respond anonymously to reading attitude and academic self-concept inventories. There was a trend for significantly higher Spanish reading performance in the E groups with no loss in their English reading proficiency. The reading attitudes of the E groups also improved significantly.