Gerald S. Hanna

Effects of Total and Partial Feedback in Multiple-Choice Testing Upon Learning

AbstractPosttest performance resulting from total feedback, partial feedback, and no feedback in a multiple-choice test were compared. About 1,400 fifth and sixth graders first took a completion-format pretest which was used to match triads. Next, they all took a multiple-choice test, one-third under each of three experimental feedback conditions-total immediate feedback, partial immediate feedback, and no feedback. Finally, all examinees took a completion-format posttest. As predicted, those taking the multiple-choice test without feedback had significantly lower posttest performance than either feedback group. An hypothesized aptitude-treatment interaction was found between achievement level and feedback treatment. An unexpected interaction between sex and feedback also emerged.

Objective Analysis of Achievement-Aptitude Discrepancies in LD Classification

A critical review of the literature dealing with quantification of aptitude-achievement differences revealed that methods developed to date are fraught with numerous and grave inadequacies. Based upon this evaluation of past methods, this article outlines and discusses criteria of new and more adequate procedures. Finally, step-by-step procedures that can be routinely followed are presented.

Toward a Unified Theory of Context Dependence.

A CRITICAL REVIEW of the literature on passage dependence concluded that contributions by both Tuinman and Pyrczak provided useful but confusing measures and concepts related to passage dependence. The major source of this confusion was traced, brought into focus, and resolved. After a vital additional element, item difficulty, was identified, the relevant concepts were integrated into a general theory of context dependence. Finally, sample items and data were discussed to illustrate the varied practical applications of the theory.

Estimating major sources of measurement error in individual intelligences scales: Taking our heads out of the sand

Abstract Four ubiquitous major sources of measurement error for individual intelligence scales were identified and discussed. It was argued that where these sources cannot be directly investigated, they should be estimated rather than ignored. The typical magnitude of error arising from each of content sampling, time sampling, scoring, and administration was estimated; their composite was found to produce a far larger standard error of measurement than manuals of individual intelligence tests typically report. Implications for practitioners are discussed.

Identifying and Adjusting for Biases in Student Evaluations of Instruction: Implications for Validity.

Student evaluations of teaching effectiveness are widely used for improvement of instruction and for administrative decision-making. Several kinds of contaminating variables may influence these ratings. Criteria for identifying such contaminants are set forth. Results of a search for extraneous, biasing variables at the high school level are reported. Of the several potential extraneous influences studied, student motivation/preference and class size meet the criteria. It is concluded that student rating forms should make provisions for adjusting for these and similar unfair influences.

A Comparison of High School Student Ratings of Teaching Effectiveness with Teacher Self-Ratings: Factor Analytic and Multitrait-Multimethod Analyses

Two classes of each of four hundred high school teachers were rated for teaching effectiveness by the teachers and by their students using the same instrument. Separate factor analyses yielded highly similar underlying dimensions. Convergent validity coefficients were significant for all factors. These findings provide support for the validity of high school student ratings of instruction.

Studies of the Seriousness of Three Threats to Passage Dependence.

The three studies presented are concerned with three kinds of multiple-choice reading comprehension questions that may be answered correctly at a higher-than-chance level when they are administered without the accompanying passage. An attempt was made to demonstrate that each of these kinds of high risk item does not necessarily lead to passage dependence invalidity. One study succeeded whereas two failed. The net results of these failures to show that the threats to validity are not necessarily damning was interpreted as evidence of the seriousness of the threats.

Instructional Sensitivity Expanded.

The presently viewed role and utility of measures of instructional sensitivity were summarized. A case was made that the rationale for the assessment of instructional sensitivity can be applied to all achievement tests and should not continue to be restricted to criterion-referenced mastery tests. Next, a line of research and theory concerning context dependence in tests containing interpretive exercises was outlined. The logic of this theory was integrated with the rationale of instructional sensitivity. Finally, practical interrelationships among content validity, reliability, instructional sensitivity, and related measures were discussed and illustrated by use of a varied sample of test items and data.

The Impact of Classroom Testing Frequency on High School Students' Achievement.

Abstract Research on the impact of classroom testing frequency was reviewed, and common research design flaws were discussed. The present study was designed to circumvent these limitations and defects. Nearly 2000 first-semester 10th-grade male biology students in Saudi Arabia were randomly assigned to be quizzed at one of two frequencies—monthly or semimonthly. The group quizzed twice a month outperformed the less frequently quizzed group by .3 standard deviations on both an end-of-semester examination and a delayed recall test administered 3 months later. Discussion focused on generalizability of past and present research to American high schools, the cost effectiveness of implementation, and desirable directions for future research.

Discriminant and Convergent Validity of High School Student Ratings of Instruction

Two studies addressed the construct validity of the IDEA Form H Survey with which high school students rate instruction. Both studies provided indirect evidence of the validity of this instrument as a measure of student progress on goals selected by the teacher of each class.