Jason Millman

An Analysis of Test-Wiseness:

error. Test-wiseness was one suggested source. General Definition &dquo;Test-wiseness&dquo; is defined as a subject’s capacity to utilize the characteristics and formats of the test and/or the test taking situation to receive a high score. Test-wiseness is logically independent of the examinee’s knowledge of the subject matter for which the items are supposedly measures. A somewhat limited concept of test-wiseness will be analyzed in this article. The analysis will exclude factors concerned with the general mental attitude (such as anxiety and confidence) and motivational state of the examinee, and it will be restricted to the actual taking of (not preparing for) objective achievement and aptitude tests. Empirical Studies of Test-Wiseness There appears to be no systematic study of either the importance of test-wiseness or the degree to which it can be taught or measured. This is true even though both professional writers (e.g. Vernon, 1962; Anastasi, 1961; Pauck, 1950) and popular writers (e.g. Whyte, 1956; Huff, 1961; Hoff-

If at First You Don't Succeed Setting Passing Scores When More Than One Attempt Is Permitted

For licensing and certification examinations designed to protect the public, passing incompetent candidates (false positives) is a more serious error than failing competent ones. Yet to avoid classification mistakes against individuals and subsequent litigation, current practices for these tests increase the relative risk of false positives. These practices are identified, and one that allows positive errors of measurement arising from multiple opportunities to pass the test is discussed in detail. Unsatisfactory ways to mitigate this problem are considered. Increasing the amount of testing or, if that is not feasible, raising the required passing score for repeaters is recommended.

Does Grade Inflation Affect the Reliability of Grades

Two studies were conducted to examine the effect of grade inflation on the piling up of grades in fewer grade categories and on the reliability of grade point averages (GPAs). In all comparisons, grades were more bunched after grade inflation, which in turn, was associated with only slight, nonsignificant decreases in GPA reliability. As expected, grades were more bunched when the traditional 5-point letter scale was used than when plus and minus grades were also allowed. In the latter case as well, grade inflation seemed to have had very little effect on the reliability of GPAs. GPA reliability began to suffer, however, for graduate programs in which almost all grades were placed into just two categories, A and B.

Relation Between Perseverance and Rate of Learning: A Test of Carroll’s Model of School Learning

Four experiments were conducted to test a deduction from John B. Carroll’s Model of School Learning. According to the model, if a learner is already willing to persevere to the extent needed for learning, increasing the learner’s perseverance will not alter degree of learning or learning rate. In all experiments, the time needed to learn a paired-associate task under an encouragement condition designed to increase perseverance was compared to the learning time under a discouragement or control condition. As predicted by the model, none of the differences was statistically significant. The findings support the claim that rewards may increase the time learners will spend on a task but not their degree or rate of learning.

1: Research Methodology—A Point of View

In the practical sense of doing, research is a matter of techniques. An event occurs or is made to happen by the researcher and some record of the event occurs or is made. Careful study of the record generates factual statements serviceable as evidence for inferences leading to generalizations, explanations, interpretations, predictions and decisions. Research technique is the process of converting events into records and records into factual statements (including tables, charts and other ways of showing relationships). In the theoretical sense of reasoning, research is a matter of using concepts, conceptual systems, constructs, models, theories, etc. An event can be studied effectively if the researcher generates a telling question. Such questions require key concepts or generative ideas which lead and guide the inquiry. Facts without concepts are blind and concepts without facts are empty (apologies to Kant). Research brings together conceptual systems and techniques. Appraisal of research is methodology (Kaplan, 1964). Gage (1964) identified three aspects of the research paradigm: the substantive, the methodological, and the logistical* Substantive means the basic concerns of the research; this category is further analyzed by Gage into concepts, variables, and the ways in which phenomena are selected, abstracted, and focused by the research workers. We call this area the context for inquiry. Expanding Gages category permits a more complete consideration of what must necessarily be thought about within the research paradigm. Gages second category is the methodological, in which he included ways of measuring variables and of collecting and analyzing data. We prefer to call this category methods of work because, as noted in the Foreword to this volume, we reserve the term methodology for the study of methods. Logistical, Gages third category, means the host of concerns with men and money, organization and administration, etc., that are necessary in planning, executing, interpreting, communicating, applying, and improving research. We do not discuss this important category here.

The new handbook of teacher evaluation : assessing elementary and secondary school teachers

Teacher Evaluation in Transition - Linda Darling-Hammond Emerging Roles and Evolving Methods PART ONE: PURPOSES OF EVALUATION Intended and Unintended Consequences - Gary Natriello Purposes and Effects of Teacher Evaluation Preservice Evaluation of Teachers - Walt Haney Licensure and Certification of Teachers - Gary Sykes An Appraisal Teacher Selection - Michael Scriven Assistance and Assessment for Beginning Teachers - Kenneth D Peterson Beyond Minimum Competence - Daniel L Duke and Richard J Stiggins Evaluation for Professional Development Evaluating Teachers for Career Awards and Merit Pay - Samuel B Bacharach, Sharon C Conley and Joseph B Shedd Evaluation for Tenure and Dismissal - Edwin M Bridges Teacher Evaluation for School Improvement - Edward F Iwanicki PART TWO: METHODS OF EVALUATION Classroom Observation - Susan S Stodolsky Teacher Ratings - Thomas L Good and Catherine Mulryan A Call for Teacher Control and Self-Evaluation Self-Assessment - Larry W Barber Using Student Test Scores to Evaluate Teachers - Gene V Glass The Schoolteachers Portfolio - Tom Bird An Essay on Possibilities Conventional Tests for Licensure - George Madaus and William A Mehrens Performance Tests, Simulations, and Other Methods - Edward H Haertel Setting Standards on Teacher Certification Tests - Richard M Jaeger Combining Evaluation Data from Multiple Sources - William A Mehrens PART THREE: CROSS-CUTTING PERSPECTIVES Legal Issues Concerning Teacher Evaluation - Michael A Rebell The Ethics of Educational Evaluation - Kenneth A Strike Governance Issues in the Evaluation of Elementary and Secondary School Teachers - Arthur E Wise and Tamar Gendler Economic Aspects of Teacher Evaluation - Stephen A Hoenack and David H Monk Embracing Contraries - Milbrey Wallin McLaughlin Implementing and Sustaining Teacher Evaluation Using the Personnel Evaluation Standards to Improve Teacher Evaluation - Daniel L Stufflebeam and James R Sanders

Nontechnical Questions About Teacher Evaluation Systems in Elementary and Secondary Schools: A Research Agenda

l h e crucial problems to be solved in implementing and operating a successful and effective teacher evaluation system are nontechnical as well as technical. The literature on teacher evaluation does not lack for wisdom and expertise concerning the nature of the evidence required to justify evaluative judgments (Millman, 1981). It is, however, one thing to understand the logic and techniques for collecting data and constructing warranted evaluative arguments, and another to implement and operate a successful and effective system in a real school district populated with real people. The latter task requires, in addition to technical competence, the ability to manage a complex organization, to resolve conflicting interests, and to make and enforce difficult and sometimes painful decisions in a just and humane way. Any decision concerning teacher evaluation needs to be rooted in a clear conception of why teacher evaluation systems exist. Most people would agree that the primary goal of teacher evaluations is to improve the quality of education that children receive. To reach this distant goal, two major, proximate roles of teacher evaluation have been distinguished: the formative role and the summative role. Formative teacher evaluation helps teachers improve their performance by providing data, judgments, and suggestions that have implications for what to teach and how. On the other hand, summative teacher evaluation serves administrative decisionmaking with respect to teacher certification, hiring and firing, promotion and tenure, assignments, and salary. The choice of which role the evaluation system will play is important because the motivation for, and likely acceptability of, the system is a function of its purpose, and because the optimum design features and implementation procedures differ for formative and summative teacher evaluation. Discussed below are questions about the context for teacher evaluation that raise such issues as implementation, acceptance, administrative feasibility, and justice. We believe these questions are particularly important for those seeking to study teacher evaluation systems. 1. What kinds of evaluation practices are consistent with the legal context in which schools operate? The overriding question about legal issues and teacher evaluation is: How best can the legal rights of teachers and the public be assured within an evaluation system that efficiently provides useful and accurate information? First we provide some background and then some specific research questions. Legal concerns are few in formative teacher evaluation because legal rights are not substantially affected by the as-

Book Reviews : Educational Research Methods by G. D. McGrath, James J. Jelinek, and Raymond E. Wochner. New York: The Ronald Press Company, 1963. Pp. vi + 301.

Several significant studies in the area of language learning have been published by the Staats, and their discussions of language behavior display their mastery of the area. Following the extensive discussions of language are applications of learning principles and language conceptualizations to the areas of personality, motivation, social interaction, child development and training, experimental educational psychology, and behavior problems and treatment. The discussions in these areas are systematic but somewhat variable in their degrees of detail. Where experimental and also non-experimental evidence exists to support discussions, such evidence is presented; but where evidence is lacking, the authors have not hesitated to logically develop ideas in their discussions and to identify them as hypothetical. In discussing principles or their applications the authors have identified differing theoretical positions and have provided references for more intensive study of differing views, but have emphasized a consistent position rather than the theoretical problems or issues. The authors recommend Complex Human Behavior for possible use as a text in courses of introductory psychology or educational psychology, or in such specific courses as child psychology, personality, or psychology of language. However, it appears to this reviewer that because of its difficulty level this book seems more appropriate for upper level courses or for above average introductory psychology students. Its major use will probably be as a supporting reference in one of the special areas of application covered in the latter chapters of the book. It should be excellent for this purpose. The major contribution of this text is its focus on the role of language in complex human behavior. An additional contribution is its generation of research ideas in the latter chapters where applications are discussed. There is one minor dissatisfaction this reviewer finds with the book. The degree of detail and of diagrammatic exactness of the early chapters was not consistently continued in the chapters on applications. Had this been done it would have greatly increased the value of the book-especially in its stimulation of research ideas. This shortcoming is particularly disappointing in the areas of personality and intelligence testing where the rigorous and detailed application of principles of behavior and of concepts of language seems well within the scope of this text and could be of value to the testing field. ADAM MILLER