J. Douglas Willms

Monitoring School Performance : A Guide For Educators

The social and political context of monitoring systems monitoring systems and the input-output model the estimation of school effects measuring schooling inputs schooling processes schooling outcomes design of a monitoring system analyses for an Annual Report a research program conclusions.

Class Size and Student Achievement

Schooling has multiple purposes. In the long run, higher levels of schooling are associated with higher earnings and economic mobility, better health, lower mortality rates, and greater democratic participation. For these reasons, most societies require children to attend school for a specified number of years or until they reach a certain age. Many of the benefits of schooling occur in part because students learn some new knowledge or skills that enhance their ability to communicate, solve problems, and make decisions. Much of the debate over schooling is essentially about how to maximize the amount of student learning, typically as measured by various assessment instruments such as standardized achievement tests. From a societal viewpoint, since resources—most notably, time—are required for learning, and are scarce, the amount of learning needs to be maximized at least cost. Learning is complex, involving cognitive processes that are not completely understood. Typically, school systems have established a primary mode of learning that involves groups of students of about the same age interacting with a single individual leading activities in a confined physical space, directed toward learning a particular topic—in other words, students are placed in classes. The number of other students in the class can vary. At the extreme, there can be one or more adults facilitating learning—teachers—with one or two students. At the other, a student may be one of a few hundred being taught by a single instructor (or, with new Internet technology, one of millions). The number of students in a class has the potential to affect how much is learned in a number of different ways. For example, it could affect how students interact with each other—the level of social engagement. This may result, for example, in more or less noise and disruptive behavior, which in turn affect the kinds of activities the teacher is able to promote. It could affect how much time the teacher is able to focus on individual students and their specific needs rather than on the group as a whole. Since it is easier to focus on one individual in a smaller group, the smaller the class size, the more likely individual attention can be given, in theory at least. The class size could also affect the teacher’s allocation of time and, hence, effectiveness, in other ways, too—for example, how much material can be covered. Teachers may choose different methods of teaching and assessment when they have smaller classes. For example, they may assign more writing, or provide more feedback on students’ written work, or use open-ended assessments, or encourage more discussions, all activities that may be more feasible with a smaller number of students. Exposure to a particular learning environment may affect learning over the time period of exposure, or it may have longer term or delayed effects (e.g., by increasing self-esteem or cognitive developments that have lasting effects). For these reasons, changes to the class size are considered a potential means of changing how much students learn. Not only is class size potentially one of the key variables in the “production” of learning or knowledge, it is one of the simplest variables for policymakers to manipulate. However, the amount of student learning is dependent on many different factors. Some are related to the classroom and school environment in which the class takes place, but others are related to the student’s own background and motivation and broader community influences. When we ask whether class size matters for achievement, it is essential to ask also, how class size matters. This is important for three reasons. First, if we can observe not only achievement differences, but also the mechanisms through which the differences are produced, this will increase our confidence that the differences are real, and not an artifact of some unmeasured or inadequately controlled condition. Second, the effects of class size may vary in different circumstances, and identifying how class size affects achievement will help us to understand why the effects of class size are variable. Third, the potential benefits of class-size reduction may be greater than what we observe. For example, suppose class-size reductions aid achievement, but only when teachers modify instructional practices to take advantage of the smaller classes. If a few teachers make such modifications, but most do not, then understanding how class size affects achievement in some cases will help reveal its potential effects, even if the potential is generally unrealized.

Alert and inert clients: The Scottish experience of parental choice of schools

Abstract In 1980 the British Government passed legislation giving parents the statutory right to place requests for schools outside their designated catchments. The study examines the characteristics of Scottish parents who exercised choice 2 years after the legislation had become operational. It also examines the social class intakes of the schools they chose, and asks whether their choices increased the likelihood of their children attaining better results on the national certificate examinations. The findings show that parents who exercised choice were more highly educated and had more prestigious occupations than those who sent their child to the designated school. Choosers tended to select schools with higher mean socioeconomic status and higher mean levels of attainment. However, the chosen schools did not differ substantially from designated schools in their attainment, once account had been taken of the background characteristics of pupils entering them. The results suggest that the choice process is increasing between-school segregation.

Catholic-School Effects on Academic Achievement: New Evidence from the High School and Beyond Follow-Up Study.

This study compares the growth in academic achievement of publicand Catholic-school students during their last two years of high school. The study uses the High School and Beyond data on over 20,000 students who participated in the 1980 base-year study and the 1982 follow-up study. Previous studies of publicand private-school effectiveness, including the controversial Coleman, Hoffer, and Kilgore (1981) study, have been limited to the 1980 cross-sectional data, which do not include information on student ability prior to entering high school. The follow-up data enable a comparison of the growth in achievement on outcome measures that were specifically designedfor that purpose. I estimated Catholic-school effects with four different statistical models. Each model regresses senior test scores on sophomore pretest scores; sophomore test scores in reading, vocabulary, and general mathematics; and a set of background variables describing student characteristics andfamily background. Three of the models include background measures from previously published studies; the fourth model is a parsimonious model including only the most important control variables. I conducted several subsidiary analyses to ensure that the results were not substantially biased due to such factors as the method of handling missing data, the differential dropout rate between the two sectors, and the ceiling effects on the outcome measures. I also examined the extent to which the achievement tests were valid indicators of student growth in academic achievement. The results suggest that there are no pervasive Catholic-school effects. Public schools had a small advantage in science and civics; Catholic schools had an advantage in reading, vocabulary, mathematics, and writing. All of the effects were very small: On average, the Catholic-school advantage was only about 5 percent of a standard deviation. However, the analyses also suggest that the tests are relatively poor measures of academic growth during the junior and senior years of high school. Therefore, we cannot be certain that the tests are sensitive enough to detect differences that might exist between public and Catholic schools in their effects on student achievement.

The Impact of Racial and Ethnic Segregation on the Achievement Gap in California High Schools

This study examines the extent and impact of racial and ethnic segregation in California high schools during the 1988–1989 school year. We find that racial and ethnic segregation is widespread in California and that the extent of segregation varies widely among ethnic groups and among the six largest school districts. We also find significant differences in achievement levels across school districts and across schools within school districts, even after adjusting for differences in the background characteristics of students. The analysis suggests that segregation can, but does not always, lead to achievement differences across schools and among ethnic groups.

Multilevel modelling of hierarchical data in developmental studies.

This report attempts to give nontechnical readers some insight into how a multilevel modelling framework can be used in longitudinal studies to assess contextual influences on child development when study samples arise from naturally formed groupings. We hope to achieve this objective by: (1) discussing the types of variables and research designs used for collecting developmental data; (2) presenting the methods and data requirements associated with two statistical approaches to developmental data--growth curve modelling and discrete-time survival analysis; (3) describing the multilevel extensions of these approaches, which can be used when the study of development includes intact clusters or naturally formed groupings; (4) demonstrating the flexibility of these two approaches for addressing a variety of research questions; and (5) placing the multilevel framework developed in this report in the context of some important issues, alternative approaches, and recent developments. We hope that readers new to these methods are able to visualize the possibility of using them to advance their work.

Equalisation and Improvement: Some Effects of Comprehensive Reorganisation in Scotland

This study examines the effects of the later stages of comprehensive reorganisation in Scotland. Nationally representative samples show that standards of attainment rose, particularly among females and pupils of lower socioeconomic status (SES). The rise was faster than could be accounted for by the rise in average SES. Inequalities of attainment associated with SES varied between communities but, overall, they declined for both males and females. Female attainment moved ahead of male attainment. Creamed comprehensives did worse than uncreamed ones, but the longer that either creamed or uncreamed schools had been established as comprehensive the higher was their average attainment. Some implications for the reduction and reproduction of inequalities of attainment are discussed.

How Effective Are Private Schools in Latin America

Using multilevel modelling and data from 10 Latin American countries, this paper provides new evidence on the relative effectiveness of public and private schools. There are substantial differences in the achievement of private and public schools, usually around one-half a standard deviation. A small portion of these differences is accounted for by the higher socioeconomic status of students in private schools. A quite substantial portion is explained by the varying peer group characteristics in private and public schools. After accounting for peer characteristics, the average private school effect across all 10 countries is zero, though with some variance around this mean (the effects range between -0.2 and 0.2 standard deviations). Evidence on selection bias is inconclusive, but the paper argues that these effects may constitute an upper bound to the true effects.

Reasons for school choice in Scotland

Abstract This study examines the reasons a sample of Scottish parents gave for rejecting their designated secondary school and choosing a different one. A consistent theme in the findings is the importance of parental socio‐economic status (SES). It was related to the reasons for rejecting the local school and for selecting a different school, and to the degree of importance accorded different sources of information. Reasons pertaining to social and reputational factors and to disciplinary climate were the dominant themes for both rejection and selection of secondary schools. The academic quality of the school was of lesser importance. The results suggest that an educational marketplace that parallels the free‐market model works only for a limited subgroup of the population that is already advantaged in the present educational system.

Monitoring School Performance for 'Standards-based Reform'

Introduction For several years, stateDepartments of Education and many schooldistricts in the US have expended considerable resources collecting and analysing data that describe the performance of their schools. In most cases, they have established monitoring systems that entail annual or biennial achievement testing, and the collection of demographic data. In a few cases, data describing schooling processes, such as homework policies, classroom discipline, and parental involvement, are also collected. The aim of these monitoring systems is to garner information that will discern how well students in their jurisdiction fare compared with those in other jurisdictions; assess whether the achievement of learning outcomes in the jurisdiction is changing over time; and determine whether there are inequalities in achievement among students with differing ethnic or socioeconomic backgrounds. This kind of information is useful to school, district, and state administrators for making day-to-day decisions concerning the allocation of resources, the efficacy of certain programmes, and the strengths and weaknesses of their schooling system. But underlying this activity is the belief that routine monitoring encourages a process of self-examination and motivates educators to provide a better education. Administrators believe that monitoring can drive school improvement. This belief is fundamental to the recent calls for ‘standards-based reform’, which maintains that if states set high standards for performance, and hold schools accountable to meeting those standards through performance-based monitoring, then teachers and schools will change their practices in ways that bring about better student outcomes. An important principle underlying the case for standards-based reform is that the means for achieving the standards set by the state should be left to the schools; that is, school principals and teachers should be given the autonomy and flexibility to change their organisational structures, curriculum, and teaching practices as they deem appropriate (Rothman & Elmore, 1998). Thus, the emphasis of monitoring efforts is on student outcomes, not on schooling processes. The case for standards-basedreform also relies heavily on the assumption that it is possible to measure student performance against prescribed standards. Several prominent educational researchers have made a convincing case that traditional norm-referenced assessment techniques cannot achieve this aim, mainly because they emphasise lower-order thinking skills, isolated from any relevant social context. Moreover, longitudinal studies have shown that test