David D. Curtis

Classical Test Theory

Measurement involves the processes of description and quantification. Questionnaires and test instruments are designed and developed to measure conceived variables and constructs accurately. Validity and reliability are two important characteristics of measurement instruments. Validity consists of a complex set of criteria used to judge the extent to which inferences, based on scores derived from the application of an instrument, are warranted. Reliability captures the consistency of scores obtained from applications of the instrument. Traditional or classical procedures for measurement were based on a variety of scaling methods. Most commonly, a total score is obtained by adding the scores for individual items, although more complex procedures in which items are differentially weighted are used occasionally. In classical analyses, criteria for the final selection of items are based on internal consistency checks. At the core of these classical approaches is an idea derived from measurement in the physical sciences: that an observed score is the sum of a true score and a measurement error term. This idea and a set of procedures that implement it are the essence of Classical Test Theory (CTT). This chapter examines underlying principles of CTT and how test developers use it to achieve measurement, as they have defined this term. In this chapter, we outline briefly the foundations of CTT and then discuss some of its limitations in order to lay a foundation for the examples of objective measurement that constitute much of the book.

Learning Across the Adult Lifespan

The past two centuries have not only seen rapid growth in the population of the world, and especially in those countries in the Asia-Pacific region, but they have also seen a marked growth worldwide in the provision for, and participation in, formal education. During the latter half of the nineteenth century the establishment of free and compulsory primary level education was strongly endorsed and in many countries enforced to achieve universal participation. During the first half of the twentieth century secondary education was firmly established. This was commonly in schools that selected students for different types of education, namely, academic, vocational or general. Then, with the growth of secondary education many countries moved towards a more comprehensive type of schooling, with a participation rate of over 80 per cent of each age-cohort remaining at school to the end of the secondary phase. In the latter half of the twentieth century tertiary education was expanded through a marked increase in the number of universities to which students had access, together with a large increase in the number of technical colleges at which students received vocational education and training. Some countries have now achieved levels of participation of around 40 per cent of the age cohort enrolled in universities, and up to 30 per cent of the age cohort enrolled in technical colleges. Within the Asia-Pacific region, wide disparities exist between countries in the extent of student participation in the three phases of education, ranging from very high levels in Japan, the Republic of Korea and Australia to the very low levels found in the countries of South Asia: Afghanistan, Bangladesh, Nepal and Pakistan. Evidence obtained from research studies indicates that, in general, the extent of educational provision is dependent on the wealth of a country and in turn, the extent of provision has an influence on the further increase in wealth of a country (see Educational Expenditure and Participation in East Asia and Australia, Higher Education and Development, Financing Higher Education in the Asia-Pacific Region). Countries with low levels of participation face decisions in the expansion of their education systems about whether to give priority to the primary, secondary or tertiary phases.

Comparing Classical and Contemporary Analyses and Rasch Measurement

Four sets of analyses were conducted on the 1996 Course Experience Questionnaire data. Conventional item analysis, exploratory factor analysis and confirmatory factor analysis were used, Finally, the Rasch measurement model was applied to this data set. This study was undertaken in order to compare conventional analytic techniques with techniques that explicitly set out to implement genuine measurement of perceived course quality. Although conventional analytic techniques are informative, both confirmatory factor analysis and in particular the Rasch measurement model reveal much more about the data set, and about the construct being measured. Meaningful estimates of individual students’ perceptions of course quality are available through the use of the Rasch measurement model. The study indicates that the perceived course quality construct is measured by a subset of the items included in the CEQ and that seven of the items of the original instrument do not contribute to the measurement of that construct. The analyses of this data set indicate that a range of analytical approaches provide different levels of information about the construct. In practice, the analysis of data arising from the administration of instruments like the CEQ would be better undertaken using the Rasch measurement model.

Enhancing Teacher Education Students’ Generic Skills Through Problem‐based Learning

Claims made for the value of problem‐based learning (PBL) as an effective method for professional education programmes draw on constructivist principles of teaching and learning to achieve essential content knowledge, higher order thinking skills, and a team approach to problem‐solving through the interdisciplinary, student‐directed study of relevant professional problems. These essential outcomes of PBL (knowledge, higher order thinking, problem‐solving, and effective team skills) are also regarded more generally across higher education as desirable qualities of graduates. The evidence that these qualities are in fact fostered through PBL is growing, but the broader implications (such as the wider impact or more far‐reaching effects) of the PBL approach have yet to be examined. This paper addresses the relationship between PBL and graduate qualities in two ways. First, it reports on a study of teacher education students’ assessment of their learning through PBL over time, across four areas of skill development: knowledge building; group processes; problem‐solving; interpersonal effectiveness. Second, the paper examines these specific outcomes in terms of the more broadly defined qualities expected of Australian university graduates.

Access, quality and equity in early childhood education and care: A South Australian study

While much is known about the factors related to student performance beyond Grade 3 less is known about the factors that are related to student performance in early childhood education and the early years in primary school. As part of the ‘I go to school’ project in South Australia, this study tracked children attending integrated preschool/childcare centres – known as Children’s Centres – as they made their transition to school. Results indicated that children who attended early childhood education programs that were of higher quality – as characterised by higher staff qualifications and a greater range and more engaging childrens activities – showed a greater gain in cognitive development than children who attended lower quality programs. Findings also suggested that children who benefitted the most from attendance in these programs were children from backgrounds of greater social disadvantage than children from less disadvantaged backgrounds.

Senior secondary workplace learning and transition success in Australia

Purpose – The purpose of this paper is to examine the effectiveness of participation in workplace learning among senior secondary students in Australia. Work placements are deemed to be effective if they meet policy objectives of improving student transitions by (a) enhancing Year 12 completion rates and (b) increasing the engagement of participants in post‐school work or study. Engagement is defined as full‐time study, full‐time work, some full‐time/part‐time combination, or two simultaneous part‐time engagements (e.g. part‐time work and part‐time study).Design/methodology/approach – Propensity score matching is used to address selection bias into work placements. After controlling for numerous student background characteristics and creating equivalent comparison groups, we estimate the influence of participation in work placements on Year 12 completion and post‐school engagement.Findings – It is found that participation in work placements during Year 11 is associated with a 5.2 percent increase in Year ...

The Effect of Image Quality Training on Reading Comprehension of EFL Students Using the Keyword Method.

Imagery training has been shown to improve reading comprehension. Recent research has also shown that the quality of visual mental imagery used is important for reading comprehension. A review of literature shows that there has been relatively little detailed research on the quality of imagery used by learners, especially in the case of students learning English as a foreign language (EFL). This study was designed to examine the influence of image quality on reading comprehension in EFL students, comparing the effects of training in the use of a focused, constrained imagery relative to that of a more standard form of visual mental imagery training. The study also examined the impact of individual differences such as gender, ability to make images, working memory capacity, and motivational beliefs on the training outcome. The findings provide evidence that constrained imagery strategy training helped EFL learners improve reading comprehension. Female participants showed higher comprehension performance than males. Other individual difference variables did not emerge as having a significant impact on change in reading comprehension performance over time.

Involving parents in early childhood research as reliable assessors

ABSTRACT This paper reports findings in relation to one aspect of the ‘I Go to School’ research project carried out in South Australia which tracked children attending integrated pre-school/childcare centres as they made their transition to school. Eight centres participated in the study involving 347 children. In order to measure some of the outcomes of the children’s participation in integrated early childhood programmes, parents and teachers completed questionnaires regarding some aspects of the children’s cognitive and social development in the year the children turned four and then again in the term they started school at age five. The researchers were thus able to ascertain some of the changes that had occurred in the children’s development during their participation in early childhood programmes. This paper reports on the comparison between parent and teacher judgements of particular aspects of the four-year-old children’s cognitive development at one point in time. The analysis presented here compares the teacher and parent/carer judgements from a subsection of the Child Development Inventory). This comparison led to the conclusion that parent ratings of particular aspects of their four-year-old children’s cognitive development were reliable and relatively consistent when compared with those of teachers.

Teaching Mathematics and Science in English at a University in Indonesia

Indonesia is a multicultural country consisting of approximately 300 ethnic groups and a multilingual society having as many as 700 local languages. However, this diverse country shares one national language, Bahasa Indonesia, which functions as the lingua franca. Accordingly, Bahasa Indonesia has been used as the language of instruction at all levels of education across the nation since its independence from the Dutch in 1945

Post-School Pathways in Australia

Upon leaving school in Australia, young people may choose to enter the labour market directly without undertaking any formal post-school education or training programs, or they may enter the vocational and technical education (VTE) or the higher education sectors in order to develop additional skills and gain post-school qualifications.