Maria Eugénia Ferrão

E‐assessment within the Bologna paradigm: evidence from Portugal

The Bologna Declaration brought reforms into higher education that imply changes in teaching methods, didactic materials and textbooks, infrastructures and laboratories, etc. Statistics and mathematics are disciplines that traditionally have the worst success rates, particularly in non‐mathematics core curricula courses. This research project, Mathematics and Statistics for the Development of Professional Skills, which is in progress at the University of Beira Interior in Portugal, has as one of its main objectives the development of the e‐assessment system as a resource for learning assessment and student self‐regulation. Based on the results of the above‐mentioned project, this paper will give evidence of how to improve the reliability of an e‐assessment system, show that e‐assessment can be a good alternative to open‐ended tests and that students tend to show a positive attitude towards its use. We show that this can be done by checking the internal consistency and measurement error of the e‐assessment tests, the analysis of the association between student scores obtained by different methods of assessment and the analysis of data survey on student opinion about e‐assessment methods.

The use of a school value-added model for educational improvement: a case study from the Portuguese primary education system

This article focuses on the use of a value-added approach for promoting school improvement. It presents yearly value-added estimates, analyses their stability over time, and discusses the contribution of this methodological approach for promoting school improvement programmes in the Portuguese system of evaluation. The value-added model is applied to longitudinal data gathered from a representative sample of students enrolled in primary school, using mathematics scores as outcome variable. The estimates are used to determine a systematic positive or negative school value-added effect. Results indicate that at least 41% of schools remain at the same level over 3 years, 15% present an instable pattern, and approximately 40% show a positive or negative trend. It is argued that these results, especially those that reveal systematically underperforming schools, could have relevant implications for educational policy geared towards school improvement.

Avaliação educacional e modelos de valor acrescentado: tópicos de reflexão

Since the beginning of the 21st century that we face a growing wave towards the implementation of educational evaluation systems with high-stake accountability purposes at the level of teachers and schools. The development and implementation of such systems is quite fast, either in terms of the influence of the educational evaluation on public policies, or in terms of the public policy evaluation. In this context, the use of the value-added concept is consensually accepted. This paper presents some reflections on its implementation and use, emerging themes such as the model choice, school and student outcomes, and its appropriateness by practitioners. In addition, some examples are given where the misuse of performance indicators had adverse consequences to education.

Avaliação do Autoconceito Infantil

Percepcao do Autoconceito Infantil (PAI) e o nome de um instrumento desenvolvido por Sanchez y Escribano (1999) destinado a avaliacao do autoconceito de criancas entre os tres e os 10 anos de idade (Pre-primaria e 1o Ciclo do Ensino Basico). Tratase de um instrumento de medida de facil aplicacao e correccao, de aplicacao tanto individual como colectiva e, que no seu constructo inclui dez dimensoes caracteristicas do auto-conceito infantil. A adaptacao portuguesa que agora se apresenta foi desenvolvida no âmbito do Projecto Eficacia Escolar no Ensino da Matematica (3EM), junto de um grupo de 954 alunos do 1o ciclo do ensino basico que fazem parte de uma amostra constituida na regiao da Cova da Beira. Os resultados obtidos permitem-nos constatar que esta versao de aplicacao colectiva apresenta uma fiabilidade razoavel (K-R 20 = 0.79) e uma estrutura factorial adequada.

Generalized Partial Credit Item Response Model: Linking scales in the assessment of learning

Abstract The paper presents a linking method between two different scales developed to measure the same construct, so that one of them may be the anchor to compare educational results over time. The Generalized Partial Credit Item Response Model is applied to multiple choice items, short answer, completion and open response items. The results suggest that scores are interchangeable. The statistical model is applied to obtain a common scale to allow the comparison of scores over time in Mathematics. It is applied to the area of educational assessment, but from the conceptual, methodological and operational point of view, it may be extended to other areas such as professional accreditation and certification, human resources evaluation and recruitment, health screening and diagnosis, psychological and behavioral evaluation.

Item Response Models in Computerized Adaptive Testing: A Simulation Study

In the digital world, any conceptual assessment framework faces two main challenges: (a) the complexity of knowledge, capacities and skills to be assessed; (b) the increasing usability of web-based assessments, which requires innovative approaches to the development, delivery and scoring of tests. Statistical methods play a central role in such framework. Item response models have been the most common statistical methods used to address such kind of measurement challenges, and they have been used in computer-based adaptive tests, which allow the item selection adaptively, from an item pool, according to the person ability during test administration. The test is tailored to each student. In this paper we conduct a simulation study based on the minimum error-variance criterion method varying the item exposure rate (0.1, 0.3, 0.5) and the test maximum length (18, 27, 36). The comparison is done by examining the absolute bias, the root mean square-error, and the correlation. Hypotheses tests are applied to compare the true and estimated distributions. The results suggest the considerable reduction of bias as the number of item administered increases, the occurrence of ceiling effect in very small size tests, the full agreement between true and empirical distributions for computerized tests of length smaller than the paper-and-pencil tests.

Repetência e indisciplina: evidências de Brasil e Portugal no Pisa 2012

Los principales objetivos de este trabajo son analizar la repitencia en Brasil y Portugal a traves de los datos del Programa para la Evaluacion Internacional de Estudiantes [Programme for International Student Assessment] -Pisa 2012 y aplicar un modelo de regresion logistica multinivel en el que la repitencia sea la variable dependiente, en el sentido de identificar caracteristicas de estudiantes y escuelas asociadas con la probabilidad de repitencia a fin de estimar la variabilidad a traves de las escuelas. Entre los principales resultados, la contribucion mas significativa de este estudio es la de explicitar una clara asociacion entre la repitencia y la indisciplina escolar. Es necesario seguir trabajando sobre la base de datos longitudinales para investigar en detalle las caracteristicas de los estudiantes y las escuelas asociadas con el cambio educativo, que influyen en la probabilidad de repitencia.

Sobre a complementaridade da teoria clássica dos testes e dos modelos de resposta ao Item: estimativas da dificuldade do item e testes adaptativos computarizados

O presente artigo tem por objetivo fornecer evidencia estatistica sobre a complementaridade entre a teoria classica dos testes e os modelos de resposta ao item para determinados fins de avaliacao educacional. Essa complementaridade pode contribuir para o desenvolvimento futuro de processos inovadores de calibracao dos items no contexto de testes adaptativos, a custo reduzido. A teoria classica dos testes e o modelo de resposta ao item de credito parcial generalizado sao aplicados a testes compostos por items de multipla escolha, de resposta curta, de completamento e de resposta aberta, parcialmente classificados. Os conjuntos de dados advem dos testes realizados junto da populacao portuguesa de estudantes inscritos no 4o e no 6o ano. Os intervalos de confianca de 95% baseados em 1.000 amostras bootstrap revelam uma forte associacao entre as estimativas da dificuldade do item, corroborando a teoria estatistica de testes psicologicos.

Concomitant Latent Class Models Applied to Mathematics Education

The research project School Effectiveness in Teaching-Learning of Mathematics allowed a longitudinal study in the primary, elementary and lower secondary education which was conducted between 2004 and 2009 in Portugal. It stated as one of the specific objectives the development and promotion of quantitative methods in education, particularly in mathematics education. This chapter presents a latent class model with concomitant variables applied to the data of a paired sample (data collected at the beginning and at the end of the academic year) of 276 students enrolled in the 7th grade. The response variable represents whether learning has or has not occurred during the year and the concomitant variables are scores to assess the level of fluid intelligence components. Model parameter estimates suggest that there are two distinct latent classes explained by verbal and spatial reasoning.