Jesús Miguel Rodríguez-Mantilla

Evaluation of the impact of intervention programmes on education organisations: Application to a Quality Management System

This paper analyses the importance of evaluating the various components of the programmes or actions carried out by education organisations. It highlights the need to assess the impact of the intervention on the organisation and consider how changes are consolidated over time in interaction with the context. We propose an impact evaluation model and as an example have chosen the implementation of Quality Management Systems in schools. The paper analyses the results obtained in 40 schools in three regions (Spanish Autonomous Communities) with varying levels of implementation. The results show overall impact on these education centres as the teachers and management teams of the centres perceive it. This impact is more evident in some of the dimensions considered in the study than in others. The results also confirm the differences between regional contexts.

Factors associated with the impact of implementing quality management systems at schools: a multilevel analysis

The aim of this study was to analyse the simultaneous effect of a set of predictors of the impact perceived by teachers and managers of implementing two Quality Management Systems (QMS): European Foundation for Quality Management (EFQM) and ISO:9001 STANDARDS. A sample of 2869 subjects from 114 Spanish schools has been taken, to which a 91-item questionnaire has been applied to assess the perceived impact that the implementation of QMS has had on the schools. Data analysis was carried out using hierarchical-linear modelling. Results show a high level of impact in Management and Planning system and medium impact in Communication, the Learning Process and External Relations. The main predictors of the impact are the position and years of service at the school (on Level 1 -Subject-) and ownership, school size and QMS implemented (on Level 2 -Schools-), finding that the impact perceived in schools with EFQM was higher. The specialised literature shows the scarcity of studies on the impact that QMS have on organisations, especially in the education sector. Thus, this study provides conclusions to the scientific and professional community with objective evidence of predictors of the impact perceived by teachers and managers of implementing QMS in schools.

Design and validation of an instrument to measure teacher training profiles in information and communication technologies

Introduction: this study is part of a research project concerning the teacher training in information and communication technologies (ICT) profile. Its aim is to develop and validate an instrument for measuring this profile in primary and secondary schools. Methodology: after developing the instrument and administering it to a sample of 1,433 teachers in the Community of Madrid, its reliability, content, and construct validity were analysed (the latter using Structural Equation Models with the IBM SPSS-AMOS program). Results: the reliability analysis gave Cronbach’s Alpha = 0.973 for the whole of the instrument. For each dimension this figure was: Curricular Aspects, 0.738; Planning and Evaluation, 0.878; Methodological Aspects, 0.903; Use of ICT, 0.935; and ICT Training, 0.894. The discrimination coefficient values of the final instrument items ranged from 0.33 to 0.74. The Confirmatory Factor Analysis demonstrates a good fit of the model to the data (CMIN/DF = 5,138; CFI = 0,905; RMSEA = 0,056; PRATIO = 0,928). Conclusions: this instrument has therefore shown that it has the necessary technical characteristics to be considered a valid and trustworthy tool for measuring the teacher training profile in ICT.

PISA 2015 : predictores del rendimiento en ciencias en España

espanolDada la creciente expansion y uso de los resultados de los estudios de PISA sobre la evaluacion de los alumnos en sus competencias cientificas, matematicas y linguisticas, el presente trabajo tiene como objetivo analizar el efecto de un conjunto de predictores del rendimiento en Ciencias en los alumnos espanoles participantes en PISA 2015. Para el estudio se ha tomado una muestra de 32.330 alumnos de 15 anos de 17 Comunidades Autonomas y se han utilizado modelos jerarquico-lineales que permiten el analisis del posible efecto de los distintos predictores, contemplando el anidamiento de los datos en distintos niveles (Alumnos, Centro y Comunidad Autonoma). Como variables independientes se han seleccionado 64 predictores, algunos de ellos incluidos en los cuestionarios de alumnos y de centros de PISA 2015 y de la base de datos facilitada por el Ministerio de Educacion, Cultura y Deporte (2016a), como ansiedad del alumno, relacion profesor-alumno, habitos deportivos, interes hacia la ciencia, recursos de los centros, porcentaje de alumnos extranjeros y cantidad de alumnos por profesor en cada Comunidad Autonoma, entre otros. Ademas, se incluyeron variables consideradas como “clasicas” en este tipo de estudios (sexo, nivel academico de los padres, titularidad y tamano del centro, inversion economica por Comunidad Autonoma, etc.). Entre los principales resultados encontramos que 27 variables (24 de Alumno y 3 de Centro y ninguna de Comunidad Autonoma) resultaron predictores significativos del rendimiento en Ciencias, analizando la varianza explicada. El trabajo concluye con la discusion fundamentada en otros estudios coincidentes o con resultados contrarios sobre las variables que han resultado significativas y no significativas en el modelo propuesto. EnglishGiven the increasing expansion and use of the results of PISA studies on the evaluation of the scientific, mathematical and linguistic competences of the students, this paper aims to analyze the effect of a set of predictors of performance in Science in Spanish students participating in PISA 2015. For the study, a sample of 32,330 15-year-olds from 17 Autonomous Communities has been taken and hierarchical-linear models have been used that allow the analysis of the possible effect of the different predictors contemplating the nesting of the data at different levels (Students, School Center and Autonomous Community). 64 predictors were selected as independent variables, some of them included in the questionnaires of PISA 2015 students and centers and the database provided by the Ministry of Education, Culture and Sport (2016a), such as anxiety of the student, teacher-student relationship, sports habits, interest in science, resources of the centers, percentage of foreign students and number of students per teacher in each Autonomous Community, among others. In addition, variables considered as “classic” were included in this type of studies (gender, academic level of the parents, ownership and size of the center, economic investment by Autonomous Community, etc.). Among the main results we found that 27 variables (24 of Student and 3 of Center and none of Autonomous Community) were significant predictors of Science performance, analyzing the explained variance. The paper concludes with the discussion based on other coincident studies or with contrary results on the variables that have been significant and not significant in the proposed model.