Tufi Machado Soares

An Integrated Bayesian Model for DIF Analysis

In this article, an integrated bayesian model for differential item functioning (DIF) analysis is proposed. The model is integrated in the sense of modeling the responses along with the DIF analysis. This approach allows DIF detection and explanation in a simultaneous setup. Previous empirical studies and/or subjective beliefs about the item parameters, including differential functioning behavior, may be conveniently expressed in terms of prior distributions. Values of indicator variables are estimated in the model, indicating which items have DIF and which do not; as a result, the data analyst may not be required to specify an “anchor set” of items that do not exhibit DIF a priori to identify the model. It reduces the iterative procedures that are commonly used for proficiency purification and DIF detection and explanation. Examples demonstrate the efficiency of this method in simulated and real situations.

Utilização da teoria da resposta ao item na produção de indicadores sócio-econômicos

In Brasil, the Item Response Theory (IRT) has been used on the production of the pupils proficience in large scale assessment. However, its different kinds of models may be used to build others indices as is the case of socio-economic status index. This study presents and compares some techniques based on IRT that can lead to these indices. As a case study, the socio-economic index of the pupils family, who have participated of the SIMAVE-2002 - assessment program of the Minas Gerais State, is produced. In addition, it is given some suggestions and guidelines for future socio-economic status index that will be produced in SIMAVE.

Modelo de três níveis hierárquicos para a proficiência dos alunos de 4ª série avaliados no teste de língua portuguesa do SIMAVE/PROEB-2002

In this paper, three-level hierarchical regression models (the first level represents the students; the second level, the classrooms; and the third level, the schools) are produced to explain levels of proficiency achieved in the Portuguese language assessment of the PROEB/SIMAVE-2002 - the Minas Gerais State Programme for the Assessment of Elementary Education, by students in the 4th year of elementary school. Results show that there is large variability explained by classroom differences, reasonable variability explained by school differences, and enormous variability explained by student differences. When the variables studied are introduced in the models, a substantial portion of the total variability is explained.

Construção de um modelo de regressão hierárquico para os dados do SIMAVE-2000

The evaluation systems developed on the last decade keep a same main objective: to find ways to efficiently improve the quality of the education. Besides identifying results of the pupils learning, the educational system is also evaluated. In this context, the Sistema Mineiro de Avaliacao da Educacao Publica (SIMAVE) was instituted in 2000 year by the Secretaria de Estado de Educacao. The present study had used multilevel models with the objective to identify the relation between the educational performance, the technic-pedagogical characteristics of the schools and the social-economic profile of the 4th. grade pupils that had made SIMAVE-2000 tests. Searching to investigate the factors that act in the classrooms differentiation in schools had been considered the multilevel models with the hierarchy levels: classroom (level 1) and school (level 2). It was found explanation for a great proportion of the variation of the scores of the classroom.

Simultaneous multifactor DIF analysis and detection in Item Response Theory

Item Response Theory (IRT) is a psychometric theory widely used in educational assessment and cognitive psychology to analyse data emerged from answers given to items contained in exams, questionnaires, etc. Standard IRT, however, is based on models which assume that items behave equally to all individuals. This may not be a reasonable assumption, especially when the individuals taking the test have different social and/or cultural backgrounds. Differential Item Functioning (DIF) is an area of IRT which allows an item to be perceived differently by distinct groups, respecting its usual characteristics. DIF hypothesis avoids neglecting items that may behave differently among groups and may also be used to provide important information about differences in the populations involved in the study. In this paper, two integrated Bayesian models for DIF analysis in IRT are proposed and compared. Both models are based on a two component mixture with one component describing DIF and the other accounting for the absence of DIF. Another contribution of this paper is the approach of the simultaneous presence of multiple factors causing DIF. Ideas from ANOVA models are used to characterize different possibilities associated with these factors. The models are also extended to account for explanation and detection in each factor. A simulation study was conducted to assess the models capabilities and to compare it against existing alternatives. Special attention has been directed to the conditions required to ensure model identification. An analysis of a Mathematics exam applied nationally to Brazilian elementary school students is made considering two DIF factors: geographical region and type of school. The results highlight the relevance of the proposed methodology to address important issues in educational studying and testing.

Modelagem do crescimento da aprendizagem nos anos iniciais com dados longitudinais da pesquisa GERES

Este artigo compara duas abordagens de valor agregado para dados oriundos do survey educacional de recorte longitudinal, chamado GERES - Estudo Longitudinal da Geracao Escolar 2005, que acompanhou uma coorte de alunos de mais de 300 escolas publicas e privadas ao longo dos primeiros quatro anos do Ensino Fundamental. Ambas as abordagens utilizam modelos lineares hierarquicos, permitindo o agrupamento natural dos dados educacionais provenientes dos tres niveis: aluno, turma e escola. Na primeira abordagem de valor agregado, constroem-se modelos cuja variavel dependente e a proficiencia do aluno em cada ano avaliado. Com um modelo distinto para cada ano e possivel detectar fatores do aluno, da turma e da escola associados ao desempenho dos alunos. A segunda abordagem cria modelos para mostrar o efeito das covariaveis de aluno, turma e escola nas curvas de evolucao da proficiencia ao longo do periodo do estudo. Quando comparados os dois tipos de modelos de valor agregado, o primeiro foi o mais eficiente em diagnosticar os efeitos do ambiente e da pratica pedagogica do professor, mas somente em determinados anos. Ja o segundo tipo de modelo foi capaz de identificar curvas de evolucao de proficiencia de formatos distintos de acordo com determinadas caracteristicas das escolas e dos alunos, mas foi menos sensivel na identificacao de variaveis associadas ao processo de formacao de grupos e a pratica pedagogica do professor. Os dois tipos de modelos de valor agregado oferecem indicacoes de processos de aprendizagem diferenciados para as disciplinas Lingua Portuguesa e Matematica que mereceriam estudos adicionais.

Análise da evasão no ProJovem Urbano: uma abordagem através do Modelo de Regressão Logística Multinível

Apesar das grandes conquistas verificadas no Brasil nas ultimas decadas em Educacao, o sistema educativo brasileiro ainda apresenta caracteristicas estruturais que constituem obstaculo ao seu desenvolvimento social e economico: as elevadas taxas de retencao e abandono escolar. Como consequencia, o nivel de instrucao da populacao em idade ativa e baixo. Sob a responsabilidade da Presidencia da Republica, foi desenhado o programa ProJovem Urbano, cuja finalidade prioritaria e promover a inclusao social dos jovens brasileiros de 18 a 29 anos que, apesar de alfabetizados, nao concluiram o Ensino Fundamental. Nao obstante o programa atrair o interesse de elevado numero de jovens, a desistencia e evasao que se verificam ao longo do programa sao muito elevadas. Este trabalho tem como principal objetivo identificar os grupos de maior incidencia da evasao, tendo como proposito apoiar a implementacao de medidas que a reduzam. Nestes termos, usando um modelo logistico multinivel, sao exploradas as seguintes caracteristicas da populacao abrangida: sexo, idade, raca-cor, ultima serie frequentada, nivel de conhecimento em Matematica e Lingua Portuguesa a entrada no Programa, nivel socioeconomico e a regiao metropolitana. Os resultados sugerem que os grupos de maior incidencia da evasao sao definidos pelas variaveis sexo, idade e regiao.* Professor do Departamento de Estatística, Universidade Federal de Juiz de Fora (UFJF); Coordenador de Pesquisa do Centro de Políticas Públicas e Avaliação da Educação (CAEd); Professor do Programa de Pós-Graduação em Educação, UFJF. ** Professora de Estatística, Universidade da Beira Interior (Portugal); Investigadora do Centro de Matemática Aplicada à Previsão e Decisão Econômica (CEMAPRE), Centro de Investigação do Instituto Superior de Economia e Gestão, Universidade Técnica de Lisboa. *** Professor do Departamento de Fundamentos da Educação, Universidade Federal do Ceará (UFC); Coordenador do ProJovem Urbano para os Estados do Ceará, Piauí e Maranhão; Professor do Programa de Pós-Graduação em Educação, UFC.

Análise bayesiana do funcionamento diferencial do item

This paper uses a Bayesian approach for parameter estimation in Item Response Theory Models for DIF - Differential Item Functioning - analysis. The models proposed are integrated, and incorporate regression structures that can be used to explain the DIF related to items associated covariates. The models are proposed for multiple groups and the approach used, naturally, consider the estimation error of the latent trace and the estimation error of the structural parameters. Examples with simulated data and real data are also presented.

Escola de Tempo Integral: resultados do projeto na proficiência dos alunos do Ensino Fundamental das escolas públicas da rede estadual de Minas Gerais

Neste trabalho avaliam-se os efeitos do Projeto Escola de Tempo Integral, desenvolvido pelo Governo do Estado de Minas Gerais desde o ano de 2007 nas escolas estaduais mineiras, nos resultados das avaliacoes em larga escala de Lingua Portuguesa e Matematica. Nesse sentido, o projeto tem apresentado resultados positivos ao promover as proficiencias dos alunos com defasagens de aprendizagem. O estudo observacional foi baseado nos resultados dos alunos de cerca de 1900 escolas que aderiram ao projeto entre 2006 e 2009, utilizando tecnicas baseadas em escore de propensao e modelos multiniveis de regressao.

Bayesian item response model: a generalized approach for the abilities' distribution using mixtures

ABSTRACT Traditional Item Response Theory models assume the distribution of the abilities of the population in study to be Gaussian. However, this may not always be a reasonable assumption, which motivates the development of more general models. This paper presents a generalized approach for the distribution of the abilities in dichotomous 3-parameter Item Response models. A mixture of normal distributions is considered, allowing for features like skewness, multimodality and heavy tails. A solution is proposed to deal with model identifiability issues without compromising the flexibility and practical interpretation of the model. Inference is carried out under the Bayesian Paradigm through a novel MCMC algorithm. The algorithm is designed in a way to favour good mixing and convergence properties and is also suitable for inference in traditional IRT models. The efficiency and applicability of our methodology is illustrated in simulated and real examples.Traditional Item Response Theory models assume the distribution of the abilities of the population in study to be Gaussian. However, this may not always be a reasonable assumption, which motivates the development of more general models. This paper presents a generalised approach for the distribution of the abilities in dichotomous 3-parameter Item Response models. A mixture of normal distributions is considered, allowing for features like skewness, multimodality and heavy tails. A solution is proposed to deal with model identifiability issues without compromising the flexibility and practical interpretation of the model. Inference is carried out under the Bayesian Paradigm through a novel MCMC algorithm. The algorithm is designed in a way to favour good mixing and convergence properties and is also suitable for inference in traditional IRT models. The efficiency and applicability of our methodology is illustrated in simulated and real examples.