Diego Zapata-Rivera

Adaptive Technologies for Training and Education: Adaptive Educational Systems

This paper elaborates on the need for adaptive educational systems, describes an evidence-based, four-process framework for analyzing adaptive educational systems, and reviews several adaptive technologies and types of adaptive learning environments. In addition, interviews with experts in the field inform our discussion on what to adapt, how to adapt, and the future of adaptive educational systems.

Exploring Various Guidance Mechanisms to Support Interaction with Inspectable Learner Models

Open or inspectable learner models have been used to support reflection, knowledge awareness, learner model accuracy, and negotiated assessment. Current systems employ many different mechanisms to present and to support human interaction with the learner model. This paper explores the interactions between learners and inspectable learner models using various guidance mechanisms (i.e. following a protocol, interacting with human peers and artificial guiding agents, interacting with the teacher, and exploring the model as part of a group). We report on a study carried out with fifth grade students who interacted with different configurations of inspectable learner modelling environments.

Assessing Science Inquiry Skills Using Trialogues

Trialogue-based tasks can be used to gather evidence that may be difficult to obtain using traditional assessment approaches, such as embedded questions. However, more research needs to be done in order to create valid, fair, and reliable conversation tasks that can be used for assessment purposes. This paper describes ongoing efforts at developing and evaluating trialogues for assessing students’ science inquiry skills.

Interlanguage Pragmatics with a Pedagogical Agent: The Request Game

This article describes the development and evaluation of a computer-assisted language learning approach which integrates a finite state dialogue engine with an animated pedagogical agent. The design of the request game is theoretically motivated by interlanguage pragmatics and Longs Interaction Hypothesis. The tutoring system creates a venue in which a student can practice pragmatic routines and particular pragmatic features in simulated academic situations. The system was programmed to recognize students written messages so that they can receive immediate and relevant feedback on their output. To test the usability of the system, a prototype of the request game has been tested with English learners in an English as a second language program in the US. The article concludes with the preliminary evaluation results of the system.

Exploring the Effectiveness of a Measurement Error Tutorial in Helping Teachers Understand Score Report Results

The goal of this study was to explore the effectiveness of a short web-based tutorial in helping teachers to better understand the portrayal of measurement error in test score reports. The short video tutorial included both verbal and graphical representations of measurement error. Results showed a significant difference in comprehension scores between each of two tutorial groups (basic and enhanced) and the control group (no tutorial) but not between the two tutorial groups. Results also provided evidence of teachers misconceptions about the meaning of measurement error and confidence bands.

Combining Learning and Assessment in Assessment-Based Gaming Environments: A Case Study from a New York City School.

Purpose – The purpose of this paper is to propose and demonstrate an evidence‐based scenario design framework for assessment‐based computer games.Design/methodology/approach – The evidence‐based scenario design framework is presented and demonstrated by using BELLA, a new assessment‐based gaming environment aimed at supporting student learning of vocabulary and math. BELLA integrates assessment and learning into an interactive gaming system that includes written conversations, math activities, oral and written feedback in both English and Spanish, and a visible psychometric model that is used to adaptively select activities as well as feedback levels. This paper also reports on a usability study carried out in a public middle school in New York City.Findings – The evidence‐based, scenario design framework proves to be instrumental in helping combine game and assessment requirements. BELLA demonstrates how advances in artificial intelligence in education, cognitive science, educational measurement, and vid...

Assessing Science Inquiry Skills in an Immersive, Conversation-Based Scenario

Innovative, interactive tasks that include conversations among humans and virtual (pedagogical) agents can be used to assess relevant cognitive skills (e.g., scientific inquiry skills). These new assessment systems aid the collection of additional information (e.g., timing data, information about conversation path sequences, and amount of help used) that provide the context for assessment and can inform assessment claims in these specific environments. In order to assess science skills, we have implemented and evaluated a game-like assessment with embedded conversations called the Volcano Scenario. This chapter describes the Volcano Scenario and highlights the techniques used to collect and analyze the data generated by the system. A hybrid approach to analyzing data from interactive, assessment environments that makes use of traditional psychometric analysis and several big data-related processes is described and illustrated through the analyses of data from 500 participants who have at least a year of college experience.

Adaptive, assessment-based educational games

Assessment-based educational games can produce useful information to guide student instruction This paper describes an approach for integrating components of video games with those of adaptive technologies and assessment into the design of educational games Three examples in the areas of English language learning and mathematics are also presented.

Interpretation of Score Reports by Diverse Subgroups of Parents

ABSTRACT The degree to which diverse score users can understand and consume information presented in score reports dictates whether they are able to draw reasonable conclusions. The goal of this study was to understand how parents (a particularly understudied and heterogeneous stakeholder population) make interpretations from a hypothetical interactive score report. We used cognitive laboratories to evaluate how 35 parents from diverse educational and language proficiency backgrounds make sense of the information presented in hypothetical K-12 score reports. Results from this study demonstrate the diversity in the levels of comprehension across the assessed subgroups of parents, and highlight the need for and importance of additional support materials that can help parents understand and make sense of the information presented in score reports. Nevertheless, due to the hypothetical nature of the score report mockup evaluated and the small sample sizes, these results should be substantiated with additional research.

Toward Caring Assessment Systems

The notion of intelligent systems that care is at the center of research in areas such as Intelligent Tutoring Systems and Adaptive Systems. This paper elaborates on the notion of caring assessment systems, and presents work towards achieving this vision that has potential for improving students assessment experiences.