Diana Pérez-Marín

Conversational Agents and Natural Language Interaction: Techniques and Effective Practices

Conversational Agents and Natural Language Interaction: Techniques and Effective Practices is a reference guide for researchers entering the promising field of conversational agents. It provides an introduction to fundamental concepts in the field, collects experiences of researchers working on conversational agents, and reviews techniques for the design and application of conversational agents. The book discusses the successes of and challenges faced by researchers, designers, and programmers who want to use conversational agents for e-commerce, help desks, website navigation, personalized service, and training or education applications.

Computer-assisted assessment of free-text answers

The automatic assessment of students’ free-text answers has recently received much attention, due to the necessity of exploring and taking advantage of new and more complex computer-based assessment methods. In this paper, a review of the state-of-art of the field is presented, focusing on the techniques that underpin these systems and their evaluation metrics. Although there is still a long way to go so as to reach the ideal system, the fact that the existing systems are already being used commercially and as a second opinion in exams such as GMAT proves the uptake of this field.

On the dynamic adaptation of computer assisted assessment of free-text answers

To our knowledge, every free-text Computer Assisted Assessment (CAA) system automatically scores the students and gives feedback to them according to their responses, but, none of them include yet personalization options. The free-text CAA system Atenea [1] had simple adaptation possibilities by keeping static student profiles [2]. In this paper, we present a new adaptive version called Willow. It is based on Atenea and adds the possibility of dynamically choosing the questions to be asked according to their difficulty level, the students’ profile and previous answers. Both Atenea and Willow have been tested with 32 students that manifested their satisfaction after using them. The results stimulate us to continue exploiting the possibilities of incorporating dynamic adaptation to free-text CAA.

Extending computer assisted assessment systems with natural language processing, user modeling, and recommendations based on human computer interaction and data mining

Willow is a free-text Adaptive Computer Assisted Assessment system, which supports natural language processing and user modeling. In this paper we discuss the benefits coming from extending Willow with recommendations. The approach combines human computer interaction methods to elicit the recommendations with data mining techniques to adjust their definition. Following a scenario-based approach, 12 recommendations were designed and delivered in a large scale evaluation with 377 learners. A statistically significant positive impact was found on indicators dealing with the engagement in the course, the learning effectiveness and efficiency, as well as the knowledge acquisition. We present the overall system functionality, the interaction among the different subsystems involved and some evaluation findings.

A Study on the Possibility of Automatically Estimating the Confidence Value of Students’Knowledge in Generated Conceptual Models

We propose a new metric to automatically evaluate the confidence that a student knows a certain concept included in his or her conceptual model. The conceptual model is defined as a simplified representation of the concepts and relationships among them that a student keeps in his or her mind about an area of knowledge. Each area of knowledge comprises several topics and each topic several concepts. Each concept can be identified by a term that the students should use. A concept can belong to one topic or to several topics. Terms are automatically extracted from the answers provided to an automatic and adaptive free-text scoring system using Machine Learning techniques. In fact, the conceptual model is fully generated from the answers provided by the students to this system. In the paper, the automatic procedure that makes it possible is reviewed in detail. Finally, concept maps are used to graphically display the conceptual model to teachers and students. In this way, they can instantly see which concepts have already been assimilated and which ones should still be reviewed.

Willow: a system to automatically assess students’ free-text answers by using a combination of shallow NLP techniques

In this paper, a new approach to automatically assessing students’ short free-text answers is presented. It has been implemented into the multilingual web-based application called Willow. Willow is able to formulate a question (chosen randomly or according to the students’ personal features) and, to score the answer, written in Spanish or in English, by using a combination of shallow NLP techniques such as stemming, removal of closed-class words, word sense disambiguation or multiword term identification. Several experiments have been carried out to identify the best combination of techniques for each language to find the optimal configuration in each case.

Personalization approaches in learning environments

Personalization approaches in learning environments can be addressed from different and complementary perspectives. PALE workshop aimed at the following issues: pedagogic conversational agents, responsive open learning environments, and user modeling for all. Hereby, we report the state of the art in this area as gathered in the workshop papers and comment on the future research directions as discussed during the sessions.

A first proposal of Pedagogic Conversational Agents to develop Computational Thinking in children

Pedagogic Conversational Agents are interactive systems that teach by talking to the students. They have been used in several domains to develop competences such as storytelling or negotiation from University to Pre-Primary Education. However, in the literature, no cases of using agents for teaching programming to develop computational thinking in children have been found. In the last decades, there is a growing interest in developing computational thinking in children. According to some authors, if children develop computational thinking, they will be able to solve not only computer problems but their daily life problems in a better way. It is under research which educational technologies and methodologies can be more adequate depending on the context to achieve this goal. In this paper, it is proposed, for the first time, the use of Pedagogic Conversational Agents to develop computational thinking in children. Given the complexity of designing this new type of agent, and as it has been done in previous occasions when trying to design a new agent, the MEDIE methodology will be followed to eventually integrate the agent into the classrooms.

Active learning through collaborative knowledge building using an automatic free-text scoring system in a b-learning environment

ABSTRACT According to active learning, students should be responsible for their own learning. Automatic free-text scoring allows teachers to provide open-ended questions with their correct answers to a computer system, so when students answer the questions, they get immediate feedback (a score, a comment, or both). However, teachers are usually overloaded with many tasks, and they may not have time to create the questions with the correct answers. Therefore, in the 2012/2013 academic year, we asked a group of 124 Pre-Primary and Primary Education students to become the creators of the questions and their correct answers in groups in a free-text scoring system, so the questions use learners’ language, not teachers’ language. From them, 41 students (group of involved students, GIS) fulfilled all the requirements during the course. Our hypothesis was that GIS would be able to increase their academic performance and levels of engagement compared to the rest of the students. The results gathered provide statistic evidence to support that hypothesis. This study pretends to help teachers who want to increase the academic performance and levels of engagement of their students in courses that they may find boring and unrelated to the main topic of their degree, or not directly related with their main academic interests.

Adapting the design and the use methodology of a pedagogical conversational agent of secondary education to childhood education

Pedagogic Conversational Agents are interactive educational systems that talk with students about a particular domain assuming the role of teacher, student or colleague. They have been used with good results from the upper areas of education to Primary Education. However, it has not been found examples in the literature of the field on its use in Early Childhood Education. During 2014/2015 several students in Early Childhood Education collaborated with computer scientists to adapt a Pedagogical Conversational Agent to lower domains of education and provided a first methodology draft of use in the classroom that was tested in a case as reported in this article.