Vanda Luengo

Fuzzy logic representation for student modelling: case study on geometry

Our aim is to develop a Fuzzy Logic based student model which removes the arbitrary specification of precise numbers and facilitates the modelling at a higher level of abstraction. Fuzzy Logic involves the use of natural language in the form of If-Then statements to demonstrate knowledge of domain experts and hence generates decisions and facilitates human reasoning based on imprecise information coming from the student-computer interaction. Our case study is in geometry. In this paper, we propose a fuzzy logic representation for student modelling and compare it with the Additive Factor Model (AFM) algorithm implemented on DataShop. Two rule-based fuzzy inference systems have been developed that ultimately predict the degree of error a student makes in the next attempt to the problem. Results indicate the rule-based systems achieve levels of accuracy matching that of the AFM algorithm.

Comparing Student Models in Different Formalisms by Predicting Their Impact on Help Success

We describe a method to evaluate how student models affect ITS decision quality – their raison d’etre. Given logs of randomized tutorial decisions and ensuing student performance, we train a classifier to predict tutor decision outcomes (success or failure) based on situation features, such as student and task. We define a decision policy that selects whichever tutor action the trained classifier predicts in the current situation is likeliest to lead to a successful outcome. The ideal but costly way to evaluate such a policy is to implement it in the tutor and collect new data, which may require months of tutor use by hundreds of students. Instead, we use historical data to simulate a policy by extrapolating its effects from the subset of randomized decisions that happened to follow the policy. We then compare policies based on alternative student models by their simulated impact on the success rate of tutorial decisions. We test the method on data logged by Project LISTEN’s Reading Tutor, which chooses randomly which type of help to give on a word. We report the cross-validated accuracy of predictions based on four types of student models, and compare the resulting policies’ expected success and coverage. The method provides a utility-relevant metric to compare student models expressed in different formalisms.

Towards an Authoring Tool to Acquire Knowledge for ITS Teaching Problem Solving Methods

We propose a process of knowledge acquisition and an authoring tool to assist teachers who are not IT specialist to explicit knowledge needed to design ITS teaching solving problems methods. This paper describes our authoring tool and the type of knowledge to acquire.

Towards a Capitalization of Processes Analyzing Learning Interaction Traces

Analyzing data coming from e-learning environments can produce knowledge and potentially improve pedagogical efficiency. Nevertheless, TEL community faces heterogeneity concerning e-learning traces, analysis processes and tools leading these analyses. Therefore, analysis processes have to be redefined when their implementation context changes: they cannot be reused, shared nor easily improved. There is no capitalization and we consider this drawback as an obstacle for the whole community. In this paper, we propose an independent formalism to describe analysis processes of e-learning interaction traces, in order to capitalize them and avoid these technical dependencies. We discuss both this capitalization and its place and effects in the iterative learning analysis procedure.

Assistance in Building Student Models Using Knowledge Representation and Machine Learning

We propose a method and a first authoring tool to assist the design and implementation of diagnostic techniques. This method is independent from the domain and allows building more than one technique at once. The method is based on knowledge representation and a semi-automatic machine learning algorithm. We tested the method in two domains, surgery and reading English. Techniques built with our method beat the majority class in terms of accuracy.

An automatic comparison between knowledge diagnostic techniques

Previous works have pointed out the crucial need for comparison between knowledge diagnostic tools in the field of Intelligent Tutoring Systems (ITS). In this paper, we present an approach to compare knowledge diagnostics. We illustrate our proposition by applying three criteria of comparison for various diagnostic tools in geometry.

Profiling students from their questions in a blended learning environment

Automatic analysis of learners questions can be used to improve their level and help teachers in addressing them. We investigated questions (N=6457) asked before the class by 1st year medicine/pharmacy students on an online platform, used by professors to prepare their on-site Q&A session. Our long-term objectives are to help professors in categorizing those questions, and to provide students with feedback on the quality of their questions. To do so, first we manually categorized students questions, which led to a taxonomy then used for an automatic annotation of the whole corpus. We identified students characteristics from the typology of questions they asked using K-Means algorithm over four courses. The students were clustered by the proportion of each question asked in each dimension of the taxonomy. Then, we characterized the clusters by attributes not used for clustering such as the students grade, the attendance, the number and popularity of questions asked. Two similar clusters always appeared: a cluster (A), made of students with grades lower than average, attending less to classes, asking a low number of questions but which are popular; and a cluster (D), made of students with higher grades, high attendance, asking more questions which are less popular. This work demonstrates the validity and the usefulness of our taxonomy, and shows the relevance of this classification to identify different students profiles.

Capitalisation of analysis processes: enabling reproducibility, openness and adaptability thanks to narration

Analysis processes of learning traces, used to gain important pedagogical insights, are yet to be easily shared and reused. They face what is commonly called a reproducibility crisis. From our observations, we identify two important factors that may be the cause of this crisis: technical constraints due to runnable necessities, and context dependencies. Moreover, the meaning of the reproducibility itself is ambiguous and a source of misunderstanding. In this paper, we present an ontological framework dedicated to taking full advantage of already implemented educational analyses. This framework shifts the actual paradigm of analysis processes by representing them from a narrative point of view, instead of a technical one. This enables a formal description of analysis processes with high-level concepts. We show how this description is performed, and how it can help analysts. The goal is to empower both expert and non-expert analysis stakeholders with the possibility to be involved in the elaboration of analysis processes and their reuse in different contexts, by improving both human and machine understanding of these analyses. This possibility is known as the capitalisation of analysis processes of learning traces.

A Hybrid Architecture for Non-Technical Skills Diagnosis

Our Virtual Learning Environment aims at improving the abilities of experienced technicians to handle critical situations through appropriate use of non-technical skills (NTS), a high-stake matter in many domains as bad mobilization of these skills is the cause of many accidents. To do so, our environment dynamically generates critical situations designed to target these NTS. As the situations need to be adapted to the learner’s skill level, we designed a hybrid architecture able to diagnose NTS. This architecture combines symbolic knowledge about situations, a neural network to drive the learner’s performance evaluation process, and a Bayesian network to model the causality links between situation knowledge and performance to reach NTS diagnosis. A proof of concept is presented in a driving critical situation.

How to Help Teachers Adapt to Learners? Teachers’ Perspective on a Competency and Error-Type Centered Dashboard

The main research goal of this paper is to reveal what information helps teachers adapt to students within a dashboard, how they use it, and how to provide better support. In this research, we observe information acquisition by teachers through a created stand-alone dashboard and interviews. The dashboard presents not only exercise performances and competency level of acquisition, but also error-type information. The analysis of these observations uncovers a first conceptual model of teacher adaptation workflow, as well as additional suggestions to ease adaptation in a future version of such a dashboard.