Sergio Salmeron-Majadas

Emotions Detection from Math Exercises by Combining Several Data Sources

Emotions detection and their management are key issues to provide personalize support in educational scenarios. Literature suggests that combining several input sources can improve the performance of affect recognition. To gain a better understanding of this issue, we carried out a large scale experiment in our laboratory where about 100 participants performed several mathematical exercises while emotional information was gathered from different input sources, including a written emotional report. As a first step, we have explored emotions detection from traditional methods by combining analysis of user behavior when typing this report with sentiment analysis on the text. Moreover, an expert labeled these reports. All these data were used to feed several machine learning algorithms to infer user’s emotions. Preliminary results are not conclusive, but lead some light on how to proceed with the analysis.

A Methodological Approach to Eliciting Affective Educational Recommendations

The emotional situation of the learner can influence the learning process. For this reason, we are researching how educational recommender systems can take advantage of affective computing to improve the recommendation support in educational scenarios. The paper reports works carried out involving 18 educators and 77 learners to elicit and design emotional feedback to be provided for learners in terms of personalized recommendations. To this end, user centered design methods and data mining techniques are used.

An Evaluation of Mouse and Keyboard Interaction Indicators towards Non-intrusive and Low Cost Affective Modeling in an Educational Context☆

Abstract In this paper we propose a series of indicators, which derive from users interactions with mouse and keyboard. The goal is to evaluate their use in identifying affective states and behavior changes in an e-learning platform by means of non-intrusive and low cost methods. The approach we have followed study users interactions regardless of the task being performed and its presentation, aiming at finding a solution applicable in any domain. In particular, mouse movements and clicks, as well as keystrokes were recorded during a math problem solving activity where users involved in the experiment had not only to score their degree of valence (i.e., pleasure versus displeasure) and arousal (i.e., high activation versus low activation) of their affective states after each problem by using the Self-Assessment-Manikin scale, but also type a description of their own feelings. By using that affective labeling, we evaluated the information provided by these different indicators processed from the original users interactions logs. In total, we computed 42 keyboard indicators and 96 mouse indicators.

Filtering of Spontaneous and Low Intensity Emotions in Educational Contexts

Affect detection is a challenging problem, even more in educational contexts, where emotions are spontaneous and usually subtle. In this paper, we propose a two-stage detection approach based on an initial binary discretization followed by a specific emotion prediction stage. The binary classification method uses several distinct sources of information to detect and filter relevant time slots from an affective point of view. An accuracy close to 75% at detecting whether the learner has felt an educationally relevant emotion on 20 second time slots has been obtained. These slots can then be further analyzed by a second classifier, to determine the specific user emotion.

Challenges for inclusive affective detection in educational scenarios

There exist diverse challenges for inclusive emotions detection in educational scenarios. In order to gain some insight about the difficulties and limitations of them, we have analyzed requirements, accommodations and tasks that need to be adapted for an experiment where people with different functional profiles have taken part. Adaptations took into consideration logistics, tasks involved and user interaction techniques. The main aim was to verify to what extent the same approach, measurements and technological infrastructure already used in previous experiments were adequate for inducing emotions elicited from the execution of the experiment tasks. In the paper, we discuss the experiment arrangements needed to cope with people with different functional profiles, which include adaptations on the analysis and results. Such analysis was validated in a pilot experiment with 3 visually impaired participants.

Inclusive Personalized e-Learning Based on Affective Adaptive Support

Emotions and learning are closely related. In the PhD research presented in this paper, that relation has to be taken advantage of. With this aim, within the framework of affective computing, the main goal proposed is modeling learner’s affective state in order to support adaptive features and provide an inclusive personalized e-learning experience. At the first stage of this research, emotion detection is the principal issue to cope with. A multimodal approach has been proposed, so gathering data from diverse sources to feed data mining systems able to supply emotional information is being the current ongoing work. On the next stages, the results of these data mining systems will be used to enhance learner models and based on these, offer a better e-learning experience to improve learner’s results.

Localisation of Pollen Grains in Digitised Real Daily Airborne Samples

Content analysis of pollen grains in the atmosphere is an important task for preventing allergy symptoms, studying crop production or detecting environmental changes. In the last decades, a lot of palynological labs have been created to collect, prepare and analyse airborne samples. Nowadays, this task is done manually with optical microscopes, requires trained experts and is time-consuming. The development of new computer vision systems and the low price of storage systems have improved the solutions towards an automated palynology. Some recognition problems have been solved with better quality images and other with 3D images, but localisation in real airborne samples, with debris, clumped and grouped pollen grains needs to be improved in order to achieve an automatic system useful for biological labs. In this manuscript, we analyse the advances achieved in the last years and explain a new low-cost methodology, that imitates the human expert labour using computational algorithms based on image characteristics and domain knowledge to detect pollen grains. The current results are promising (81.92% of recall and 18.5% of precision) but not enough to develop an automated palynology system.

Towards Multimodal Affective Detection in Educational Systems Through Mining Emotional Data Sources

This paper introduces the work being carried out in an ongoing PhD research focused on the detection of the learners’ affective states by combining different available sources (from physiological sensors to keystroke analysis). Different data mining algorithms and data labeling techniques have been used generating 735 prediction models. Results so far show that predictive models on affective state detection from multimodal-based approaches provide better accuracy rates than single-based.

Supporting Growers with Recommendations in RedVides: Some Human Aspects Involved

This paper discusses some human aspects that are to be considered when designing recommendations for RedVides, a cloud based networking environment that collects the status of the crop with sensors and can take decisions through corresponding actuators. The goal behind is to support growers in decision making processes, which can be benefited from collaborations among growers and with other stakeholders.