Samuel Alexander

See Me, Teach Me: Facial Expression and Gesture Recognition for Intelligent Tutoring Systems

Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if intelligent tutoring systems, ATMs and ticketing machines could recognise when users were confused, frustrated or angry they could provide remedial help so improving the service. This paper presents research leading to the development of Easy with Eve, an affective tutoring systems (ATS) for mathematics. The system detects student emotion, adapts to students and displays emotion via a lifelike agent called Eve. Eves is guided by a case-based system which uses data that was generated by an observational study. This paper presents the observational study, the case-based method, and the ATS

Interfaces That Adapt like Humans

Whenever people talk to each other, non-verbal behaviour plays a very important role in regulating their interaction. However, almost all human-computer interactions take place using a keyboard or mouse – computers are completely oblivious to the non-verbal behaviour of their users. This paper outlines the plan for an interface that aims to adapt like a human to the non-verbal behaviour of users. An Intelligent Tutoring System (ITS) for counting and addition is being implemented in conjunction with the New Zealand Numeracy Project. The system’s interface will detect the student’s non-verbal behaviour using in-house image processing software, enabling it to adapt to the student’s non-verbal behaviour in similar ways to a human tutor. We have conducted a video study of how human tutors interpret the non-verbal behaviour of students, which has laid the foundation for this research.

Foundation of an affective tutoring system: learning how human tutors adapt to student emotion

The developing field of Affective Tutoring Systems (ATSs) has created a need to understand how such tutoring systems should adapt to the emotional state of students. To this end, an observational study of human tutors was conducted to learn how human tutors adapt to the affective state of students. This knowledge can then be used to implement the tutoring strategies of an ATS and is hence a critical foundation in its development. This paper presents the methodology and results of this observational study of human tutors and looks ahead to the future development of an animated pedagogical agent capable of detecting, expressing and adapting to emotion.

Affective Tutoring System for Better Learning

Intelligent tutoring systems (ITS) are still not as effective as one-on-one human tutoring. The next generation of intelligent tutors are expected to be able to take into account the emotional state of students. This paper presents research on the development of an Affective Tutoring System (ATS). The system called “Easy with Eve†adapts to students via a lifelike animated agent who is able to detect student emotion through facial expression analysis, and can display emotion herself. Eve’s adaptations are guided by a case-based method for adapting to student states; this method uses data that was generated by an observational study of human tutors. This paper presents an analysis of facial expressions of students engaged in learning with human tutors and how a facial expression recognition system, a life like agent and a case based system based on this analysis have been integrated to develop an ATS for mathematics.

Modeling emotions from non-verbal behaviour in an affective tutoring system

Emotions are an important issue in user modeling. This paper presents a proposal for an Affective Tutoring System (ATS) that can recognise emotions through automated facial expression and gesture analysis, and show emotions through an animated agent. The domain of the system will be addition for 8 to 9 year olds. An observational study of human tutors has been conducted as a basis for developing the tutoring strategies of the ATS.