Abdolhossein Sarrafzadeh

An adaptive real-time skin detector based on Hue thresholding: A comparison on two motion tracking methods

Various applications like face and hand tracking and image retrieval have made skin detection an important area of research. However, currently available algorithms are based on static features of the skin colour, or require a significant amount of computation. On the other hand, skin detection algorithms are not robust to deal with real-world conditions, like background noise, change of intensity and lighting effects. This situation can be improved by using dynamic features of the skin colour in a sequence of images. This article proposes a skin detection algorithm based on adaptive Hue thresholding and its evaluation using two motion detection technique. The skin classifier is based on the Hue histogram of skin pixels, and adapts itself to the colour of the skin of the persons in the video sequence. This algorithm has demonstrated improvement in comparison to the static skin detection method.

Facial expression analysis for estimating learner's emotional state in intelligent tutoring systems

Intelligent tutoring systems (ITS) provide individualized instruction. They offer many advantages over the traditional classroom scenario: they are always available, nonjudgmental and provide tailored feedback resulting in increased and effective learning. However, they are still not as effective as one-on-one human tutoring. The next generation of intelligent tutors is expected to be able to take into account the cognitive and emotional state of students. We present a proposed contribution of affect to student modeling, and reports on the progress made in the development of a facial expression analysis component for intelligent tutoring systems.

Dynamic class imbalance learning for incremental LPSVM

Linear Proximal Support Vector Machines (LPSVMs), like decision trees, classic SVM, etc. are originally not equipped to handle drifting data streams that exhibit high and varying degrees of class imbalance. For online classification of data streams with imbalanced class distribution, we propose a dynamic class imbalance learning (DCIL) approach to incremental LPSVM (IncLPSVM) modeling. In doing so, we simplify a computationally non-renewable weighted LPSVM to several core matrices multiplying two simple weight coefficients. When data addition and/or retirement occurs, the proposed DCIL-IncLPSVM(1) accommodates newly presented class imbalance by a simple matrix and coefficient updating, meanwhile ensures no discriminative information lost throughout the learning process. Experiments on benchmark datasets indicate that the proposed DCIL-IncLPSVM outperforms classic IncSVM and IncLPSVM in terms of F-measure and G-mean metrics. Moreover, our application to online face membership authentication shows that the proposed DCIL-IncLPSVM remains effective in the presence of highly dynamic class imbalance, which usually poses serious problems to previous approaches.

See Me, Teach Me: Facial Expression and Gesture Recognition for Intelligent Tutoring Systems

Many software systems would significantly improve performance if they could adapt to the emotional state of the user, for example if intelligent tutoring systems, ATMs and ticketing machines could recognise when users were confused, frustrated or angry they could provide remedial help so improving the service. This paper presents research leading to the development of Easy with Eve, an affective tutoring systems (ATS) for mathematics. The system detects student emotion, adapts to students and displays emotion via a lifelike agent called Eve. Eves is guided by a case-based system which uses data that was generated by an observational study. This paper presents the observational study, the case-based method, and the ATS

Interfaces That Adapt like Humans

Whenever people talk to each other, non-verbal behaviour plays a very important role in regulating their interaction. However, almost all human-computer interactions take place using a keyboard or mouse – computers are completely oblivious to the non-verbal behaviour of their users. This paper outlines the plan for an interface that aims to adapt like a human to the non-verbal behaviour of users. An Intelligent Tutoring System (ITS) for counting and addition is being implemented in conjunction with the New Zealand Numeracy Project. The system’s interface will detect the student’s non-verbal behaviour using in-house image processing software, enabling it to adapt to the student’s non-verbal behaviour in similar ways to a human tutor. We have conducted a video study of how human tutors interpret the non-verbal behaviour of students, which has laid the foundation for this research.

Face tracking using mean-shift algorithm: a fuzzy approach for boundary detection

Face and hand tracking are important areas of research, related to adaptive human-computer interfaces, and affective computing. In this article we have introduced two new methods for boundary detection of the human face in video sequences: (1) edge density thresholding, and (2) fuzzy edge density. We have analyzed these algorithms based on two main factors: convergence speed and stability against white noise. The results show that “fuzzy edge density” method has an acceptable convergence speed and significant robustness against noise. Based on the results we believe that this method of boundary detection together with the mean-shift and its variants like cam-shift algorithm, can achieve fast and robust tracking of the face in noisy environment, that makes it a good candidate for use with cheap cameras and real-world applications.

Designing games to educate diabetic children

The use of computer games as common vehicles for education, as opposed to pure entertainment, has gained popularity in recent years. Traditional method for diabetes education relies heavily on written materials and there is only a limited amount of resources targeted at educating diabetic children. In this paper, we present a novel approach for designing computer games aimed for educating children with diabetes. Our game design was applied to an existing open source game (Mario Brothers). The results of a pilot study showed that participants enjoyed playing the game and found it valuable for educating diabetic patients.

Referential kNN Regression for Financial Time Series Forecasting

In this paper we propose a new multivariate regression approach for financial time series forecasting based on knowledge shared from referential nearest neighbors. Our approach defines a two-tier architecture. In the top tier, the nearest neighbors that bear referential information for a target time series are identified by exploiting the financial correlation from the historical data. Next, the future status of the target financial time series is inferred from heritage of the time series by using a multivariate k-Nearest-Neighbour (kNN) regression model exploiting the aggregated knowledge from all relevant referential nearest neighbors. The performance of the proposed multivariate kNN approach is assessed by empirical evaluation on the 9-year S&P 500 stock data. The experimental results show that the proposed approach provides enhanced forecasting accuracy than the referred univariate kNN regression.

Towards real-time sign language analysis via markerless gesture tracking

This paper introduces the gesture and hand posture tracking systems for a prototype real-time New Zealand sign language recognition system. The novelty of this work is in the markerless tracking of 13 gestures plus an unknown gesture category. Currently the gesture set is limited, but over time a more extensive gesture library can be developed and trained using the same technique. The hand posture system currently uses markers to obtain the high level of accuracy required for recognition of spelling of words in sign language using finger-pointing. Markerless hand posture detection has been shown to be more challenging especially with different signers who have not been used to train the system.

Foundation of an affective tutoring system: learning how human tutors adapt to student emotion

The developing field of Affective Tutoring Systems (ATSs) has created a need to understand how such tutoring systems should adapt to the emotional state of students. To this end, an observational study of human tutors was conducted to learn how human tutors adapt to the affective state of students. This knowledge can then be used to implement the tutoring strategies of an ATS and is hence a critical foundation in its development. This paper presents the methodology and results of this observational study of human tutors and looks ahead to the future development of an animated pedagogical agent capable of detecting, expressing and adapting to emotion.