Manuel Bouillon

User and System Cross-Learning of Gesture Commands on Pen-Based Devices

This paper presents a new design and evaluation of customizable gesture commands on pen-based devices. Our objective is to help users during the definition of gestures by detecting confusion among gestures. We also help the memorization gestures with the guide of a new type of menu “Customizable Gesture Menus”. These menus are associated with an evolving gesture recognition engine that learns incrementally, starting from few data samples. Our research focuses on making user and recognition system learn at the same time, hence the term “cross-learning”. Three experimentations are presented in details in this paper to support these ideas.

Multi-criteria handwriting quality analysis with online fuzzy models

IntuiScript is an innovative project aiming at the development of a digital workbook providing feedback during the handwriting learning process for children from three to seven years old. In this context, the paper presents a method to analyse handwriting quality that responds to the expectations of the IntuiScript educational scenario: on-line and real time feedback for children, an automatic detection of children mistakes guiding the pedagogical progression, and a precise analysis of children writing saved to help teacher to understand children writing skills. The presented method introduces a multi-criteria architecture to analyse handwriting quality based on three different aspects: shape, order and direction. The validation of the proposed approach is done on a realistic dataset collected in preschools and primary schools with 952 children. Results show a positive feedback of children and teachers about the use of tactile digital devices, and a significant improvement of the performances of the multi-criteria architecture compared to the previous analyser. The ground truth has been annotated by experts with different levels of confidence. Specific evaluation metrics are introduced to deal with confidence annotations.

Decremental learning of evolving fuzzy inference systems: application to handwritten gesture recognition

This paper tackles the problem of incremental and decremental learning of an evolving and customizable fuzzy inference system for classification. We explain the interest of integrating a forgetting capacity in such an evolving system to improve its performances in changing environments. In this paper, we describe two decremental learning strategies to introduce a forgetting capacity in evolving fuzzy inference systems. Both techniques use a sliding window to introduce forgetting in the optimization process of fuzzy rules conclusions. The first approach is based on a downdating technique of least squares solutions for unlearning old data. The second integrates differed directional forgetting in the covariance matrices used in the recursive least square algorithm. These techniques are first evaluated on handwritten gesture recognition tasks in changing environments. They are also evaluated on some well-known classification benchmarks. In particular, it is shown that decremental learning allow to adapt to concept drifts. It is also demonstrated that decremental learning is necessary to maintain the system capacity of learning new classes over time, making decremental learning essential for the life-time use of an evolving and customizable classification system.

Using Confusion Reject to Improve (User and) System (Cross) Learning of Gesture Commands

This paper presents a new method to help users defining personalized gesture commands (on pen-based devices) that maximize recognition performance from the classifier. The use of gesture commands give rise to a cross-learning situation where the user has to learn and memorize the command gestures and the classifier has to learn and recognize drawn gestures. The classification task associated with the use of customized gesture commands is complex because the classifier only has very few samples per class to start learning from. We thus need an evolving recognition system that can start from scratch or very few data samples and that will learn incrementally to achieve good performance after some using time. Our objective is to make the user aware of the recognizer difficulties during the definition of commands, by detecting confusion among gesture classes, in order to help him define a gesture set that yield good recognition performance from the beginning. To detect confusing classes we apply confusion reject principles to our evolving recognizer, which is based on a first order fuzzy inference system. A realistic experiment has been made on 55 persons to validate our confusion detection technique, and it shows that our method leads to a significant improvement of the classifier recognition performance.

User Interaction Optimization for an Evolving Classifier of Handwritten Gesture Commands

Touch sensitive interface enables new interaction methods, like using gesture commands. The use of gesture commands give rise to a cross-learning situation where the user has to learn and memorize the command gestures and the classifier has to learn and recognize drawn gestures. To easily memorize more than a dozen of gesture commands, it is important to be able to customize them. The classification task associated with the use of customized gesture commands is complex because the classifier only has very few samples per class to start learning from. We thus need an evolving recognition system that can start from very few data samples and that will learn incrementally to achieve good performance after some using time. This article presents the impact of using rejection based user interactions to supervise the on-line training of the evolving classifier. The objective is to obtain a gesture command system that cooperates as best as possible with the user: to learn from its mistakes without soliciting him too often. To detect confusing classes we apply confusion reject principles to our evolving recognizer, which is based on a first order fuzzy inference system. A significant user experiment has been performed on 63 persons that validates our approach. This user experiment shows the interest of optimizing user interactions by taking into account the confusion detection capability of our recognition system.

Supervision Strategies for the Online Learning of an Evolving Classifier for Gesture Commands

Touch sensitive interfaces enable new interaction methods like using gesture commands. To easily memorize more than a dozen of gesture commands, it is important to be able to customize them. The classifier used to recognize drawn symbols must hence be customisable, able to learn from very few data, and evolving, able to learn and improve during its use. This work studies different supervision strategies for the online training of the evolving classifier. We compare six supervision strategies, depending on user interaction (solicitation by the system), and self-evaluation capacities (notion of reject). In particular, there is a trade-off between the number of user interactions, to supervise the online training, and the error rate of the classifier. We show in this paper that the strategy giving the best results is to learn from data validated by the user, when the confidence of the recognition is too low, and from data implicitly validated.

Decremental Learning of Evolving Fuzzy Inference Systems Using a Sliding Window

This paper tackles the problem of decremental learning of an evolving classification system. We study the use of decremental learning to improve performance of evolving recognizers in non-stationary scenarios. Our on-line recognizer is based on an evolving fuzzy inference system. In this paper, we propose a new strategy to introduce decremental learning, with the use of a sliding window, in the optimization of fuzzy rules conclusions. This approach is based on a downdating technique of least squares solutions for unlearning old data. This technique is evaluated on handwritten gesture recognition tasks. In particular, it is shown that this downdating techniques allow to adapt to concept drifts and that we face a precision reactiveness trade-off. It is also demonstrated that decremental learning is necessary to maintain the system learning capacity over time, making decremental learning essential for the life-time use of an evolving classification system.