Jean-Yves Antoine

Sibylle, An Assistive Communication System Adapting to the Context and Its User

In this article, we describe the latest version of Sibylle, an AAC system that permits persons who have severe physical disabilities to enter text with any computer application, as well as to compose messages to be read out through speech synthesis. The system consists of a virtual keyboard comprising a set of keypads that allow for the entering of characters or full words by a single-switch selection process. It also includes a sophisticated word prediction component which dynamically calculates the most appropriate words for a given context. This component is auto-adaptive, that is, it learns with every text the user enters. It thus adapts its predictions to the users language and the current topic of communication as well. So far, the system works for French, German and English. Earlier versions of Sibylle have been used since 2001 in a rehabilitation center (Kerpape, France).

SIBYLLE: a system for alternative communication adapting to the context and its user

In this paper, we describe the latest version of SIBYLLE, an AAC system that permits persons suffering from severe physical disabilities to enter text with any computer application and also to compose messages to be read out by a speech synthesis module. The system consists of a virtual keyboard comprising a set of keypads which allow entering characters or full words by a single-switch selection process. It also comprises a sophisticated word prediction component which dynamically calculates the most appropriate words for a given context. This component is auto-adaptive, i.e. it learns on every text the user has entered. It thus adapts its predictions to the users language and the current topic of communication as well. So far the system works for French, German and English. Earlier versions of SIBYLLE have been used since 2001 in the Kerpape rehabilitation center (Brittany, France).

Logical Approach to Natural Language Understanding in a Spoken Dialogue System

We present a logical approach of spoken language understanding for a human-machine dialogue system. The aim of the analysis is to provide a logical formula, or a conceptual graph, by assembling concepts related to a delimited application domain. This flexible structure is gradually built during an incremental parsing, which is meant to combine syntactic and semantic criteria. Then, a contextual understanding step leads to completing this structure. The evaluations of the current system are encouraging. This approach is a preliminary for a logical dialogue that uses the form of the semantic representations.

Effect of dynamic keyboard and word-prediction systems on text input speed in persons with functional tetraplegia.

Information technology plays a very important role in society. People with disabilities are often limited by slow text input speed despite the use of assistive devices. This study aimed to evaluate the effect of a dynamic on-screen keyboard (Custom Virtual Keyboard) and a word-prediction system (Sibylle) on text input speed in participants with functional tetraplegia. Ten participants tested four modes at home (static on-screen keyboard with and without word prediction and dynamic on-screen keyboard with and without word prediction) for 1 mo before choosing one mode and then using it for another month. Initial mean text input speed was around 23 characters per minute with the static keyboard and 12 characters per minute with the dynamic keyboard. The results showed that the dynamic keyboard reduced text input speed by 37% compared with the standard keyboard and that the addition of word prediction had no effect on text input speed. We suggest that current forms of dynamic keyboards and word prediction may not be suitable for increasing text input speed, particularly for subjects who use pointing devices. Future studies should evaluate the optimal ergonomic design of dynamic keyboards and the number and position of words that should be predicted.

Pattern Mining for Named Entity Recognition

Many evaluation campaigns have shown that knowledge-based and data-driven approaches remain equally competitive for Named Entity Recognition. Our research team has developed CasEN, a symbolic system based on finite state transducers, which achieved promising results during the Ester2 French-speaking evaluation campaign. Despite these encouraging results, manually extending the coverage of such a hand-crafted system is a difficult task. In this paper, we present a novel approach based on pattern mining for NER and to supplement our system’s knowledge base. The system, mXS, exhaustively searches for hierarchical sequential patterns, that aim at detecting Named Entity boundaries. We assess their efficiency by using such patterns in a standalone mode and in combination with our existing system.

Deeper Spoken Language Understanding for Man-Machine Dialogue on Broader Application Domains: A Logical Alternative to Concept Spotting

LOGUS is a French-speaking spoken language understanding (SLU) system which carries out a deeper analysis than those achieved by standard concept spotters. It is designed for multi-domain conversational systems or for systems that are working on complex application domains. Based on a logical approach, the system adapts the ideas of incremental robust parsing to the issue of SLU. The paper provides a detailed description of the system as well as results from two evaluation campaigns that concerned all of current French-speaking SLU systems. The observed error rates suggest that our logical approach can stand comparison with concept spotters on restricted application domains, but also that its behaviour is promising for larger domains. The question of the generality of the approach is precisely addressed by our current investigations on a new task: SLU for an emotional robot companion for young hospital patents.

A non-intrusive monitoring system for ambient assisted living service delivery

Automation of smart home for ambient assisted living is currently based on a widespread use of sensors. In this paper, we propose a monitoring system based on the semantic analysis of home automation logs (user requests). Our goal is to replace as many sensors as possible by using advanced tools to infer information usually sensored. To take up this challenge, an ontology, automatically derived from a model-driven process, firstly defines user-system interactions. Then, the use of rules allows an inference engine to deduce user location and intention leading to adapted service delivery.

Emologus: a compositional model of emotion detection based on the propositional content of spoken utterances

The ANR EmotiRob project aims at detecting emotions in an original application context: realizing an emotional companion robot for weakened children. This paper presents a system which aims at characterizing emotions by only considering the linguistic content of utterances. It is based on the assumption of compositionality: simple lexical words have an intrinsic emotional value, while verbal and adjectival predicates act as a function on the emotional values of their arguments. The paper describes the semantic component of the system, the algorithm of compositional computation of the emotion value and the lexical emotional norm used by this algorithm. A quantitative and qualitative analysis of the differences between system outputs and expert annotations is given, which shows satisfactory results, with the right detection of emotional valency in 90% of the test utterances.

Spontaneous speech and natural language processing. ALPES: a robust semantic-led parser

The need for robust parsers is becoming more and more essential as spoken human machine communication is developed. Because of its uncontrolled nature, spontaneous speech presents a high rate of extagrammatical constructions (hesitations, repetitions, self corrections, etc.). As a result, spontaneous speech rapidly catches out most syntactic parsers, in spite of the frequent addition of some corrective methods (S. Seneff, 1992). Therefore, most dialog systems restrict the linguistic analysis of the spoken utterances to a simple extraction of keywords (D. Appelt and E. Jakson, 1992). This selective approach led to significant results in some restricted applications (ATIS), but it does not seem appropriate for higher level tasks, for which the utterances cannot be reduced to a simple set of keywords. As a result, neither the syntactic methods nor the selective approaches can fully satisfy the constraints of robustness and exhaustibility required by the human machine communication. The paper precisely presents a detailed semantic parser (ALPES) which masters most spoken utterances.

Influence of the Number of Predicted Words on Text Input Speed in Participants With Cervical Spinal Cord Injury

OBJECTIVES To determine whether the number of words displayed in the word prediction software (WPS) list affects text input speed (TIS) in people with cervical spinal cord injury (SCI), and whether any influence is dependent on the level of the lesion. DESIGN A cross-sectional trial. SETTING A rehabilitation center. PARTICIPANTS Persons with cervical SCI (N=45). Lesion level was high (C4 and C5, American Spinal Injury Association [ASIA] grade A or B) for 15 participants (high-lesion group) and low (between C6 and C8, ASIA grade A or B) for 30 participants (low-lesion group). INTERVENTION TIS was evaluated during four 10-minute copying tasks: (1) without WPS (Without); (2) with a display of 3 predicted words (3Words); (3) with a display of 6 predicted words (6Words); and (4) with a display of 8 predicted words (8Words). MAIN OUTCOME MEASURES During the 4 copying tasks, TIS was measured objectively (characters per minute, number of errors) and subjectively through subject report (fatigue, perception of speed, cognitive load, satisfaction). RESULTS For participants with low-cervical SCI, TIS without WPS was faster than with WPS, regardless of the number of words displayed (P<.001). For participants with high-cervical SCI, the use of WPS did not influence TIS (P=.99). There was no influence of the number of words displayed in a word prediction list on TIS; however, perception of TIS differed according to lesion level. CONCLUSIONS For persons with low-cervical SCI, a small number of words should be displayed, or WPS should not be used at all. For persons with high-cervical SCI, a larger number of words displayed increases the comfort of use of WPS.