François Portet

From data to text in the Neonatal Intensive Care Unit: Using NLG technology for decision support and information management

Contemporary Neonatal Intensive Care Units collect vast amounts of patient data in various formats, making efficient processing of information by medical professionals difficult. Moreover, different stakeholders in the neonatal scenario, which include parents as well as staff occupying different roles, have different information requirements. This paper describes recent and ongoing work on building systems that automatically generate textual summaries of neonatal data. Our evaluation results show that the technology is viable and comparable in its effectiveness for decision support to existing presentation modalities. We discuss the lessons learned so far, as well as the major challenges involved in extending current technology to deal with a broader range of data types, and to improve the textual output in the form of more coherent summaries.

Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges

One of the greatest challenges in Ambient Assisted Living is to design health smart homes that anticipate the needs of its inhabitant while maintaining their safety and comfort. It is thus essential to ease the interaction with the smart home through systems that naturally react to voice command using microphones rather than tactile interfaces. However, efficient audio analysis in such noisy environment is a challenging task. In this paper, a real-time audio analysis system, the AuditHIS system, devoted to audio analysis in smart home environment is presented. AuditHIS has been tested thought three experiments carried out in a smart home that are detailed. The results show the difficulty of the task and serve as basis to discuss the stakes and the challenges of this promising technology in the domain of AAL.

The sweet-home project: Audio technology in smart homes to improve well-being and reliance

The Sweet-Home project aims at providing audio-based interaction technology that lets the user have full control over their home environment, at detecting distress situations and at easing the social inclusion of the elderly and frail population. This paper presents an overview of the project focusing on the multimodal sound corpus acquisition and labelling and on the investigated techniques for speech and sound recognition. The user study and the recognition performances show the interest of this audio technology.

Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living

This chapter presents the AUDITHIS system which performs real-time sound analysis from eight microphone channels in Health Smart Home associated to the autonomous speech analyzer RAPHAEL. The evaluation of AUDITHIS and RAPHAEL in different settings showed that audio modality is very promising to acquire information that are not available through other classical sensors. Audio processing is also the most natural way for a human to interact with his environment. Thus, this approach particularly fits Health Smart Homes that include home automation (e.g., voice command) or other high level interactions (e.g., dialogue). The originality of the work is also to include sounds of daily living as indicators to distinguish distress from normal situations. First development gave acceptable results for the sound recognition (72% correct classification) and we are working on the reduction of missed-alarm rate to improve performance in the near future. Although the current system suffers a number of limitations and that we raised numerous challenges that need to be addressed, the pair AUDITHIS and RAPHAEL is, to the best of our knowledge, one of the first serious attempts to build a real-time system that consider sound and speech analysis for ambient assisted living. This work also includes several evalu- ations on data acquired from volunteers in a real health smart home condition. Further work will include refinement of the acoustic models to adapt the speech recognition to the aged population as well as connexion to home automation systems.

The importance of narrative and other lessons from an evaluation of an NLG system that summarises clinical data

The BABYTALK BT-45 system generates textual summaries of clinical data about babies in a neonatal intensive care unit. A recent task-based evaluation of the system suggested that these summaries are useful, but not as effective as they could be. In this paper we present a qualitative analysis of problems that the evaluation highlighted in BT-45 texts. Many of these problems are due to the fact that BT-45 does not generate good narrative texts; this is a topic which has not previously received much attention from the NLG research community, but seems to be quite important for creating good data-to-text systems.

Automatic Generation of Textual Summaries from Neonatal Intensive Care Data

Intensive care is becoming increasingly complex. If mistakes are to be avoided, there is a need for the large amount of clinical data to be presented effectively to the medical staff. Although the most common approach is to present the data graphically, it has been shown that textual summarisation can lead to improved decision making. As the first step in the BabyTalk project, a prototype is being developed which will generate a textual summary of 45 minutes of continuous physiological signals and discrete events (e.g.: equipment settings and drug administration). Its architecture brings together techniques from the different areas of signal analysis, medical reasoning, and natural language generation. Although the current system is still being improved, it is powerful enough to generate meaningful texts containing the most relevant information. This prototype will be extended to summarize several hours of data and to include clinical interpretation.

Making Context Aware Decision from Uncertain Information in a Smart Home: A Markov Logic Network Approach

This research addresses the issue of building home automation systems reactive to voice for improved comfort and autonomy at home. The focus of this paper is on the context-aware decision process which uses a dedicated Markov Logic Network approach to benefit from the formal logical representation of domain knowledge as well as the ability to handle uncertain facts inferred from real sensor data. The approach has been experiemented in a real smart home with naive and users with special needs.

P wave detector with PP rhythm tracking: evaluation in different arrhythmia contexts

Automatic detection of atrial activity (P waves) in an electrocardiogram (ECG) is a crucial task to diagnose the presence of arrhythmias. The P wave is difficult to detect and most of the approaches in the literature have been evaluated on normal sinus rhythms and rarely considered arrhythmia contexts other than atrial flutter and fibrillation. A novel knowledge-based P wave detector algorithm is presented. It is self-adaptive to the patient and able to deal with certain arrhythmias by tracking the PP rhythm. The detector has been tested on 12 records of the MIT-BIH arrhythmia database containing several ventricular and supra-ventricular arrhythmias. On the overall records, the detector demonstrates Se = 96.60% and Pr = 95.46%; for the normal sinus rhythm, it reaches Se = 97.76% and Pr = 96.80% and, in the case of Mobitz type II, it demonstrates Se = 72.79% and Pr = 99.51%. It also shows good performance for trigeminy and bigeminy, and outperforms some more sophisticated techniques. Although the results emphasize the difficulty of P wave detection in difficult arrhythmias (supra and ventricular tachycardias), it shows that domain knowledge can efficiently support signal processing techniques.

Evaluation of a Context-Aware Voice Interface for Ambient Assisted Living: Qualitative User Study vs. Quantitative System Evaluation

This article presents an experiment with seniors and people with visual impairment in a voice-controlled smart home using the Sweet-Home system. The experiment shows some weaknesses in automatic speech recognition that must be addressed, as well as the need for better adaptation to the user and the environment. Users were disturbed by the rigid structure of the grammar and were eager to adapt it to their own preferences. Surprisingly, while no humanoid aspect was introduced in the system, the senior participants were inclined to embody the system. Despite these aspects to improve, the system has been favorably assessed as diminishing most participant fears related to the loss of autonomy.

Challenges in the processing of audio channels for Ambient Assisted Living

One of the greatest challenges in Ambient Assisted Living is to design health smart homes that could be able to anticipate the needs of its inhabitant while maintaining their comfort and their safety with an adaptation of the house environment and a facilitation of the connections to the outside world. The most likely to benefit from these smart homes are people in loss of autonomy such as the disabled people or the elderly with cognitive deficiencies. But it becomes essential to ease the interactions with the smart home through dedicated interfaces, in particular, thanks to systems reactive to vocal orders. Audio recognition is also a promising way to ensure more safety by contributing to detection of distress situations. This paper presents the stakes and the challenges of this domain based on some experiments carried out concerning distress call recognition and sound classification at home.