Loredana Cerrato

Presentation quality assessment using acoustic information and hand movements

This study focuses on prosodic and gestural features that contribute to the positive judgement of public oral presentations. The general hypothesis is that certain prosodic characteristics, such as high pitch variation and perceived loudness, together with the production of natural hand gestures, influence the audiences perception of the speaker as a good presenter. Being able to identify features that can give an indication of a good presenter is useful for applications in the field of skills training, where automatic feedback could be provided to trainees at the end of their presentation about the extent to which they have been able to use their voices and gestures to keep the audience engaged. For this reason, we also propose a method, based on prosodic and visual features, able to categorise presentation quality with high accuracy.

Engagement in Dialogue with Social Robots

It is becoming increasingly clear that social and interactive skills are necessary requirements in many application areas where robots interact, communicate and collaborate with humans or other connected devices. The social aspects of human-computer interaction and the connection between humans and robots has recently received considerable attention in the fields of artificial intelligence, cognitive science, healthcare, companion technologies, and industrial/commercial robotics. This article addresses some dimensions of near-term future HRI, with focus on engagement detection for conversational efficiency. We present some findings from HRI research conducted at the Speech Communication Lab at Trinity College Dublin, report our experiences with a publicly exhibited conversational robot, and discuss some future research trends.

Attitude recognition of video bloggers using audio-visual descriptors

In social media, vlogs (video blogs) are a form of unidirectional communication, where the vloggers (video bloggers) convey their messages (opinions, thoughts, etc.) to a potential audience which cannot give them feedback in real time. In this kind of communication, the non-verbal behaviour and personality impression of a video blogger tends to influence viewers attention because non-verbal cues are correlated with the messages conveyed by a vlogger. In this study, we use the acoustic and visual features (body movements that are captured by low-level visual descriptors) to predict the six different attitudes (amusement, enthusiasm, friendliness, frustration, impatience and neutral) annotated in the speech of 10 video bloggers. The automatic detection of attitude can be helpful in a scenario where a machine has to automatically provide feedback to bloggers about their performance in terms of the extent to which they manage to engage the audience by displaying certain attitudes. Attitude recognition models are trained using the random forest classifier. Results show that: 1) acoustic features provide better accuracy than the visual features, 2) while fusion of audio and visual features does not increase overall accuracy, it improves the results for some attitudes and subjects, and 3) densely extracted histograms of flow provide better results than other visual descriptors. A three-class (positive, negative and neutral attitudes) problem has also been defined. Results for this setting show that feature fusion degrades overall classifier accuracy, and the classifiers perform better on the original six-class problem than on the three-class setting.

A Speech-to-Speech, Machine Translation Mediated Map Task: An Exploratory Study

The aim of this study is to investigate how the language technologies of Automatic Speech Recognition ASR, Machine Translation MT, and Text To Speech TTS synthesis affect users during an interlingual interaction. In this paper, we describe the prototype system used for the data collection, we give details of the collected data and report the results of a usability test run to assess how the users of the interlingual system evaluate the interactions in a collaborative map task. We use widely adopted usability evaluation measures: ease of use, effectiveness and users satisfaction, and look at both qualitative and quantitative measures. Results indicate that both users taking part in the dialogues instructions giver and follower found the system similarly satisfactory in terms of ease of learning, ease of use, and pleasantness, even if they were less satisfied with its effectiveness in supporting the task. Users employed different strategies in order to adapt to the shortcomings of the technology, such as hyper-articulation, and rewording of utterances in relation to error of the ASR. We also report the results of a comparison of the map task in two different settings --- one that includes a constant video stream video-on and one that does not no-video. Surprisingly, users rated the no-video setting consistently better.