Charlie Cullen

Investigating ultrasonic positioning on mobile phones

In this paper we evaluate the innate ability of mobile phone speakers to produce ultrasound and the possible uses of this ability for accurate indoor positioning. The frequencies in question are a range between 20 and 22 KHz, which is high enough to be inaudible but low enough to be generated by standard sound hardware. A range of tones is generated at different volume settings on several popular modern mobile phones with the aim of finding points of failure. Our results indicate that it is possible to generate the given range of frequencies without significant distortions, provided the signal volume is not excessively high. This is preceded by the discussion of why such ability on off-the-shelf mobile devices is important for Location Based Services (LBS) applications research. Specifically, this ability could be used for indoor sound trilateration positioning. Such an approach is uniquely characterized by the high accuracy inherent to sound trilateration, with little computational burden on the mobile device, and no specialized hardware or audible noise. Combined with a fast internet connection and other sensors present in modern smartphones, such as accelerometers and magnetometers, our approach confirms mobile phones as a suitable platform for indoor LBS applications.

Using Crowdsourcing for Labelling Emotional Speech Assets

The success of supervised learning approaches for the classification of emotion in speech depends highly on the quality of the training data. The manual annotation of emotion speech assets is the primary way of gathering training data for emotional speech recognition. This position paper proposes the use of crowdsourcing for the rating of emotion speech assets. Recent developments in learning from crowdsourcing offer opportunities to determine accurate ratings for assets which have been annotated by large numbers of non-expert individuals. The challenges involved include identifying good annotators, determining consensus ratings and learning the bias of annotators.

Asynchronous ultrasonic trilateration for indoor positioning of mobile phones

In this paper we discuss how the innate ability of mobile phone speakers to produce ultrasound can be used for accurate indoor positioning. The frequencies in question are in a range between 20 and 22 KHz, which is high enough to be inaudible by humans but still low enough to be generated by todays mobile phone sound hardware. Our tests indicate that it is possible to generate the given range of frequencies without significant distortions, provided the signal volume is not turned excessively high. In this paper we present and evaluate the accuracy of our asynchronous trilateration method (Lok8) for mobile positioning without requiring knowledge of the time the ultrasonic signal was sent. This approach shows that only the differences in time of arrival to multiple microphones (control points) placed throughout the indoor environment is sufficient. Consequently, any timing issues with client and server synchronization are avoided.

Obtaining speech assets for judgement analysis on low-pass filtered emotional speech

Investigating the emotional content in speech from acoustic characteristics requires separating the semantic content from the acoustic channel. For natural emotional speech, a widely used method to separate the two channels is the use of cue masking. Our objective is to investigate the use of cue masking in non-acted emotional speech by analyzing the extent to which filtering impacts the perception of emotional content of the modified speech material. However, obtaining a corpus of emotional speech can be quite difficult whereby verifying the emotional content is an issue thoroughly discussed. Currently, speech research is showing a tendency toward constructing corpora of natural emotion expression. In this paper we outline the procedure used to obtain the corpus containing high audio quality and ‘natural’ emotional speech. We review the use of Mood Induction Procedures which provides a method to obtain spontaneous emotional speech in a controlled environment. Following this, we propose an experiment to investigate the effects of cue masking on natural emotional speech.

CorpVis: An Online Emotional Speech Corpora Visualisation Interface

Our research in emotional speech analysis has led to the construction of several dedicated high quality, online corpora of natural emotional speech assets. The requirements for querying, retrieval and organization of assets based on both their metadata descriptors and their analysis data led to the construction of a suitable interface for data visualization and corpus management. The CorpVis interface is intended to assist collaborative work between several speech research groups working with us in this area, allowing online collaboration and distribution of assets to be performed. This paper details the current CorpVis interface into our corpora, and the work performed to achieve this.

Reusable, Interactive, Multilingual Online Avatars

This paper details a system for delivering reusable, interactive multilingual avatars in online children’s games. The development of these avatars is based on the concept of an intelligent media object that can be repurposed across different productions. The system is both language and character independent, allowing content to be reused in a variety of contexts and locales. In the current implementation, the user is provided with an interactive animated robot character that can be dressed with a range of body parts chosen by the user in real-time. The robot character reacts to each selection of a new part in a different manner, relative to simple narrative constructs that define a number of scripted responses. Once configured, the robot character subsequently appears as a help avatar throughout the rest of the game. At time of writing, the system is currently in beta testing on the My Tiny Planets website to fully assess its effectiveness.

Emotional Speech Corpus Creation, Structure, Distribution and Re-Use

This paper details the on-going creation of a natural emotional speech corpus, its structure, distribution, and re-use. Using Mood Induction Procedures (MIPs), high quality emotional speech assets are obtained, analysed, tagged (for acoustic features), annotated and uploaded to an online speech corpus. This method structures the corpus in a logical and coherent manner, allowing it to be utilized for more than one purpose, ensuring distribution via a URL and ease of access through a web browser. This i s vital to ensuring the reusability of the corpus by third party’s and third party applications.

mobiSurround: An Auditory User Interface for Geo-Service Delivery

This paper describes original research carried out in the area of Location-Based Services (LBS) with an emphasis on Auditory User Interfaces (AUI) for content delivery. Previous work in this area has focused on accurately determining spatial interactions and informing the user mainly by means of the visual modality. mobiSurround is new research that builds upon these principles with a focus on multimodal content delivery and navigation and in particular the development of an AUI. This AUI enables the delivery of rich media content and natural directions using audio. This novel approach provides a hands free method for navigating a space while experiencing rich media content dynamically constructed using techniques such as phrase synthesis, algorithmic music and 3D soundscaping. This paper outlines the innovative ideas employed in the design and development of the AUI that provides an overall immersive user experience.

Judging Emotion from Low-Pass Filtered Naturalistic Emotional Speech

In speech, low frequency regions play a significant role in paralinguistic communication such as the conveyance of emotion or mood. The extent to which lower frequencies signify or contribute to affective speech is still an area for investigation. To investigate paralinguistic cues, and remove interference from linguistic cues, researchers can low-pass filter the speech signal on the assumption that certain acoustic cues characterizing affect are still discernible. Low-pass filtering is a practical technique to investigate paralinguistic phenomena, and is used here to investigate the inference of naturalistic emotional speech. This paper investigates how listeners perceive the level of Activation, and evaluate negative and positive levels, on low-pass filtered naturalistic speech, which has been developed through the use of Mood Inducing Procedures.

Prominence Driven Character Animation

This paper details the development of a fully automated system for character animation implemented in Autodesk Maya. The system uses prioritised speech events to algorithmically generate head, body, arms and leg movements alongside eyeblinks, eyebrow movements and lip-synching. In addition, gaze tracking is also generated automatically relative to the definition of focus objects- contextually important objects in the character’s worldview. The plugin uses an animation profile to store the relevant controllers and movements for a specific character, allowing any character to run with the system. Once a profile has been created, an audio file can be loaded and animated with a single button click. The average time to animate is between 2-3 minutes for 1 minute of speech, and the plugin can be used either as a first pass system for high quality work or as part of a batch animation workflow for larger amounts of content as exemplified in television and online dissemination channels.