Simon Moncrieff

Affect computing in film through sound energy dynamics

We develop an algorithm for the detection and classification of affective sound events underscored by specific patterns of sound energy dynamics. We relate the portrayal of these events to proposed high level affect or emotional coloring of the events. In this paper, four possible characteristic sound energy events are identified that convey well established meanings through their dynamics to portray and deliver certain affect, sentiment related to the horror film genre. Our algorithm is developed with the ultimate aim of automatically structuring sections of films that contain distinct shades of emotion related to horror themes for nonlinear media access and navigation. An average of 82% of the energy events, obtained from the analysis of the audio tracks of sections of four sample films corresponded correctly to the proposed affect. While the discrimination between certain sound energy event types was low, the algotithm correctly detected 71% of the occurrences of the sound energy events within audio tracks of the films analyzed, and thus forms a useful basis for determining affective scenes characteristic of horror in movies.

Dynamic privacy assessment in a smart house environment using multimodal sensing

Surveillance applications in private environments such as smart houses require a privacy management policy if such systems are to be accepted by the occupants of the environment. This is due to the invasive nature of surveillance, and the private nature of the home. In this article, we propose a framework for dynamically altering the privacy policy applied to the monitoring of a smart house based on the situation within the environment. Initially the situation, or context, within the environment is determined; we identify several factors for determining environmental context, and propose methods to quantify the context using audio and binary sensor data. The context is then mapped to an appropriate privacy policy, which is implemented by applying data hiding techniques to control access to data gathered from various information sources. The significance of this work lies in the examination of privacy issues related to assisted-living smart house environments. A single privacy policy in such applications would be either too restrictive for an observer, for example, a carer, or too invasive for the occupants. We address this by proposing a dynamic method, with the aim of decreasing the invasiveness of the technology, while retaining the purpose of the system.

Detecting indexical signs in film audio for scene interpretation

In this paper, we study the sound tracks in films and their indexical semiotic usage by developing a classification system that detects complex sound scenes and their constituent sound events in cinema. We investigate two main issues in this paper: Determination of what constitutes the presence of a high level sound scene and inferences about the thematic content of the scene that can be drawn from this presence, and classification of environmental sounds in the audio track of the scene, to assist in the automatic detection of the high level scene. Experiments with our classification system on pure sounds resulted in a correct event classification rate of 88.9%. When the audio content of a number of film scenes was examined, though a lower accuracy resulted with sound event detection due to the presence of mixed sounds, the film audio samples were generally classified with the correct high-level sound scene label, enabling correct inferences about the story content of the scenes.

Horror film genre typing and scene labeling via audio analysis

We examine localised sound energy patterns, or events, that we associate with high level affect experienced with films. The study of sound energy events in conjunction with their intended affect enable the analysis of film at a higher conceptual level, such as genre. The various affect/emotional responses we investigate in this paper are brought about by well established patterns of sound energy dynamics employed in audio tracks of horror films. This allows the examination of the thematic content of the films in relation to horror elements. We analyse the frequency of sound energy and affect events at a film level as well as at a scene level, and propose measures indicative of the film genre and scene content. Using 4 horror, and 2 non-horror movies as experimental data we establish a correlation between the sound energy event types and horrific thematic content within film, thus enabling an automated mechanism for genre typing and scene content labeling in film.

Dynamic Privacy in Public Surveillance

In implementing privacy protection in surveillance systems, designers must maximize privacy while retaining the systems purpose. One way to achieve this is to combine data-hiding techniques with context-aware policies governing access to securely collected and stored data.

Dynamic Privacy in a Smart House Environment

A smart house can be regarded as a surveillance environment in which the person being observed carries out activities that range from intimate to more public. What can be observed depends on the activity, the person observing (e.g. a carer) and policy. In assisted living smart house environments, a single privacy policy, applied throughout, would be either too invasive for an occupant, or too restrictive for an observer, due to the conflicting goals of surveillance and private environments. Hence, we propose a dynamic method for altering the level of privacy in the environment based on the context, the situation within the environment, encompassing factors relevant to ensuring the occupants safety and privacy. The context is mapped to an appropriate level of privacy, which is implemented by controlling access to data sources (e.g. video) using data hiding techniques. The aim of this work is to decrease the invasiveness of the technology, while retaining the purpose of the system.

Privacy and the access of information in a smart house environment

In this paper we present a framework for addressing privacy issues raised by the monitoring of assisted living smart house environments. In home environments, the conflict between the goals of the surveillance, and the private nature of the home, raises the issue of occupant privacy. This issue needs to be addressed if applications are to be accepted by the occupant. We identify four key properties required for the design of privacy sensitive ubiquitous computing applications. Subsequently, we develop a dynamic and flexible method for implementing privacy measures through controlling access to data, and an interface to provide feedback to the occupant, enabling them to control the implemented privacy measures. We form a generic framework for implementing privacy sensitive ubiquitous computing applications based on previous applications within the field. This framework was then extended and used to develop a specific framework for a privacy sensitive smart house. The approach proposed in the framework dynamically applies privacy measures to multi-modal data according to the situation, or context, of the environment. We further test an implementation of the privacy measures, and detail methods to implement feedback and control. The approach aims to decrease the invasiveness of the surveillance, while retaining the purpose of the assisted living environment.

A Framework for the design of privacy preserving pervasive healthcare

Privacy is an important aspect of pervasive and ubiquitous computing systems, and, in particular, pervasive healthcare. With reference to previous approaches on developing privacy sensitive pervasive healthcare applications, we detail a framework for the design of such systems that aims to minimise the impact of privacy on such systems. In reviewing previous approaches, we extract and combine common elements in order to unify the approaches and create a more formal methodology for designing privacy mechanisms in pervasive healthcare applications. In doing so we also consider the manner in which ubiquitous technologies impact on privacy and methods for reducing this impact. We demonstrate how the framework can be applied by using examples from the previous approaches. In addressing privacy issues, the framework aims to remove a large obstacle to deployment of pervasive healthcare systems, acceptance of the technology.

Online audio background determination for complex audio environments

We present a method for foreground/background separation of audio using a background modelling technique. The technique models the background in an online, unsupervised, and adaptive fashion, and is designed for application to long term surveillance and monitoring problems. The background is determined using a statistical method to model the states of the audio over time. In addition, three methods are used to increase the accuracy of background modelling in complex audio environments. Such environments can cause the failure of the statistical model to accurately capture the background states. An entropy-based approach is used to unify background representations fragmented over multiple states of the statistical model. The approach successfully unifies such background states, resulting in a more robust background model. We adaptively adjust the number of states considered background according to background complexity, resulting in the more accurate classification of background models. Finally, we use an auxiliary model cache to retain potential background states in the system. This prevents the deletion of such states due to a rapid influx of observed states that can occur for highly dynamic sections of the audio signal. The separation algorithm was successfully applied to a number of audio environments representing monitoring applications.

Context aware privacy in visual surveillance

In this paper we present preliminary work implementing dynamic privacy in public surveillance. The aim is to maximise the privacy of those under surveillance, while giving an observer access to sufficient information to perform their duties. As these aspects are in conflict, a dynamic approach to privacy is required to balance the systempsilas purpose with the systempsilas privacy. Dynamic privacy is achieved by accounting for the situation, or context, within the environment. The context is determined by a number of visual features that are combined and then used to determine an appropriate level of privacy.