Nigel Crook

Efficient Salient Region Detection with Soft Image Abstraction

Detecting visually salient regions in images is one of the fundamental problems in computer vision. We propose a novel method to decompose an image into large scale perceptually homogeneous elements for efficient salient region detection, using a soft image abstraction representation. By considering both appearance similarity and spatial distribution of image pixels, the proposed representation abstracts out unnecessary image details, allowing the assignment of comparable saliency values across similar regions, and producing perceptually accurate salient region detection. We evaluate our salient region detection approach on the largest publicly available dataset with pixel accurate annotations. The experimental results show that the proposed method outperforms 18 alternate methods, reducing the mean absolute error by 25.2% compared to the previous best result, while being computationally more efficient.

Using input parameter influences to support the decisions of feedforward neural networks

Abstract Whilst rules extracted from neural networks assist the explanation process, in isolation they do not illustrate the relative importance of each input parameter nor how sensitive the networks output is to these parameters. In this paper we discuss three related measures of input parameter influence which can be used to support explanation facilities for neural networks. An algorithm for generating rules from real-valued networks based on the influence measures is also presented.

ImageSpirit: Verbal Guided Image Parsing

Humans describe images in terms of nouns and adjectives while algorithms operate on images represented as sets of pixels. Bridging this gap between how humans would like to access images versus their typical representation is the goal of image parsing, which involves assigning object and attribute labels to pixels. In this article we propose treating nouns as object labels and adjectives as visual attribute labels. This allows us to formulate the image parsing problem as one of jointly estimating per-pixel object and attribute labels from a set of training images. We propose an efficient (interactive time) solution. Using the extracted labels as handles, our system empowers a user to verbally refine the results. This enables hands-free parsing of an image into pixel-wise object/attribute labels that correspond to human semantics. Verbally selecting objects of interest enables a novel and natural interaction modality that can possibly be used to interact with new generation devices (e.g., smartphones, Google Glass, livingroom devices). We demonstrate our system on a large number of real-world images with varying complexity. To help understand the trade-offs compared to traditional mouse-based interactions, results are reported for both a large-scale quantitative evaluation and a user study.

Interaction strategies for an affective conversational agent

The development of embodied conversational agents (ECA) as companions brings several challenges for both affective and conversational dialogue. These include challenges in generating appropriate affective responses, selecting the overall shape of the dialogue, providing prompt system response times, and handling interruptions. We present an implementation of such a companion showing the development of individual modules that attempt to address these challenges. Further, to resolve resulting conflicts, we present encompassing interaction strategies that attempt to balance the competing requirements along with dialogues from our working prototype to illustrate these interaction strategies in operation. Finally, we provide the results of an evaluation of the companion using an evaluation methodology created for conversational dialogue and including analysis using appropriateness annotation.

Unsupervised Classification of Dialogue Acts using a Dirichlet Process Mixture Model

In recent years Dialogue Acts have become a popular means of modelling the communicative intentions of human and machine utterances in many modern dialogue systems. Many of these systems rely heavily on the availability of dialogue corpora that have been annotated with Dialogue Act labels. The manual annotation of dialogue corpora is both tedious and expensive. Consequently, there is a growing interest in unsupervised systems that are capable of automating the annotation process. This paper investigates the use of a Dirichlet Process Mixture Model as a means of clustering dialogue utterances in an unsupervised manner. These clusters can then be analysed in terms of the possible Dialogue Acts that they might represent. The results presented here are from the application of the Dirichlet Process Mixture Model to the Dihana corpus.

Nonlinear transient computation

A novel transient computation device is presented which performs computations on time-varying input signals. The inputs perturb the device causing transients in its internal dynamics. These transients are characteristic of the inputs and are reflected in the devices output. Previous approaches to transient computation have used large reservoirs of neurons. The proposed device consists of only two neurons with nonlinear internal dynamics. Experimental evidence is given to demonstrate that this device possesses two properties required for performing computations on time-dependent signals: a separation and an approximation property. It is also shown that this device can perform noise resistant pattern recognition.

Self-organised dynamic recognition states for chaotic neural networks

Chaos offers several advantages to the Engineer over other non-chaotic dynamics. One is that chaotic systems are often significantly easier to control than other linear or non-linear systems, requiring only small, appropriately timed perturbations to constrain them within specific unstable periodic orbits (UPOs). Another is that chaotic attractors contain an infinite number of these UPOs. If individual UPOs can be made to represent specific internal states of a system, then a chaotic attractor can be turned into an infinite state machine. In this paper we investigate this possibility with respect to chaotic neural networks. We present a method by which a network can self-select UPOs in response to specific input values. These UPOs correspond to network recognition states for these input values.

Pattern recall in networks of chaotic neurons

This research investigates the potential utility of chaotic dynamics in neural information processing. A novel chaotic spiking neural network model is presented which is composed of non-linear dynamic state (NDS) neurons. The activity of each NDS neuron is driven by a set of non-linear equations coupled with a threshold based spike output mechanism. If time-delayed self-connections are enabled then the network stabilises to a periodic pattern of activation. Previous publications of this work have demonstrated that the chaotic dynamics which drive the network activity ensure that an extremely large number of such periodic patterns can be generated by this network. This paper presents a major extension to this model which enables the network to recall a pattern of activity from a selection of previously stabilised patterns.

Persuasive dialogue based on a narrative theory: an ECA implementation

Embodied Conversational Agents (ECA) are poised to constitute a specific category within persuasive systems, in particular through their ability to support affective dialogue. One possible approach consists in using ECA as virtual coaches or personal assistants and to make persuasion part of a dialogue game implementing specific argumentation or negotiation features. In this paper, we explore an alternative framework, which emerges from the long-term development of ECA as “Companions” supporting free conversation with the user, rather than task-oriented dialogue. Our system aims at influencing user attitudes as part of free conversation, albeit on a limited set of topics. We describe the implementation of a Companion ECA to which the user reports on his working day, and which can assess the user’s emotional attitude towards daily events in the office, trying to influence such attitude using affective strategies derived from a narrative model. This discussion is illustrated through examples from a first fully-implemented prototype.

Scalable Cascade Inference for Semantic Image Segmentation.

Semantic image segmentation is a problem of simultaneous segmentation and recognition of an input image into regions and their associated categorical labels, such as person, car or cow. A popular way to achieve this goal is to assign a label to every pixel in the input image and impose simple structural constraints on the output label space. Efficient approximation algorithms for solving this labelling problem such as α-expansion have, at best, linear runtime complexity with respect to the number of labels, making them practical only when working in a specific domain that has few classes-of-interest. However when working in a more general setting where the number of classes could easily reach tens of thousands, sub-linear complexity is desired. In this paper we propose meeting this requirement by performing cascaded inference that wraps around the α-expansion algorithm. The cascade both divides the large label set into smaller more manageable ones by way of a hierarchy, and dynamically subdivides the image into smaller and smaller regions during inference. We test our method on the SUN09 dataset with 107 accurately hand labelled classes.