Eric J. Pauwels

Visible and infrared image registration in man-made environments employing hybrid visual features

We present a new method to register a pair of images captured in different image modalities. Unlike most of existing systems that register images by aligning single type of visual features, e.g., interest point or contour, we try to align hybrid visual features, including straight lines and interest points. The entire algorithm is carried out in two stages: line-based global transform approximation and point-based local transform adaptation. In the first stage, straight lines derived from edge pixels are employed to find correspondences between two images in order to estimate a global perspective transformation. In the second stage, we divide the entire image into non-overlapping cells with fixed size. The point having the strongest corner response within each cell is selected as the interest point. These points are transformed to other image based on the global transform, and then used to bootstrap a local correspondence search. Experimental evidence shows this method achieves better accuracy for registering visible and long wavelength infrared images/videos as compared to state-of-the-art approaches.

Sensor Networks for Ambient Intelligence

Due to rapid advances in networking and sensing technology we are witnessing a growing interest in sensor networks, in which a variety of sensors are connected to each other and to computational devices capable of multimodal signal processing and data analysis. Such networks are seen to play an increasingly important role as key enablers in emerging pervasive computing technologies. In the first part of this paper we give an overview of recent developments in the area of multimodal sensor networks, paying special attention to ambient intelligence applications. In the second part, we discuss how the time series generated by data streams emanating from the sensors can be mined for temporal patterns, indicating cross-sensor signal correlations.

Fast saliency-aware multi-modality image fusion

This paper proposes a saliency-aware fusion algorithm for integrating infrared (IR) and visible light (ViS) images (or videos) with the aim to enhance the visualization of the latter. Our algorithm involves saliency detection followed by a biased fusion. The goal of the saliency detection is to generate a saliency map for the IR image, highlighting the co-occurrence of high brightness values (hot spots) and motion. Markov Random Fields (MRFs) are used to combine these two sources of information. The subsequent fusion step is employed to bias the end result in favor of the ViS image, except when a region shows clear IR saliency, in which case the IR image gains (local) dominance. By doing so, the fused image succeeds in depicting both the salient foreground object (gleaned from the IR image), against as an easily recognizable background as supplied by the ViS image. An evaluation of the proposed saliency detection method indicates improvements in detection accuracy when compared to state-of-the-art alternatives. Moreover, both objective and subjective assessments reveal the effectiveness of the proposed fusion algorithm in terms of visual context enhancement.

Multimodal identification and localization of users in a smart environment

Detecting the location and identity of users is a first step in creating context-aware applications for technologically-endowed environments. We propose a system that makes use of motion detection, person tracking, face identification, feature-based identification, audio-based localization, and audio-based identification modules, fusing information with particle filters to obtain robust localization and identification. The data streams are processed with the help of the generic client-server middleware SmartFlow, resulting in a flexible architecture that runs across different platforms.

Computer-assisted tree taxonomy by automated image recognition

We present an algorithm that performs image-based queries within the domain of tree taxonomy. As such, it serves as an example relevant to many other potential applications within the field of biodiversity and photo-identification. Unsupervised matching results are produced through a chain of computer vision and image processing techniques, including segmentation and automatic shape matching. The matching itself is based on a nearest neighbours search in an appropriate feature space. Finally, we briefly report on our efforts to set up a webservice to allow the general public to perform such queries online.

T-Patterns Revisited: Mining for Temporal Patterns in Sensor Data

The trend to use large amounts of simple sensors as opposed to a few complex sensors to monitor places and systems creates a need for temporal pattern mining algorithms to work on such data. The methods that try to discover re-usable and interpretable patterns in temporal event data have several shortcomings. We contrast several recent approaches to the problem, and extend the T-Pattern algorithm, which was previously applied for detection of sequential patterns in behavioural sciences. The temporal complexity of the T-pattern approach is prohibitive in the scenarios we consider. We remedy this with a statistical model to obtain a fast and robust algorithm to find patterns in temporal data. We test our algorithm on a recent database collected with passive infrared sensors with millions of events.

Searching for Temporal Patterns in AmI Sensor Data

Anticipation is a key property of human-human communication, and it is highly desirable for ambient environments to have the means of anticipating events to create a feeling of responsiveness and intelligence in the user. In a home or work environment, a great number of low-cost sensors can be deployed to detect simple events: the passing of a person, the usage of an object, the opening of a door. The methods that try to discover re-usable and interpretable patterns in temporal event data have several shortcomings. Using a testbed that we have developed for this purpose, we first contrast current approaches to the problem. We then extend the best of these approaches, the T-Pattern algorithm, with Gaussian Mixture Models, to obtain a fast and robust algorithm to find patterns in temporal data. Our algorithm can be used to anticipate future events, as well as to detect unexpected events as they occur.

Panoramic, adaptive and reconfigurable interface for similarity search

We outline the architecture of a content-based image retrieval (CBIR)-interface that offers the user a graphical tool to create new features by showing (as opposed to telling!) the system what he means. It allows him to interactively classify images by dragging and dropping them into different piles and instructing the interface to come up with features that can mimic this classification. We show how logistic regression and Sammon projection can be used to supervise this search mode.

One class classification for anomaly detection: support vector data description revisited

The Support Vector Data Description (SVDD) has been introduced to address the problem of anomaly (or outlier) detection. It essentially fits the smallest possible sphere around the given data points, allowing some points to be excluded as outliers. Whether or not a point is excluded, is governed by a slack variable. Mathematically, the values for the slack variables are obtained by minimizing a cost function that balances the size of the sphere against the penalty associated with outliers. In this paper we argue that the SVDD slack variables lack a clear geometric meaning, and we therefore re-analyze the cost function to get a better insight into the characteristics of the solution. We also introduce and analyze two new definitions of slack variables and show that one of the proposed methods behaves more robustly with respect to outliers, thus providing tighter bounds compared to SVDD.

Visible and infrared image registration employing line-based geometric analysis

We present a new method to register a pair of visible (ViS) and infrared (IR) images. Unlike most of existing systems that align interest points of two images, we align lines derived from edge pixels, because the interest points extracted from both images are not always identical, but most major edges detected from one image do appear in another image. To solve feature matching problem, we emphasize the geometric structure alignment of features (lines), instead of descriptor-based individual feature matching. This is due to the fact that image properties and patch statistics of corresponding features might be quite different, especially when one compares ViS image with long wave IR images (thermal information). However, the spatial layout of features for both images always preserves consistency. The last step of our algorithm is to compute the image transform matrix, given minimum 4 pairs of line correspondence. The comparative evaluation for algorithms demonstrates higher accuracy attained by our method when compared to the state-of-the-art approaches.