Gaëtan Martens

Robust spatio-temporal multimodal background subtraction for video surveillance

Background subtraction is a method commonly used to segment objects of interest in image sequences. By comparing new frames to a background model, regions of interest can be found. To cope with highly dynamic and complex environments, a mixture of several models has been proposed in the literature. We propose a novel background subtraction technique derived from the popular mixture of Gaussian models technique (MGM). We discard the Gaussian assumptions and use models existing of an average and an upper and lower threshold. Additionally, we include a maximum difference with the previous value and present an intensity allowance to cope with gradual lighting changes and photon noise, respectively. Moreover, edge-based image segmentation is introduced to improve the results of the proposed technique. This combination of temporal and spatial information results in a robust object detection technique that deals with several difficult situations. Experimental analysis shows that our system is more robust than MGM and more recent techniques, resulting in less false positives and negatives. Finally, a comparison of processing speed shows that our system can process frames up to 50% faster.

Unsupervised texture segmentation and labeling using biologically inspired features

Due to the semantic gap, describing high-level semantic concepts with low-level visual features is a very challenging task. The classification of textures in scene images is intricate because of the high variation of the data. Therefore, the application of appropriate features is of utter importance. This paper presents biologically inspired features for texture segmentation and an unsupervised method to link those texture features with semantic concepts. The calculation of the features is inspired by the human visual system and corresponds to cell outputs in the first stage of the visual cortex. Analogously to the processing principles of the cortex, self-organizing maps are employed for classification. The performance of the texture segmentation and labeling is evaluated on textures from the Brodatz album and on a real-life scenery image dataset. For both methods, a high percentage of pixels is correctly classified.

Personal content management system: A semantic approach

The amount of multimedia resources that is created and needs to be managed is increasing considerably. Additionally, a significant increase of metadata, either structured (metadata fields of standardized metadata formats) or unstructured (free tagging or annotations) is noticed. This increasing amount of data and metadata, combined with the substantial diversity in terms of used metadata fields and constructs, results in severe problems to manage and retrieve these multimedia resources. Standardized metadata schemes can be used but the plethora of these schemes results in interoperability issues. In this paper, we propose a metadata model suited for personal content management systems. We create a layered metadata service that implements the presented model as an upper layer and combines different metadata schemes in the lower layers. Semantic web technologies are used to define and link formal representations of these schemes. Specifically, we create an ontology for the DIG35 metadata standard and elaborate on how it is used within this metadata service. To illustrate the service, we present a representative use case scenario consisting of the upload, annotation, and retrieval of multimedia content within a personal content management system.

Improved background mixture models for video surveillance applications

Background subtraction is a method commonly used to segment objects of interest in image sequences. By comparing new frames to a background model, regions of interest can be found. To cope with highly dynamic and complex environments, a mixture of several models has been proposed. This paper proposes an update of the popular Mixture of Gaussian Models technique. Experimental analysis shows a lack of this technique to cope with quick illumination changes. A different matching mechanism is proposed to improve the general robustness and a comparison with related work is given. Finally, experimental results are presented to show the gain of the updated technique, according to the standard scheme and the related techniques.

Semantic web technologies for video surveillance metadata

Video surveillance systems are growing in size and complexity. Such systems typically consist of integrated modules of different vendors to cope with the increasing demands on network and storage capacity, intelligent video analytics, picture quality, and enhanced visual interfaces. Within a surveillance system, relevant information (like technical details on the video sequences, or analysis results of the monitored environment) is described using metadata standards. However, different modules typically use different standards, resulting in metadata interoperability problems. In this paper, we introduce the application of Semantic Web Technologies to overcome such problems. We present a semantic, layered metadata model and integrate it within a video surveillance system. Besides dealing with the metadata interoperability problem, the advantages of using Semantic Web Technologies and the inherent rule support are shown. A practical use case scenario is presented to illustrate the benefits of our novel approach.

Noise- and compression-robust biological features for texture classification

Texture classification is an important aspect of many digital image processing applications such as surface inspection, content-based image retrieval, and biomedical image analysis. However, noise and compression artifacts in images cause problems for most texture analysis methods. This paper proposes the use of features based on the human visual system for texture classification using a semisupervised, hierarchical approach. The texture feature consists of responses of cells which are found in the visual cortex of higher primates. Classification experiments on different texture libraries indicate that the proposed features obtain a very high classification near 97%. In contrast to other well-established texture analysis methods, the experiments indicate that the proposed features are more robust to various levels of speckle and Gaussian noise. Furthermore, we show that the classification rate of the textures using the presented biologically inspired features is hardly affected by image compression techniques.

Mixture models based background subtraction for video surveillance applications

Background subtraction is a method commonly used to segment objects of interest in image sequences. By comparing new frames to a background model, regions of interest can be found. To cope with highly dynamic and complex environments, a mixture of several models has been proposed in the literature. This paper proposes a novel background subtraction technique based on the popular Mixture of Gaussian Models technique. Moreover edge-based image segmentation is used to improve the results of the proposed technique. Experimental analysis shows that our system outperforms the standard system both in processing speed and detection accuracy.

Lifting a Metadata Model to the Semantic Multimedia World

This paper describes best-practices in lifting an image metadata standard to the Semantic Web. We provide guidelines on how an XML-based metadata format can be converted into an OWL ontology. Additionally, we discuss how this ontology can be mapped to the W3Cs Media Ontology. This ontology is a standardization effort of the W3C to provide a core vocabulary for multimedia annotations. The approach presented here can be applied to other XML-based metadata standards.

Bridging the Semantic Gap using Human Vision System Inspired Features

In the last decade, digital imaging has experienced a worldwide revolution of growth in both the number of users and the range of applications. The amount of digital image content produced on a daily basis is still increasing drastically. As from the very beginning of photography, those who took pictures tried to capture as much information as possible about the photograph and in todays digital age, the need for appending metadata is even bigger. However, it is obvious that manually annotating images is a cumbersome, time consuming and expensive task for large image databases, and it is often subjective, contextsensitive and incomplete. Furthermore, it is difficult for the traditional text-based methods to support a variety of task-dependent queries solely relying on textual metadata since visual information is a more capable medium of conveying ideas and is more closely related to human perception of the real world. The dynamic image characteristics require sophisticated methodologies for data visualization, indexing and similarity management and, as a result, have attracted significant research efforts in providing tools for contentbased retrieval of visual data. Content-based image retrieval uses the visual contents of an image such as color, shape, texture, and spatial layout to represent and index the image. Early content-based image retrieval systems were based on the search for the best match to a user-provided query image or sketch (Flickner et al., 1995; Mehrotra et al., 1997; Laaksonen et al., 2002). Such systems decompose each image into a number of low-level visual features (e.g., color histograms, edge information) and the retrieval process is formulated as the search for the best match to the feature vector(s) extracted from a query image. However, it was quickly realized that the design of a fully functional retrieval system would require support for semantic queries (Picard, 1995). The basic idea is to automatically associate semantic keywords with each image by building models of visual appearance of the semantic concepts of interest. However, the critical point in the advancement of contentbased image retrieval is the semantic gap. The semantic gap is the major discrepancy in computer vision: the user wants to retrieve images on a semantic level, but the image characterizations can only provide a low-level similarity. As a result, describing high-level semantic concepts with low-level visual features is a challenging task. The first efforts targeted the extraction of specific semantics under the framework of binary classification, such as indoor versus outdoor (Szummer & Picard, 1998), and city versus landscape 16

Intelligent pre-processing for fast-moving object detection

Detection and segmentation of objects of interest in image sequences is the first major processing step in visual surveillance applications. The outcome is used for further processing, such as object tracking, interpretation, and classification of objects and their trajectories. To speed up the algorithms for moving object detection, many applications use techniques such as frame rate reduction. However, temporal consistency is an important feature in the analysis of surveillance video, especially for tracking objects. Another technique is the downscaling of the images before analysis, after which the images are up-sampled to regain the original size. This method, however, increases the effect of false detections. We propose a different pre-processing step in which we use a checkerboard-like mask to decide which pixels to process. For each frame the mask is inverted to avoid that certain pixel positions are never analyzed. In a post-processing step we use spatial interpolation to predict the detection results for the pixels which were not analyzed. To evaluate our system we have combined it with a background subtraction technique based on a mixture of Gaussian models. Results show that the models do not get corrupted by using our mask and we can reduce the processing time with over 45% while achieving similar detection results as the conventional technique.