Chris Poppe

Enabling context-aware multimedia annotation by a novel generic semantic problem-solving platform

Automatic generation of metadata, facilitating the retrieval of multimedia items, potentially saves large amounts of manual work. However, the high specialization degree of feature extraction algorithms makes them unaware of the context they operate in, which contains valuable and often necessary information. In this paper, we show how Semantic Web technologies can provide a context that algorithms can interact with. We propose a generic problem-solving platform that uses Web services and various knowledge sources to find solutions to complex requests. The platform employs a reasoner-based composition algorithm, generating an execution plan that combines several algorithms as services. It then supervises the execution of this plan, intervening in case of errors or unexpected behavior. We illustrate our approach by a use case in which we annotate the names of people depicted in a photograph.

Silhouette-based multi-sensor smoke detection: coverage analysis of moving object silhouettes in thermal and visual registered images

Fire is one of the leading hazards affecting everyday life around the world. The sooner the fire is detected, the better the chances are for survival. Today’s fire alarm systems, such as video-based smoke detectors, however, still pose many problems. In order to accomplish more accurate video-based smoke detection and to reduce false alarms, this paper proposes a multi-sensor smoke detector which takes advantage of the different kinds of information represented by visual and thermal imaging sensors. The detector analyzes the silhouette coverage of moving objects in visual and long-wave infrared registered (~aligned) images. The registration is performed using a contour mapping algorithm which detects the rotation, scale and translation between moving objects in the multi-spectral images. The geometric parameters found at this stage are then further used to coarsely map the silhouette images and coverage between them is calculated. Since smoke is invisible in long-wave infrared its silhouette will, contrarily to ordinary moving objects, only be detected in visual images. As such, the coverage of thermal and visual silhouettes will start to decrease in case of smoke. Due to the dynamic character of the smoke, the visual silhouette will also show a high degree of disorder. By focusing on both silhouette behaviors, the system is able to accurately detect the smoke. Experiments on smoke and non-smoke multi-sensor sequences indicate that the automated smoke detection algorithm is able to coarsely map the multi-sensor images. Furthermore, using the low-cost silhouette analysis, a fast warning, with a low number of false alarms, can be given.

Semantic web technologies for video surveillance metadata

Video surveillance systems are growing in size and complexity. Such systems typically consist of integrated modules of different vendors to cope with the increasing demands on network and storage capacity, intelligent video analytics, picture quality, and enhanced visual interfaces. Within a surveillance system, relevant information (like technical details on the video sequences, or analysis results of the monitored environment) is described using metadata standards. However, different modules typically use different standards, resulting in metadata interoperability problems. In this paper, we introduce the application of Semantic Web Technologies to overcome such problems. We present a semantic, layered metadata model and integrate it within a video surveillance system. Besides dealing with the metadata interoperability problem, the advantages of using Semantic Web Technologies and the inherent rule support are shown. A practical use case scenario is presented to illustrate the benefits of our novel approach.

Semantic Mastering: content adaptation in the creative drama production workflow

In order to provide audiences with a proper universal multimedia experience, all classes of media consumption devices, from high definition displays to mobile media players, must receive a product that is not only adapted to their capabilities and usage environments, but also conveys the semantics and cinematography behind the narrative in an optimal way. This paper introduces a semantic video adaptation system that incorporates the media adaptation process in the center of the drama production process. Producers, directors and other creative staff instruct the semantic adaptation system using common cinematographic terminology and vocabulary, thereby seamlessly extending the drama production process into the realm of content adaptation. The multitude of production metadata obtained from various steps in the production process provides a valuable context of narrative semantics that is exploited by the adaptation process. As such, high definition imagery can be intelligently adapted to smaller resolutions while optimally fulfilling the filmmaker’s dramatic intentions with respect to the original narrative and obeying various rules of cinematographic grammar.

Lifting a Metadata Model to the Semantic Multimedia World

This paper describes best-practices in lifting an image metadata standard to the Semantic Web. We provide guidelines on how an XML-based metadata format can be converted into an OWL ontology. Additionally, we discuss how this ontology can be mapped to the W3Cs Media Ontology. This ontology is a standardization effort of the W3C to provide a core vocabulary for multimedia annotations. The approach presented here can be applied to other XML-based metadata standards.

Compressed-domain shot boundary detection for H.264/AVC using intra partitioning maps

In this paper, a novel technique for shot boundary detection operating on H.264/AVC-compressed sequences is presented. Due to new and improved coding tools in H.264/AVC, the characteristics of the obtained sequences differ from former video coding standards. Although several algorithms working on this new standard are already proposed, the presence of IDR frames can still lead to a low accuracy for abrupt transitions. To solve this issue, we present the motion-compensated intra partitioning map which relies on the intra partitioning modes and the motion vectors present in the compressed video stream. Experimental results show that this motion-compensated map achieves a high accuracy and exceeds related work.

Automatic Passengers Counting In Public Rail Transport Using Wavelets

Previously, we introduced a passengers counting algorithm in public rail transport. The main disadvantage of that algorithm is it lacks automatic event detection. In this article, we implement two automatic wavelet-based passengers counting algorithms. The new algorithms employ the spatial-domain Laplacian-of-Gaussian-based wavelet, and the frequency-domain applied Non-Linear Difference of Gaussians-based wavelet bandpass video scene filters to extract illumination invariant scene features and to combine them efficiently into the background reference frame. Manual segmentation of the scene into rectangles and tiles for detecting an object as seated is no longer needed as we now apply a boundary box tracker on the segmented moving objects blobs. A scene map is combined with the wavelet-based methods and the boundary box for multi-camera object registration. We have developed a novel holistic geometrical approach for exploiting the scene map and the recorded video sequences from both cameras installed in each train coach to separate the detected objects and locate their positions on the scene map. We test all the algorithms with several video sequences recorded from the both cameras installed in each train coach. We compare the previously developed non-automatic passengers counting algorithm with the two new automatic wavelet-based passengers counting algorithms, and an additional spatial-domain automatic non-wavelet based Simple Mixture of Gaussian Models algorithm.

Immersive video conferencing architecture using game engine technology

This paper introduces the use of gaming technology for the creation of immersive video conferencing systems. The system integrates virtual meeting rooms with avatars and life video feeds, shared across different clients. Video analysis is used to create a sense of immersiveness by introducing aspects of the real world in the virtual environment. This architecture will ease and stimulate the development of immersive and intelligent telepresence systems.