Sarah De Bruyne

Moving object detection in the H.264/AVC compressed domain for video surveillance applications

In this paper a novel method is presented to detect moving objects in H.264/AVC [T. Wiegand, G. Sullivan, G. Bjontegaard, G. Luthra, Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, 13 (7) (2003) 560-576] compressed video surveillance sequences. Related work, within the H.264/AVC compressed domain, analyses the motion vector field to find moving objects. However, motion vectors are created from a coding perspective and additional complexity is needed to clean the noisy field. Hence, an alternative approach is presented here, based on the size (in bits) of the blocks and transform coefficients used within the video stream. The system is restricted to the syntax level and achieves high execution speeds, up to 20 times faster than the related work. To show the good detection results, a detailed comparison with related work is presented for different challenging video sequences. Finally, the influence of different encoder settings is investigated to show the robustness of our system.

A compressed-domain approach for shot boundary detection on H.264/AVC bit streams

The amount of digital video content has grown extensively during recent years, resulting in a rising need for the development of systems for automatic indexing, summarization, and semantic analysis. A prerequisite for video content analysis is the ability to discover the temporal structure of a video sequence. In this paper, a novel shot boundary detection technique is introduced that operates completely in the compressed domain using the H.264/AVC video standard. As this specification contains a number of new coding tools, the characteristics of a compressed bit stream are different from prior video specifications. Furthermore, the H.264/AVC specification introduces new coding structures such as hierarchical coding patterns, which can have a major influence on video analysis algorithms. First, a shot boundary detection algorithm is proposed which can be used to segment H.264/AVC bit streams based on temporal dependencies and spatial dissimilarities. This algorithm is further enhanced to exploit hierarchical coding patterns. As these sequences are characterized by a pyramidal structure, only a subset of frames needs to be considered during analysis, allowing the reduction of the computational complexity. Besides the increased efficiency, experimental results also show that the proposed shot boundary detection algorithm achieves a high accuracy.

Estimating motion reliability to improve moving object detection in the H.264/AVC domain

This paper presents a new algorithm for moving object detection in the H.264/AVC compressed domain which relies on motion vector information. In contrast to other motion vectorbased algorithms, special attention is paid to noisy motion vectors as they highly decrease the performance of these algorithms. We propose to estimate the reliability of motion vectors by comparing them with projected motion vectors from surrounding frames. As such, noisy motion vectors are localized. By combining this information with the magnitude of motion vectors, foreground objects are distinguished. Experimental results demonstrate that our algorithm achieves significantly better segmentation quality compared to other motion vector-based approaches.

Robust spatio-temporal multimodal background subtraction for video surveillance

Background subtraction is a method commonly used to segment objects of interest in image sequences. By comparing new frames to a background model, regions of interest can be found. To cope with highly dynamic and complex environments, a mixture of several models has been proposed in the literature. We propose a novel background subtraction technique derived from the popular mixture of Gaussian models technique (MGM). We discard the Gaussian assumptions and use models existing of an average and an upper and lower threshold. Additionally, we include a maximum difference with the previous value and present an intensity allowance to cope with gradual lighting changes and photon noise, respectively. Moreover, edge-based image segmentation is introduced to improve the results of the proposed technique. This combination of temporal and spatial information results in a robust object detection technique that deals with several difficult situations. Experimental analysis shows that our system is more robust than MGM and more recent techniques, resulting in less false positives and negatives. Finally, a comparison of processing speed shows that our system can process frames up to 50% faster.

Format-independent and metadata-driven media resource adaptation using semantic web technologies

Adaptation of media resources is an emerging field due to the growing amount of multimedia content on the one hand and an increasing diversity in usage environments on the other hand. Furthermore, to deal with a plethora of coding and metadata formats, format-independent adaptation systems are important. In this paper, we present a new format-independent adaptation system. The proposed adaptation system relies on a model that takes into account the structural metadata, semantic metadata, and scalability information of media bitstreams. The model is implemented using the web ontology language. Existing coding formats are mapped to the structural part of the model, while existing metadata standards can be linked to the semantic part of the model. Our new adaptation technique, which is called RDF-driven content adaptation, is based on executing SPARQL Protocol and RDF Query Language queries over instances of the model for media bitstreams. Using different criteria, RDF-driven content adaptation is compared to other adaptation techniques. Next to real-time execution times, RDF-driven content adaptation provides a high abstraction level for the definition of adaptations and allows a seamless integration with existing semantic metadata standards.

Multi-Camera Analysis of Soccer Sequences

The automatic detection of meaningful phases in a soccergame depends on the accurate localization of playersand the ball at each moment. However, the automatic analysisof soccer sequences is a challenging task due to thepresence of fast moving multiple objects. For this purpose,we present a multi-camera analysis system that yields theposition of the ball and players on a common ground plane.The detection in each camera is based on a code-book algorithmand different features are used to classify the detectedblobs. The detection results of each camera are transformedusing homography to a virtual top-view of the playing field.Within this virtual top-view we merge trajectory informationof the different cameras allowing to refine the foundpositions. In this paper, we evaluate the system on a publicSOCCER dataset and end with a discussion of possibleimprovements of the dataset.

Multi-view Object Localization in H.264/AVC Compressed Domain

This paper presents a multi-view homography-based approach for object localization in H.264/AVC compressed video surveillance sequences. The proposed novel, low-complexity method is able to accurately localize moving objects on a ground plane using multiple camera data. Contrary to existing work that exploits motion vectors for object detection and tracking, our compressed domain multi-view object localization solely uses macroblock (MB) partition information. Foreground segmentation is performed on single view compressed video data using MB partition-based temporal differencing. Blob merging, convex hull fitting and noise removal are applied on the resulting foreground views to extract objects. Once relevant objects are found in single views, they are projected onto a ground plane by exploiting the homography constraint. Since projected foreground MB views of multiple cameras will only overlap on points where foreground intersects the ground plane, object locations can be extracted by detecting local maxima on the accumulated ground plane image.

Annotation based personalized adaptation and presentation of videos for mobile applications

Personalized multimedia content which suits user preferences and the usage environment, and as a result improves the user experience, gains more importance. In this paper, we describe an architecture for personalized video adaptation and presentation for mobile applications which is guided by automatically generated annotations. By including this annotation information, more intelligent adaptation techniques can be realized which primarily reduce the quality of unimportant regions in case a bit rate reduction is necessary. Furthermore, a presentation layer is added to enable advanced multimedia viewers to adequately present the interesting parts of a video in case the user wants to zoom in. This architecture is the result of collaborative research done in the EU FP6 IST INTERMEDIA project.

Generic architecture for event detection in broadcast sports video

An increasing amount of digital sports content is generated and made available through broadcast and Internet. To deliver meaningful access for an end-user, summarizations or highlights of the content are necessary. Hence, the automatic extraction of these summarizations is a pre-requisite for efficient content delivery. In this paper, we will present an architecture that allows this automatic annotation of broadcast sports video. Sports video are particularly popular for end-users and have characteristics that can be exploited for automated analysis. However the large variations of such content (e.g., different soccer matches or even different sports) require a system that is generic or easily adaptable. As such, the focus of this paper is on the creation of a generic architecture for automated event detection in sports video. The different aspects of the architecture are explained and the systems is evaluated on different sports sequences.

System architecture for semantic annotation and adaptation in content sharing environments

This paper describes a system architecture, which enables the automatic semantic annotation and adaptation of multimedia content in context-aware content sharing environments. The discussed architecture is the result of research done in the EU FP6 IST INTERMEDIA project. Generating a common vision on user-centric multimedia services in shared content environments to provide users with content personalized to their user preferences and usage environment is one of the objectives of the project. The work presented in this paper describes how media formats with their related metadata are automatically annotated and dynamically adapted. Based on the architecture, a full-featured demonstrator is built.