Florian H. Seitner

Evaluation of data-parallel splitting approaches for H.264 decoding

The high computational demands of the H.264 decoding process pose serious challenges on current processor architectures. A natural way to tackle this problem is the use of multi-core systems. The contribution of this paper lies in a systematic overview and performance evaluation of parallel video decoding approaches. Our study investigates six methods for accomplishing data-parallel splitting in strongly resource-restricted environments inherent to mobile devices. These methods are compared against each other in terms of run-time complexity, core usage, inter-communication and bus transfers. We present benchmark results using different numbers of processor cores. Our results shall aid in finding a splitting strategy that is best suited for the targeted hardware-architecture.

Pedestrian Tracking Based on Colour and Spatial Information

This paper describes a tracking with appearance modelling system for pedestrians. A cascade of boosted classifiers and Haar-like rectangular features [6, 12] are used for the pedestrian detection. Statistical modelling in the HSV colour space is used for adaptive background modelling and subtraction, where the use of circular statistics for hue is proposed. By using the background model in combination with the detector, the system extracts a feature vector based on colour statistics and the spatial information. Circular [9] and linear statistics are applied on the extracted features to robustly track the pedestrians and other moving objects through the scene. An adaptive appearance model copes with partial or full occlusions and addresses the problem of missing or wrong detections in single frames.

A macroblock-level analysis on the dynamic behaviour of an H.264 decoder

This work targets the optimization of multiprocessor H.264 decoder implementations. We have extended the simulator of a multi-core VLIW media processor to enable cycle-accurate function profiling on a sub-macroblock level, which allows measuring the effects of coding modes on the computational complexity with very fine granularity. This knowledge helps the system designer to optimize the system performance and memory sizes to reduce system costs1.

Evaluation of data-parallel H.264 decoding approaches for strongly resource-restricted architectures

Decoding of an H.264 video stream is a computationally demanding multimedia application which poses serious challenges on current processor architectures. For processors with strongly limited computational resources, a natural way to tackle this problem is the use of multi-core systems. The contribution of this paper lies in a systematic overview and performance evaluation of parallel video decoding approaches. We focus on decoder splittings for strongly resource-restricted environments inherent to mobile devices. For the evaluation, we introduce a high-level methodology which can estimate the runtime behaviour of multi-core decoding architectures. We use this methodology to investigate six methods for accomplishing data-parallel splitting of an H.264 decoder. These methods are compared against each other in terms of runtime complexity, core usage, inter-communication and bus transfers. We present benchmark results using different numbers of processor cores. Our results shall aid in finding the splitting strategy that is best-suited for the targeted hardware-architecture.

A high-level simulator for the H.264/AVC decoding process in multi-core systems

ABSTRACT H.264 as a new-generation video coding algorithm is becoming increasingly important for international broadcasting standards such as DVB-H and DMB. In comparison to its predecessors MPEG-2 and MEPG-4 SP/ASP, H.264 achieves improved compression effciency at the cost of increased computational complexity. Real-time execution of the H.264 decoding process poses a large challenge on mobile devices due to low processing capabilities. Multi-core systems provide an elegant and power-effcient solution to overcome this performance limitation. However, effciently distributing the video algorithm among multiple processing units is a non-trivial task. It requires detailed knowledge about the algorithmic complexity, dynamic variations and inter-dependencies between functional blocks. The objective of this paper is an investigation on the dynamic behavior of the H.264 decoding process and on the interaction between the main decoding tasks in the context of multi-core environments. We use an H.264 decoder model to investigate the effciency of a decoding system under various conditions (e.g. different FIFO buffer sizes, bitstreams, coding features and bitrates). The gained insights are finally used to optimize the runtime behavior of a multi-core decoding system and to find a good trade-off between core usage and buffer sizes.

International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW)

Introduction ObesiTIC is a project which aims to investigate innovative information and communication technologies resulting in a new ICT tool specifically designed for children and teenagers, in order to acquire healthy lifestyles, promoting physical activity and avoiding health and social problems associated with obesity and overweight. This is achieved through its co-design and validation with children and teens following a Living Lab approach through SPORTIS Living Lab, a European Network of Living Lab’s effective member. Objectives 1. To develop an innovative solution that would enable healthrelated behaviour changes, increase motivation, promote physical activity and reduce prolonged sedentary time in users, thanks to persuasive and ubiquitous computing techniques. 2. To be validated by SPORTIS Living Lab. Following SPORTIS aim to involve society in the innovation process, ObesiTIC will be validated by end-users (children and teenagers) combined with the development of the application and final product, in order to suit and respect all the needs and aspects of the users’ requirements. Methods A Living Lab methodology is implemented:

Trifocal system for high-quality inter-camera mapping and virtual view synthesis

We present an innovative L-shaped trifocal system involving a high-end main camera with a compact assistant stereo rig attached to it. The system can accurately reconstruct dense disparity information which enables the creation and processing of high-quality 3D film content based on virtual view synthesis. In particular, our system provides compactness and increased flexibility in the spatial camera arrangement, improving upon previous approaches which have focused on linear camera arrays. We provide technical details on our whole workflow, ranging from the hardware design to the key algorithms including rectification, disparity computation and virtual view synthesis based on inter-camera mapping. A subjective evaluation based on a user study demonstrates the effectiveness of our proposed approach in terms of quality of the 3D viewing experience.

Efficient quality enhancement of disparity maps based on alpha matting

We propose an efficient disparity map enhancement method that improves the alignment of disparity edges and color edges even in the presence of mixed pixels and provides alpha values for pixels at disparity edges as a byproduct. In contrast to previous publications, the proposed method addresses mixed pixels at disparity edges and does not introduce mixed disparities that can lead to object deformations in synthesized views. The proposed algorithm computes transparencies by performing alpha matting per disparity-layer. These alpha values indicate the degree of affiliation to a disparity-layer and can hence be used as an indicator for a disparity reassignment that aligns disparity edges with color edges and accounts for mixed pixels. We demonstrate the capabilities of the proposed method on various images and corresponding disparity maps, including images that contain fuzzy object borders (e.g., fur). Furthermore, the proposed method is qualitatively and quantitatively evaluated using disparity ground truth and compared to previously published disparity post-processing methods.

Depth map post-processing for depth-image-based rendering: a user study

We analyse the impact of depth map post-processing techniques on the visual quality of stereo pairs that contain a novel view. To this end, we conduct a user study, in which we address (1) the effects of depth map post processing on the quality of stereo pairs that contain a novel view and (2) the question whether objective quality metrics are suitable for evaluating them. We generate depth maps of six stereo image pairs and apply six different post-processing techniques. The unprocessed and the post-processed depth maps are used to generate novel views. The original left views and the novel views form the stereo pairs that are evaluated in a paired comparison study. The obtained results are compared with the results delivered by the objective quality metrics. We show that post-processing depth maps significantly enhances the perceived quality of stereo pairs that include a novel view. We further observe that the correlation between subjective and objective quality is weak.

Runtime-optimised intra-4×4 mode-decision for H.264/AVC video encoding

We describe a method that considerably improves the computational behaviour of H.264 intra-only encoders. Such intra-only encoders come to use in video-cutting and low-latency video coding where temporal prediction via using inter-frames is no feasible option. We identify the spatial prediction step as the computational bottleneck in intra-only encoders. In this step, the encoder tests various modes that represent predictions of the current macroblocks or sub-macroblocks texture from spatial neighbouring pixels in order to find the mode of lowest residuum. Unfortunately, testing the complete set of allowed modes is computational expensive. However, as is demonstrated by an analysis provided in this paper, it is reasonable to assume that a large percentage of blocks preserve their prediction modes over time. Based on this assumption we develop two algorithms that improve the computation time in the prediction step. These algorithms differ by their criteria used to decide whether a blocks coding mode can be propagated from a temporal preceding frame. Computational speed is enhanced, since we test the full set of modes only for blocks that fail these criteria. Experimental results show that our methods considerably improve the execution time of an intra-only encoder and only show small impact on data-rate and image quality.