Charles-Frederik Hollemeersch

Parallel Deblocking Filtering in MPEG-4 AVC/H.264 on Massively Parallel Architectures

The deblocking filter in the MPEG-4 AVC/H.264 standard is computationally complex because of its high content adaptivity, resulting in a significant number of data dependencies. These data dependencies interfere with parallel filtering of multiple macroblocks (MBs) on massively parallel architectures. In this letter, we introduce a novel MB partitioning scheme for concurrent deblocking in the MPEG-4 AVC/H.264 standard, based on our idea of deblocking filter independency, a corrected version of the limited error propagation effect proposed in the letter. Our proposed scheme enables concurrent MB deblocking of luma samples with limited synchronization effort, independently of slice configuration, and is compliant with the MPEG-4 H.264/AVC standard. We implemented the method on the massively parallel architecture of the graphics processing unit (GPU). Experimental results show that our GPU implementation achieves faster-than real-time deblocking at 1309 frames per second for 1080p video pictures. Both software-based deblocking filters and state-of-the-art GPU-enabled algorithms are outperformed in terms of speed by factors up to 10.2 and 19.5, respectively, for 1080p video pictures.

Silhouette-based multi-sensor smoke detection: coverage analysis of moving object silhouettes in thermal and visual registered images

Fire is one of the leading hazards affecting everyday life around the world. The sooner the fire is detected, the better the chances are for survival. Today’s fire alarm systems, such as video-based smoke detectors, however, still pose many problems. In order to accomplish more accurate video-based smoke detection and to reduce false alarms, this paper proposes a multi-sensor smoke detector which takes advantage of the different kinds of information represented by visual and thermal imaging sensors. The detector analyzes the silhouette coverage of moving objects in visual and long-wave infrared registered (~aligned) images. The registration is performed using a contour mapping algorithm which detects the rotation, scale and translation between moving objects in the multi-spectral images. The geometric parameters found at this stage are then further used to coarsely map the silhouette images and coverage between them is calculated. Since smoke is invisible in long-wave infrared its silhouette will, contrarily to ordinary moving objects, only be detected in visual images. As such, the coverage of thermal and visual silhouettes will start to decrease in case of smoke. Due to the dynamic character of the smoke, the visual silhouette will also show a high degree of disorder. By focusing on both silhouette behaviors, the system is able to accurately detect the smoke. Experiments on smoke and non-smoke multi-sensor sequences indicate that the automated smoke detection algorithm is able to coarsely map the multi-sensor images. Furthermore, using the low-cost silhouette analysis, a fast warning, with a low number of false alarms, can be given.

Hybrid path planning for massive crowd simulation on the GPU

In modern day games, it is often desirable to have many agents navigating intelligently through detailed environments. However, intelligent navigation remains a computationally expensive and complicated problem. In the past, the continuum crowds algorithm demonstrated the value of using a dynamic potential field to guide many agents to a common goal location. However this algorithm is prohibitively resource intensive for real time applications using large and detailed virtual worlds. In this paper, we propose a novel hybrid system that first uses a coarse A* path finding step. This helps to eliminate unnecessary work during the potential field generation by excluding areas of the world from the potential field calculation. Additionally, we show how an optimized potential field solver can be implemented on the GPU using the concepts of persistent threads and inter-block communication. Results show that our system achieves considerable speedups compared to existing path planning systems and that up to 100,000 agents can be simulated and rendered in real time on a mainstream GPU.

Ultra High Definition video decoding with Motion JPEG XR using the GPU

Many applications require real-time decoding of high-resolution video pictures, for example, quick editing of video sequences in video editing applications. To increase decoding speed, parallelism can be exploited, yet, block-based image and video coding standards are difficult to decode in parallel because of the high number of dependencies between blocks. This paper investigates the parallel decoding capabilities of the new JPEG XR image coding standard for use on the massively-parallel architecture of the GPU. The potential of parallelism of the hierarchical frequency coding scheme used in the standard is addressed and a parallel decoding scheme is described suitable for real-time decoding of Ultra High Definition (4320p) Motion JPEG XR video sequences. Our results show a decoding speed of up to 46 frames per second for Ultra High Definition (4320p) sequences with high-dynamic range (32-bit/4:2:0) luma and chroma components.

A new approach to combine texture compression and filtering

Texture mapping has been widely used to improve the quality of 3D rendered images. To reduce the storage and bandwidth impact of texture mapping, compression systems are commonly used. To further increase the quality of the rendered images, texture filtering is also often adopted. These two techniques are generally considered to be independent. First, a decompression step is executed to gather texture samples, which is then followed by a separate filtering step. We have investigated a system based on linear transforms that merges both phases together. This allows more efficient decompression and filtering at higher compression ratios. This paper formally presents our approach for any linear transformation, how the commonly used discrete cosine transform can be adapted to this new approach, and how this method can be implemented in real time on current-generation graphics cards using shaders. Through reuse of the existing hardware filtering, fast magnification and minification filtering is achieved. Our implementation provides fully anisotropically filtered samples four to six times faster than an implementation using two separate phases for decompression and filtering. Additionally, our transform-based compression also provides increased and variable compression ratios over standard hardware compression systems at a comparable or better quality level.

Multi-modal time-of-flight based fire detection

This paper proposes two novel time-of-flight based fire detection methods for indoor and outdoor fire detection. The indoor detector is based on the depth and amplitude image of a time-of-flight camera. Using this multi-modal information, flames can be detected very accurately by fast changing depth and amplitude disorder detection. In order to detect the fast changing depth, depth differences between consecutive frames are accumulated over time. Regions which have multiple pixels with a high accumulated depth difference are labeled as candidate flame regions. Simultaneously, the amplitude disorder is also investigated. Regions with high accumulative amplitude differences and high values in all detail images of the amplitude image its discrete wavelet transform, are also labeled as candidate flame regions. Finally, if one of the depth and amplitude candidate flame regions overlap, fire alarm is given. The outdoor detector, on the other hand, only differs from the indoor detector in one of its multi-modal inputs. As depth maps are unreliable in outdoor environments, the outdoor detector uses a visual flame detector instead of the fast changing depth detection. Experiments show that the proposed detectors have an average flame detection rate of 94% with no false positive detections.

Real-time visualizations of gigapixel texture data sets using HTML5

With the recent standardization of WebGL as part of HTML5, new possibilities have arisen for graphically intensive web-based applications. This paper presents our gigapixel texture visualization system which runs entirely within the limitations of a standards-compatible browser. Compared to existing approaches, our system offers high- performance 3D texture visualization and streaming without any dedicated plugins. We show that real-time performance can be achieved (less than 12ms render time per frame) on current-generation desktop hardware for texture data sets of at least 15 gigapixels.

Data-parallel intra decoding for block-based image and video coding on massively parallel architectures

With the increasing number of processor cores available in modern computing architectures, task or data parallelism is required to maximally exploit the available hardware and achieve optimal processing speed. Current state-of-the-art data-parallel processing methods for decoding image and video bitstreams are limited in parallelism by dependencies introduced by the coding tools and the number of synchronization points introduced by these dependencies, only allowing task or coarse-grain data parallelism. In particular, entropy decoding and data prediction are bottleneck coding tools for parallel image and video decoding. We propose a new data-parallel processing scheme for block-based intra sample and coefficient prediction that allows fine-grain parallelism and is suitable for integration in current and future state-of-the-art image and video codecs. Our prediction scheme enables maximum concurrency, independent of slice or tile configuration, while minimizing synchronization points. This paper describes our data-parallel processing scheme for one- and two-dimensional prediction and investigates its application to block-based image and video codecs using JPEG XR and H.264/AVC Intra as a starting point. We show how our scheme enables faster decoding than the state-of-the-art wavefront method with speedup factors of up to 21.5 and 7.9 for JPEG XR and H.264/AVC Intra coding tools respectively. Using the H.264/AVC Intra coding tool, we discuss the requirements of the algorithm and the impact on decoded image quality when these requirements are not met. Finally, we discuss the impact on coding rate in order to allow for optimal parallel intra decoding.

Compressed-domain shot boundary detection for H.264/AVC using intra partitioning maps

In this paper, a novel technique for shot boundary detection operating on H.264/AVC-compressed sequences is presented. Due to new and improved coding tools in H.264/AVC, the characteristics of the obtained sequences differ from former video coding standards. Although several algorithms working on this new standard are already proposed, the presence of IDR frames can still lead to a low accuracy for abrupt transitions. To solve this issue, we present the motion-compensated intra partitioning map which relies on the intra partitioning modes and the motion vectors present in the compressed video stream. Experimental results show that this motion-compensated map achieves a high accuracy and exceeds related work.

Graphics for Serious Games: Infinitex: An interactive editing system for the production of large texture data sets

Recent advancements in graphics hardware have made the use of texture streaming methods feasible for real-time applications. Using these methods, not only texture resolution and detail can be increased up to gigapixel resolution, but when used together with well authored textures these techniques can offer dramatically improved visual quality. However, systems aiding in texture data production itself have received a lot less attention than the streaming and rendering problem. When using current production methods in a texture streaming environment, these methods tend to break down and reduce artist efficiency to the point where the technology is no longer used to its full potential. In this paper we describe the details behind our Infinitex system. Infinitex is a texture creation and editing system that allows the users, i.e. artists, to produce large texture data in an intuitive and interactive way. Our system goes beyond a simple editor, as it incorporates the whole production process from the initial empty environment until the final finished product and addresses all the challenges that arise along the way when producing gigabytes of texture data. In particular, we will focus on versioning, management, continuity, and security. We show how our system, through the use of just-in-time tile generation, offers interactive editing and management operations while meeting all the other constraints imposed on the system.