Charles Hollemeersch

Motion estimation for H.264/AVC on multiple GPUs using NVIDIA CUDA

To achieve the high coding efficiency the H.264/AVC standard offers, the encoding process quickly becomes computationally demanding. One of the most intensive encoding phases is motion estimation. Even modern CPUs struggle to process high-definition video sequences in real-time. While personal computers are typically equipped with powerful Graphics Processing Units (GPUs) to accelerate graphics operations, these GPUs lie dormant when encoding a video sequence. Furthermore, recent developments show more and more computer configurations come with multiple GPUs. However, no existing GPU-enabled motion estimation architectures target multiple GPUs. In addition, these architectures provide no early-out behavior nor can they enforce a specific processing order. We developed a motion search architecture, capable of executing motion estimation and partitioning for an H.264/AVC sequence entirely on the GPU using the NVIDIA CUDA (Compute Unified Device Architecture) platform. This paper describes our architecture and presents a novel job scheduling system we designed, making it possible to control the GPU in a flexible way. This job scheduling system can enforce real-time demands of the video encoder by prioritizing calculations and providing an early-out mode. Furthermore, the job scheduling system allows the use of multiple GPUs in one computer system and efficient load balancing of the motion search over these GPUs. This paper focuses on the execution speed of the novel job scheduling system on both single and multi-GPU systems. Initial results show that real-time full motion search of 720p high-definition content is possible with a 32 by 32 search window running on a system with four GPUs.

GPU-assisted decoding of video samples represented in the YCoCg-R color space

Although pixel shaders were designed for the creation of programmable rendering effects, they can also be used as generic processing units for vector data. In this paper, attention is paid to an implementation of the YCoCg-R to RGB color space transform, as defined in the H.264/AVC Fidelity Range Extensions, by making use of pixel shaders. Our results show that a significant speedup can be achieved by relying on the processing power of the GPU, relative to the CPU. To be more specific, high definition video (1080p), represented in the YCoCg-R color space, could be decoded to RGB at 30 Hz on a PC with an AMD Athlon XP 2800+ CPU, an AGP bus and an NVIDIA GeForce 6800 graphics card, an effort that could not be realized in real-time by the CPU.

Multiview Video Coding Using Video Game Context Information

Remote rendering of video games for 3DTV becomes a hot topic with the emergence of 3D-enabled mobile devices and cloud-based services. It is however a very challenging task that requires live encoding at very low latency for user interactivity as well as optimal encoding decisions for an acceptable QoE. One key-aspect is that most video games make use of a 3D engine, which is typically accelerated on a GPU, containing information on the composition of the 3D scene and its objects as well as their motion. In this paper, we explore how to extract this information from the GPU and how to exploit it in order to successfully offload the most time consuming tasks of a multiview video encoder. We show that near-optimal encoding decisions can be taken while minimizing the encoder computational complexity as well as the total delay.

Future directions for video fire detection

To accomplish more valuable and more accurate video fire detection, this paper points out future directions and discusses first steps which are now being taken to improve the vision-based detection of smoke and flames. First, an overview is given of the state-of-the-art detection methods in the visible and infrared spectral range. Then, a novel multi-sensor smoke and flame detector is proposed which combines the multimodal information of low-cost visual and thermal infrared detection results. Experiments on fire and nonfire multi-sensor sequences indicate that the combined detector yields more accurate results, with fewer false alarms, than either detector alone. Next, a framework for multi-view fire analysis is discussed to overcome the lack in a video-based fire analysis tool and to detect valuable fire characteristics at the early stage of the fire. As prior experimental results show, this combined analysis from different viewpoints provides more valuable fire characteristics. Information about 3-D fire location, size and growth rate can be extracted from the video data in practically no time. Finally, directions towards standardized evaluation and video-driven fire forecasting are suggested.