Toygar Akgun

GPU implementation of an anisotropic Huber-L1 dense optical flow algorithm using OpenCL

Optical flow estimation aims at inferring a dense pixel-wise correspondence field between two images or video frames. It is commonly used in video processing and computer vision applications, including motion-compensated frame processing, extracting temporal features, computing stereo disparity, understanding scene context/dynamics and understanding behavior. Dense optical flow estimation is a computationally complex problem. Fortunately, a wide range of optical flow estimation algorithms are embarrassingly parallel and can efficiently be accelerated on GPUs. In this work we discuss a massively multi-threaded GPU implementation of the anisotropic Huber-L1 optical flow estimation algorithm using OpenCL framework, which achieves per frame execution time speed-up factors up to almost 300×. Overall algorithm flow, GPU specific implementation details and performance results are presented.

Using high-level synthesis for rapid design of video processing pipes

In this work, we share our experience in using High-Level Synthesis (HLS) for rapid development of an optical flow design on FPGA. We have performed HLS using Vivado HLS as well as a HLS tool we have developed for the optical flow design at hand and similar video processing problems. The paper first describes the design problem we have and then discusses our own HLS tool. The tool we developed has turned out to be pretty general-purpose except for the ability to handle cyclic inter-iteration dependencies. It also introduces some novel concepts to HLS, such as “pipelined multiplexers”. The synthesis results show that we can achieve better timing or better area results compared to Vivado HLS. Furthermore, the Verilog RTL our HLS tool outputs is much more readable than the one from Vivado HLS. This makes it much easier for the designer to debug and modify the RTL.

Acoustic direction finding in highly reverberant environment with single acoustic vector sensor

We propose a novel wideband acoustic direction finding method for highly reverberant environments using measurements from a single Acoustic Vector Sensor (AVS). Since an AVS is small in size and can be effectively used within the full acoustic frequency bands, the proposed solution is suitable for wideband acoustic source localization. In particular, we introduce a novel approach to extract the signal portions that are not distorted with multipath signals and noise. We do not make any stochastic and sparseness assumptions regarding the underlying signal source. Hence, our approach can be applied to a wide range of wideband acoustic signals. We present experiments with acoustic signals that are specially exposed to long reverberations, where the Signal-to-Noise Ratio is as low as 0 dB. In these experiments, the proposed method reliably estimates the source direction with less than 5 degrees of error even under the introduced significantly high reverberation conditions.

A model-based analysis of extinction ratio effects on phase-OTDR distributed acoustic sensing system performance

Extinction ratio is an inherent limiting factor that has a direct effect on the detection performance of phase-OTDR based distributed acoustics sensing systems. In this work we present a model based analysis of Rayleigh scattering to simulate the effects of extinction ratio on the received signal under varying signal acquisition scenarios and system parameters. These signal acquisition scenarios are constructed to represent typically observed cases such as multiple vibration sources cluttered around the target vibration source to be detected, continuous wave light sources with center frequency drift, varying fiber optic cable lengths and varying ADC bit resolutions. Results show that an insufficient ER can result in high optical noise floor and effectively hide the effects of elaborate system improvement efforts.

FPGA implementation of a dense optical flow algorithm using altera openCL SDK

FPGA acceleration of compute-intensive algorithms is usually not regarded feasible because of the long Verilog or VHDL RTL design efforts they require. Data-parallel algorithms have an alternative platform for acceleration, namely, GPU. Two languages are widely used for GPU programming, CUDA and OpenCL. OpenCL is the choice of many coders due to its portability to most multi-core CPUs and most GPUs. OpenCL SDK for FPGAs and High-Level Synthesis (HLS) in general make FPGA acceleration truly feasible. In data-parallel applications, OpenCL based synthesis is preferred over traditional HLS as it can be seamlessly targeted to both GPUs and FPGAs. This paper shares our experiences in targeting a demanding optical flow algorithm to a high-end FPGA as well as a high-end GPU using OpenCL. We offer throughput and power consumption results on both platforms.

Output Domain Downscaler

This paper offers an area-efficient video downscaler hardware architecture, which we call Output Domain Downscaler (ODD). ODD is demonstrated through an implementation of the bilinear interpolation method combined with Edge Detection and Sharpening Spatial Filter. We compare ODD to a straight-forward implementation of the same combination of methods, which we call Input Domain Downscaler (IDD). IDD tries to output a new pixel of the downscaled video frame every time a new pixel of the original video frame is received. However, every once in a while, there is no downscaled pixel to produce, and hence, IDD stalls. IDD sometimes also skips a complete row of input pixels. ODD, on the other hand, spreads out the job of producing downscaled pixels almost uniformly over a frame. As a result, ODD is able to employ more resource sharing, i.e., can do the same job with fewer arithmetic units, thus offers a more area-efficient solution than IDD. In this paper, we explain how ODD and IDD work and also share their FPGA synthesis results.

Acoustic direction finding under high reverberation

One of the major challanges for acoustic source localization is to eliminate the multipath distortions due to the reverberation. In this paper, we propose a novel direction finding method that is robust to multipath distortions. Since we do not make any stochastic and sparseness assumptions regarding the underlying signal source, our method can be applied to a wide range of wideband acoustic signals. In particular, we introduce a novel approach to extract the signal portions that are not distorted with multipath signals and noise. Hence, the DOA estimation can be performed without being affected from the reverberation. Simulation results show that the proposed method reliably estimates the source direction even under the significantly high reverberation conditions.

Towards Cognitive and Perceptive Video Systems

In this chapter we cover research and development issues related to smart cameras. We discuss challenges, new technologies and algorithms, applications and the evaluation of today’s technologies. We will cover problems related to software, hardware, communication, embedded and distributed systems, multi-modal sensors, privacy and security. We also discuss future trends and market expectations from the customer’s point of view.

Accelerating Super-Resolution Reconstruction Using GPU by CUDA

This paper demonstrates a massively multi-threaded implementation of super-resolution image formation on the NVIDIA CUDA architecture. On the algorithm side maximum a-posteriori (MAP) reconstruction is adopted with sub-pixel translational motion estimation algorithm for spatial resolution enhancement. Resulting algorithm is implemented in CUDA using a low end GT 640 GPU, and an overall speed up of 10 – 11 times is achieved compared to ANSI C implementation running on a Core i5 CPU.

Tools and Techniques for Implementation of Real-time Video Processing Algorithms

This paper describes flexible tools and techniques that can be used to efficiently design/generate quite a variety of hardware IP blocks for highly parameterized real-time video processing algorithms. The tools and techniques discussed in the paper include host software, FPGA interface IP (PCIe, USB 3.0, DRAM), high-level synthesis, RTL generation tools, synthesis automation as well as architectural concepts (e.g., nested pipelining), an architectural estimation tool, and verification methodology. The paper also discusses a specific use case to deploy the mentioned tools and techniques for hardware design of an optical flow algorithm. The paper shows that in a fairly short amount of time, we were able to implement 11 versions of the optical flow algorithm running on 3 different FPGAs (from 2 different vendors), while we generated and synthesized several thousand designs for architectural trade-off.