Honggang Qi

A Hardware-Efficient Multi-Resolution Block Matching Algorithm and its VLSI Architecture for High Definition MPEG-Like Video Encoders

High throughput, heavy bandwidth requirement, huge on-chip memory consumption, and complex data flow control are major challenges in high definition integer motion estimation hardware implementation. This paper proposes an efficient very large scale integration architecture for integer multi-resolution motion estimation based on optimized algorithm. There are three major contributions in this paper. First, this paper proposes a hardware friendly multi-resolution motion estimation algorithm well-suited for high definition video encoder. Second, parallel processing element (PE) array structure is proposed to implement three-level hierarchical motion estimation, only 256PEs are enough for one reference frame real-time high definition motion estimation by efficient PE reuse. Third, efficient on-chip reference pixel buffer sharing mechanism between integer and fractional motion estimation is proposed with almost 50% SRAM saving and memory bandwidth reduction. The proposed multi-resolution motion estimation algorithm reached a good balance between complexity and performance with rate distortion optimized variable block size motion estimation support. Also, we have achieved moderate logic circuit and on-chip SRAM consumption. The proposed architecture is well-suited for all MPEG-like video coding standards such as H.264, audio video coding standard, and VC-1.

A Low-Cost Very Large Scale Integration Architecture for Multistandard Inverse Transform

In this brief, a low-cost very large scale integration (VLSI) architecture is designed for multistandard inverse transform. The proposed architecture is used in multistandard decoder of MPEG-2, MPEG-4 ASP, H.264/AVC and VC-1. Two circuit share strategies, factor share (FS) and adder share (AS), are applied to the inverse transform architecture for saving its circuit resource. It is shown that the proposed multistandard inverse transform architecture can support the real-time decoding of 1920 × 1080@60 Hz high-definition video at the cost of low circuit resource.

Online Deformable Object Tracking Based on Structure-Aware Hyper-Graph

Recent advances in online visual tracking focus on designing part-based model to handle the deformation and occlusion challenges. However, previous methods usually consider only the pairwise structural dependences of target parts in two consecutive frames rather than the higher order constraints in multiple frames, making them less effective in handling large deformation and occlusion challenges. This paper describes a new and efficient method for online deformable object tracking. Different from most existing methods, this paper exploits higher order structural dependences of different parts of the tracking target in multiple consecutive frames. We construct a structure-aware hyper-graph to capture such higher order dependences, and solve the tracking problem by searching dense subgraphs on it. Furthermore, we also describe a new evaluating data set for online deformable object tracking (the Deform-SOT data set), which includes 50 challenging sequences with full annotations that represent realistic tracking challenges, such as large deformations and severe occlusions. The experimental result of the proposed method shows considerable improvement in performance over the state-of-the-art tracking methods.

Geometric Hypergraph Learning for Visual Tracking

Graph-based representation is widely used in visual tracking field by finding correct correspondences between target parts in different frames. However, most graph-based trackers consider pairwise geometric relations between local parts. They do not make full use of the target’s intrinsic structure, thereby making the representation easily disturbed by errors in pairwise affinities when large deformation or occlusion occurs. In this paper, we propose a geometric hypergraph learning-based tracking method, which fully exploits high-order geometric relations among multiple correspondences of parts in different frames. Then visual tracking is formulated as the mode-seeking problem on the hypergraph in which vertices represent correspondence hypotheses and hyperedges describe high-order geometric relations among correspondences. Besides, a confidence-aware sampling method is developed to select representative vertices and hyperedges to construct the geometric hypergraph for more robustness and scalability. The experiments are carried out on three challenging datasets (VOT2014, OTB100, and Deform-SOT) to demonstrate that our method performs favorably against other existing trackers.

Parallelizing video transcoding with load balancing on cloud computing

Cloud computing is emerging as a very promising technology for computing and storage services. However, the multi-resources load balancing over heterogeneous cluster or cloud is a NP-hard problem. To obtain an optimized solution, in this paper, we propose a heuristic algorithm named Minimum Longest Queue Finish Time (MLFT). In the proposed scheme, we first divide the high computation task into multiple sub-tasks, and re-organize all the tasks into multiple task queues to shorten the entire finish time of all the tasks submitted to the cluster and launched in parallel according to load balancing. In the task division process, an adaptive segmentation algorithm is proposed according to the complexity and maximum segmentation granularity of the input task. Based on the proposed algorithm, an efficient parallel video transcoding framework with cloud computing is presented. Experimental results show that the proposed algorithm outperforms the existing algorithms significantly on the entire finish time of the tasks and approaches to the optimal solution closely.

Abnormal event detection in crowded scenes based on Structural Multi-scale Motion Interrelated Patterns

Detecting abnormal events in crowded scenes remains challenging due to the diversity of events defined by various applications. Among the many application situations, motion analysis for event representation is suited for crowded scenes. In this paper, we propose a novel abnormal event detection method via likelihood estimation of dynamic-texture motion representation, called Structural Multi-scale Motion Interrelated Patterns (SMMIP). SMMIP combines both original motion patterns and their structural spatio-temporal information, which effectively represents localized events by different resolutions of motion patterns. To model normal events, the Gaussian mixture model is trained with the observed normal events, then the likelihood estimation for testing events is computed to judge whether they are abnormal. Meanwhile, the proposed model can be learned online by updating the parameters incrementally. The proposed approach is evaluated on several publicly available datasets and outperforms several other methods proposed before, which is shown that the structural spatio-temporal information added in motion representation helps increasing the anomalies detection rate.

Efficient macroblock pipeline structure in high definition AVS video encoder VLSI architecture

In traditional four-stage pipeline structures for H.264 video encoder hardware implementation, rate distortion optimization (RDO) based mode decision was turned off, and dual-port or ping-pang on-chip search window SRAM was used to achieve data reuse between the integer and fractional pixel motion estimation. To support RDO based mode decision for efficient high definition AVS video coding implementation, we propose an improved four-stage MB pipeline structure. Also on-chip buffer structure is optimized to achieve the balance between circuit consumption and coding performance. The Jizhun profile AVS video encoder is successfully mapped into hardware implementation with the proposed pipeline structure with small performance degradation.

Reconfigurable video coding framework and decoder reconfiguration instantiation of AVS

In 2004, a new standardization activity called reconfigurable video coding (RVC) was started by MPEG with the purpose of offering a framework which provides reconfiguration capabilities for standard video coding technology. The essential idea of RVC framework is a dynamic dataflow mechanism of constructing new video codecs by a collection of video coding tools from video tool libraries. With this objective, RVC framework is not restricted to specific coding standard, but defined at coding tools level with interoperability to achieve high flexibility and reusability. Three elements are normative in RVC framework: decoder description (DD), video tool library (VTL) and abstract decoder model (ADM). With these elements, a standard or new decoder is able to be reconfigured in RVC framework. This paper presents the procedure of describing a reconfigured decoder in DD, reusing and exchanging tools from VTLs and initializing ADM in the dataflow formalism of RVC framework. A decoder configuration which can be instantiated as AVS intra decoder configuration or other new decoder configurations in RVC framework is described as an example by using coding tools from China audio video coding standard (AVS) and MPEG series. It is shown that the process mechanism offered by RVC framework is versatile and flexible to achieve high reusability and exchangeability in decoder configurations.

Multiple layer parallel motion estimation on GPU for High Efficiency Video Coding (HEVC)

This paper provides a multiple-layer parallel motion estimation (ME) scheme implemented on GPU for High Efficiency Video Coding (HEVC). The scheme is hierarchically structured, including four layers: coding tree unit (CTU), prediction unit (PU), motion vector (MV) selection and instruction optimization. In PU-layer, costs of various PU sizes were obtained through a SAD (sum of absolute differences) look-up table instead of progressive cost merging. And during MV selection, GPUs comparison instruction was used to avoid branches. At the same time, concurrent CTUs processing and SIMD (Single Instruction, Multiple Data) optimization also improve the performance significantly. Experimental results show that the proposed scheme can take full advantage of GPU and achieves over 90 times speedup compared with the HM10.0 using fast ME.

VLSI friendly me search window buffer structure optimization and algorithm verification for high definition H.264/AVS video encoder

High external memory bandwidth requirement is one major challenge for efficient hardware motion estimation (ME) implementation. Large double-buffered on-chip search window (SW) buffer is usually employed to increase the throughput. In this paper, we focus on SW buffer structure optimization in a systematic viewpoint. An efficient buffer share mechanism is proposed to minimize the memory consumption, simultaneously alleviate the external memory access bandwidth. Moreover, variable block size ME (VBSME) and large SW in high definition (HD) video encoder are both supported with good trade off among throughput, data regularity, and rate distortion performance. The simplified algorithm can achieve nearly 50% SW buffer saving with less than 0.15dB PSNR degradation at the worst case compared with full search VBSME.