Chongyang Zhang

Long-term correlation tracking

In this paper, we address the problem of long-term visual tracking where the target objects undergo significant appearance variation due to deformation, abrupt motion, heavy occlusion and out-of-view. In this setting, we decompose the task of tracking into translation and scale estimation of objects. We show that the correlation between temporal context considerably improves the accuracy and reliability for translation estimation, and it is effective to learn discriminative correlation filters from the most confident frames to estimate the scale change. In addition, we train an online random fern classifier to re-detect objects in case of tracking failure. Extensive experimental results on large-scale benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art methods in terms of efficiency, accuracy, and robustness.

GOP-level transmission distortion modeling for mobile streaming video

Unequal loss protection is an effective tool in delivering compressed video streaming over packet-switched networks robustly. A critical component in any unequal-loss-protection scheme is a metric for evaluating the importance of different frames in a Group-Of-Pictures (GOP). In the case of video streaming over 3G mobile networks, packet loss usually corresponds to whole-frame loss due to low bandwidth and small picture size, which results in high error rates and thus most of the existing low-complexity transmission-distortion-estimate models may be ineffective. In this paper, we firstly develop a recursive algorithm to compute the GOP-level transmission distortion at pixel-level precision using pre-computed video information. Based on the study on the propagating behavior of the whole-frame-loss transmission distortion, we then propose a piecewise linear-fitting approach to achieve low-complexity transmission distortion modeling. The simulation results demonstrate that the proposed two models are accurate and robust. The proposed transmission distortion models are fast and accurate importance assessment tools in allocating limited channel resources optimally for the mobile streaming video.

Improved image deblurring based on salient-region segmentation

Image deblurring techniques play important roles in many image processing applications. As the blur varies spatially across the image plane, it calls for robust and effective methods to deal with the spatially-variant blur problem. In this paper, a Saliency-based Deblurring (SD) approach is proposed based on the saliency detection for salient-region segmentation and a corresponding compensate method for image deblurring. We also propose a PDE-based deblurring method which introduces an anisotropic Partial Differential Equation (PDE) model for latent image prediction and employs an adaptive optimization model in the kernel estimation and deconvolution steps. Experimental results demonstrate the effectiveness of the proposed algorithm.

Sketch Retrieval via Dense Stroke Features.

Sketch retrieval aims at retrieving most similar sketches from a large database based on one hand-drawn query. Successful retrieval hinges on an effective representation of sketch images and an efficient search method. In this paper, we propose a representation scheme which takes sketch strokes into account with local features, thereby facilitating efficient retrieval with codebooks. Stroke features are detected via densely sampled points on stroke lines from which local gradients are further enhanced and described by a quantized histogram of gradients. A codebook is organized in a hierarchical vocabulary tree, which maintains structural information of visual words and enables efficient retrieval in sub-linear time. Experimental results on three data sets demonstrate the merits of the proposed algorithm for effective and efficient sketch retrieval.

Intra-and-Inter-Constraint-Based Video Enhancement Based on Piecewise Tone Mapping

Video enhancement plays an important role in various video applications. In this paper, we propose a new intra-and-inter-constraint-based video enhancement approach aiming to: 1) achieve high intraframe quality of the entire picture where multiple regions-of-interest (ROIs) can be adaptively and simultaneously enhanced, and 2) guarantee the interframe quality consistencies among video frames. We first analyze features from different ROIs and create a piecewise tone mapping curve for the entire frame such that the intraframe quality can be enhanced. We further introduce new interframe constraints to improve the temporal quality consistency. Experimental results show that the proposed algorithm obviously outperforms the state-of-the-art algorithms.

Sketch retrieval via local dense stroke features

Sketch retrieval aims at retrieving the most similar sketches from a large database based on one hand-drawn query. Successful retrieval hinges on an effective representation of sketch images and an efficient search method. In this paper, we propose a representation scheme which takes sketch strokes into account with local features, thereby facilitating efficient retrieval with codebooks. Stroke features are detected via densely sampled points on stroke lines with crucial corners as anchor points, from which local gradients are enhanced and described by a quantized histogram of gradients. A codebook is organized in a hierarchical vocabulary tree, which maintains structural information of visual words and enables efficient retrieval in sub-linear time. Experimental results on three data sets demonstrate the merits of the proposed algorithm for effective and efficient sketch retrieval. Display Omitted

Learning a temporally invariant representation for visual tracking

In this paper, we propose to learn temporally invariant features from a large number of image sequences to represent objects for visual tracking. These features are trained on a convolutional neural network with temporal invariance constraints and robust to diverse motion transformations. We employ linear correlation filters to encode the appearance templates of targets and perform the tracking task by searching for the maximum responses at each frame. The learned filters are updated online and adapt to significant appearance changes during tracking. Extensive experimental results on challenging sequences show that the proposed algorithm performs favorably against state-of-the-art methods in terms of efficiency, accuracy, and robustness.

Dynamic background subtraction based on appearance and motion pattern

Moving objects detection plays a critical role in computer vision application, since it usually is the first phase in video processing. Most traditional methods are used in static scenes, however not perform well in dynamic situations, or they can only overcome limited perturbation. In this paper, we propose a stereo local binary pattern based on appearance and motion (SLBP-AM) descriptor for back-ground modeling and objects detection. We regard the motion of pixels as dynamic texture in ellipsoidal domain, and combine texture histograms in the XY, XT, YT planes in the ellipsoid as the new descriptor for background subtraction. Compared with traditional local binary pattern (LBP) descriptor, experiment results show that the new proposed method can not only be robust to slight disturbance, but also adapt quickly to the large-scale and sudden changes.

Distortion-Minimized Video Slicing for Unequal Loss Protection

This letter proposes a distortion-minimized slicing scheme for the unequal loss protection of compressed video bitstreams transported over packet networks. Unlike most existing slicing methods where each slice includes nearly equal number of video macroblocks, the proposed scheme reorders the macroblocks in one video frame according to their importance, and then divides them into two slices with an unequal proportion. According to given channel conditions and a novel unequal loss protection scheme, the more appropriate macroblock division ratio can be found out to achieve minimized end-to-end distortion. Simulation results show that the proposed slicing scheme outperforms state-of-the-art approaches with fixed macroblock division ratio.

Cross-Layer Frame Discarding for Cellular Video Coding

In the case of delivering real-time video over the 3G cellular networks, burst frame losses may be inevitable and unpredictable, which may cause severe quality degradation. Based on cross-layer frame discarding (CLFD), this paper proposes an enhanced error-resilient video coding scheme for cellular video communication. By using unequal retransmission at the radio link (RL) layer, a base station can provide reliable transmission for the relatively important frames in one video sequence. Relying on the unequal protection at the RL layer, the encoder at the application (APP) layer can actively discard a certain number of frames according to the received acknowledgement messages. Thus, unpredictable burst frame losses during transmission can be transformed into selective frame discarding at the encoder. Experiments results show that the proposed scheme can enhance the error resilience of the cellular video communication significantly.