Bineng Zhong

Deep Network Cascade for Image Super-resolution

In this paper, we propose a new model called deep network cascade (DNC) to gradually upscale low-resolution images layer by layer, each layer with a small scale factor. DNC is a cascade of multiple stacked collaborative local auto-encoders. In each layer of the cascade, non-local self-similarity search is first performed to enhance high-frequency texture details of the partitioned patches in the input image. The enhanced image patches are then input into a collaborative local auto-encoder (CLA) to suppress the noises as well as collaborate the compatibility of the overlapping patches. By closing the loop on non-local self-similarity search and CLA in a cascade layer, we can refine the super-resolution result, which is further fed into next layer until the required image scale. Experiments on image super-resolution demonstrate that the proposed DNC can gradually upscale a low-resolution image with the increase of network layers and achieve more promising results in visual quality as well as quantitative performance.

Visual tracking via weakly supervised learning from multiple imperfect oracles

Long-term persistent tracking in ever-changing environments is a challenging task, which often requires addressing difficult object appearance update problems. To solve them, most top-performing methods rely on online learning-based algorithms. Unfortunately, one inherent problem of online learning-based trackers is drift, a gradual adaptation of the tracker to non-targets. To alleviate this problem, we consider visual tracking in a novel weakly supervised learning scenario where (possibly noisy) labels but no ground truth are provided by multiple imperfect oracles (i.e., trackers), some of which may be mediocre. A probabilistic approach is proposed to simultaneously infer the most likely object position and the accuracy of each tracker. Moreover, an online evaluation strategy of trackers and a heuristic training data selection scheme are adopted to make the inference more effective and fast. Consequently, the proposed method can avoid the pitfalls of purely single tracking approaches and get reliable labeled samples to incrementally update each tracker (if it is an appearance-adaptive tracker) to capture the appearance changes. Extensive comparing experiments on challenging video sequences demonstrate the robustness and effectiveness of the proposed method.

Towards semantic embedding in visual vocabulary

Visual vocabulary serves as a fundamental component in many computer vision tasks, such as object recognition, visual search, and scene modeling. While state-of-the-art approaches build visual vocabulary based solely on visual statistics of local image patches, the correlative image labels are left unexploited in generating visual words. In this work, we present a semantic embedding framework to integrate semantic information from Flickr labels for supervised vocabulary construction. Our main contribution is a Hidden Markov Random Field modeling to supervise feature space quantization, with specialized considerations to label correlations: Local visual features are modeled as an Observed Field, which follows visual metrics to partition feature space. Semantic labels are modeled as a Hidden Field, which imposes generative supervision to the Observed Field with WordNet-based correlation constraints as Gibbs distribution. By simplifying the Markov property in the Hidden Field, both unsupervised and supervised (label independent) vocabularies can be derived from our framework. We validate our performances in two challenging computer vision tasks with comparisons to state-of-the-arts: (1) Large-scale image search on a Flickr 60,000 database; (2) Object recognition on the PASCAL VOC database.

CNNTracker: Online discriminative object tracking via deep convolutional neural network

Abstract Object appearance model is a crucial module for object tracking and numerous schemes have been developed for object representation with impressive performance. Traditionally, the features used in such object appearance models are predefined in a handcrafted offline way but not tuned for the tracked object. In this paper, we propose a deep learning architecture to learn the most discriminative features dynamically via a convolutional neural network (CNN). In particular, we propose to enhance the discriminative ability of the appearance model in three-fold. First, we design a simple yet effective method to transfer the features learned from CNNs on the source tasks with large scale training data to the new tracking tasks with limited training data. Second, to alleviate the tracker drifting problem caused by model update, we exploit both the ground truth appearance information of the object labeled in the initial frames and the image observations obtained online. Finally, a heuristic schema is used to judge whether updating the object appearance models or not. Extensive experiments on challenging video sequences from the CVPR2013 tracking benchmark validate the robustness and effectiveness of the proposed tracking method.

Robust tracking via patch-based appearance model and local background estimation

In this paper, to simultaneously address the tracker drift and occlusion problem, we propose a robust visual tracking algorithm via a patch-based adaptive appearance model driven by local background estimation. Inspired by human visual mechanisms (i.e., context-awareness and attentional selection), an object is represented with a patch-based appearance model, in which each patch outputs a confidence map during the tracking. Then, these confidence maps are combined via a robust estimator to finally get more robust and accurate tracking results. Moreover, we present a local spatial co-occurrence based background modeling approach to automatically estimate the local context background model of an interested object captured from a single camera, which may be stationary or moving. Finally, we utilize local background estimation to provide supervision to an analysis of possible occlusions and the adaption of patch-based appearance model of an object. Qualitative and quantitative experimental results on challenging videos demonstrate the robustness of the proposed method.

Kernel Similarity Modeling of Texture Pattern Flow for Motion Detection in Complex Background

This paper proposes a novel kernel similarity modeling of texture pattern flow (KSM-TPF) for background modeling and motion detection in complex and dynamic environments. The texture pattern flow encodes the binary pattern changes in both spatial and temporal neighborhoods. The integral histogram of texture pattern flow is employed to extract the discriminative features from the input videos. Different from existing uniform threshold based motion detection approaches which are only effective for simple background, the kernel similarity modeling is proposed to produce an adaptive threshold for complex background. The adaptive threshold is computed from the mean and variance of an extended Gaussian mixture model. The proposed KSM-TPF approach incorporates machine learning method with feature extraction method in a homogenous way. Experimental results on the publicly available video sequences demonstrate that the proposed approach provides an effective and efficient way for background modeling and motion detection.

Structured partial least squares for simultaneous object tracking and segmentation

Segmentation-based tracking methods are a class of powerful tracking methods that have been highly successful in alleviating model drift during online-learning of the trackers. These methods typically include a detection component and a segmentation component, in which the tracked objects are first located by detection; then the results from detection are used to guide the process of segmentation to reduce the noises in the training data. However, one of the limitations is that the processes of detection and segmentation are treated entirely separately. The drift from detection may affect the results of segmentation. This also aggravates the trackers drift. In this paper, we propose a novel method to address this limitation by incorporating structured labeling information in the partial least square analysis algorithms for simultaneous object tracking and segmentation. This allows for novel structured labeling constraints to be placed directly on the tracked objects to provide useful contour constraint to alleviate the drifting problem. We show through both visual results and quantitative measurements on the challenging sequences that our method produces more robust tracking results while obtaining accurate object segmentation results.

Hierarchical background subtraction using local pixel clustering

We propose a robust hierarchical background subtraction technique which takes the spatial relations of neighboring pixels in a local region into account to detect objects in difficult conditions. Our algorithm combines a per-pixel with a per-region background model in a hierarchical manner, which accentuates the advantages of each. This is a natural combination because the two models have complementary strengths. The per-pixel background model is achieved by mixture of Gaussians models (GMM) with RGB feature. Although precisely describing background change in high resolution, it suffers from the sensitivity to quick variations in dynamic environment. To tolerate these quick variations, we further develop a novel GMM based per-region background model, which is updated by the cluster centers obtained from a k-means clustering of the pixelspsila RGB feature in the region. Numerical and qualitative experimental results on challenging videos demonstrate the robustness of the proposed method.

Kernel Learning of Histogram of Local Gabor Phase Patterns for Face Recognition

This paper proposes a new face recognition method, named kernel learning of histogram of local Gabor phase pattern (K-HLGPP), which is based on Daugmans method for iris recognition and the local XOR pattern (LXP) operator. Unlike traditional Gabor usage exploiting the magnitude part in face recognition, we encode the Gabor phase information for face classification by the quadrant bit coding (QBC) method. Two schemes are proposed for face recognition. One is based on the nearest-neighbor classifier with chi-square as the similarity measurement, and the other makes kernel discriminant analysis for HLGPP (K-HLGPP) using histogram intersection and Gaussian-weighted chi-square kernels. The comparative experiments show that K-HLGPP achieves a higher recognition rate than other well-known face recognition systems on the large-scale standard FERET, FERET200, and CAS-PEAL-R1 databases.

Robust infrared target tracking based on particle filter with embedded saliency detection

Infrared target tracking has attracted extensive research efforts in recent years. However, effective and efficient infrared target tracking is still a hard problem due to the low signal-to-noise ratio, difficulty of robustly describing complicated appearance variations as well as the abrupt motion of targets. In this paper, we propose a tracking method under the Particle Filtering framework by using a hierarchical sampling method, in which two complementary appearance models are used. Firstly, a saliency appearance model is proposed to suppress the cluttered background and properly guide particles to appropriate states. Then the eigen space model is employed as the other observation method to accurately estimate the target state. The hierarchical sampling process is proposed to incorporate the two complementary observation models to account for the abrupt motion efficiently. Experimental results on AMCOM FLIR sequences and comparisons with the state-of-the-art methods demonstrate that the proposed method is robust to appearance changes as well as drastic abrupt motions.