Dawei Du

JOTS: Joint Online Tracking and Segmentation

We present a novel Joint Online Tracking and Segmentation (JOTS) algorithm which integrates the multi-part tracking and segmentation into a unified energy optimization framework to handle the video segmentation task. The multi-part segmentation is posed as a pixel-level label assignment task with regularization according to the estimated part models, and tracking is formulated as estimating the part models based on the pixel labels, which in turn is used to refine the model. The multi-part tracking and segmentation are carried out iteratively to minimize the proposed objective function by a RANSAC-style approach. Extensive experiments on the SegTrack and SegTrack v2 databases demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

Online Deformable Object Tracking Based on Structure-Aware Hyper-Graph

Recent advances in online visual tracking focus on designing part-based model to handle the deformation and occlusion challenges. However, previous methods usually consider only the pairwise structural dependences of target parts in two consecutive frames rather than the higher order constraints in multiple frames, making them less effective in handling large deformation and occlusion challenges. This paper describes a new and efficient method for online deformable object tracking. Different from most existing methods, this paper exploits higher order structural dependences of different parts of the tracking target in multiple consecutive frames. We construct a structure-aware hyper-graph to capture such higher order dependences, and solve the tracking problem by searching dense subgraphs on it. Furthermore, we also describe a new evaluating data set for online deformable object tracking (the Deform-SOT data set), which includes 50 challenging sequences with full annotations that represent realistic tracking challenges, such as large deformations and severe occlusions. The experimental result of the proposed method shows considerable improvement in performance over the state-of-the-art tracking methods.

Geometric Hypergraph Learning for Visual Tracking

Graph-based representation is widely used in visual tracking field by finding correct correspondences between target parts in different frames. However, most graph-based trackers consider pairwise geometric relations between local parts. They do not make full use of the target’s intrinsic structure, thereby making the representation easily disturbed by errors in pairwise affinities when large deformation or occlusion occurs. In this paper, we propose a geometric hypergraph learning-based tracking method, which fully exploits high-order geometric relations among multiple correspondences of parts in different frames. Then visual tracking is formulated as the mode-seeking problem on the hypergraph in which vertices represent correspondence hypotheses and hyperedges describe high-order geometric relations among correspondences. Besides, a confidence-aware sampling method is developed to select representative vertices and hyperedges to construct the geometric hypergraph for more robustness and scalability. The experiments are carried out on three challenging datasets (VOT2014, OTB100, and Deform-SOT) to demonstrate that our method performs favorably against other existing trackers.

Abnormal event detection in crowded scenes based on Structural Multi-scale Motion Interrelated Patterns

Detecting abnormal events in crowded scenes remains challenging due to the diversity of events defined by various applications. Among the many application situations, motion analysis for event representation is suited for crowded scenes. In this paper, we propose a novel abnormal event detection method via likelihood estimation of dynamic-texture motion representation, called Structural Multi-scale Motion Interrelated Patterns (SMMIP). SMMIP combines both original motion patterns and their structural spatio-temporal information, which effectively represents localized events by different resolutions of motion patterns. To model normal events, the Gaussian mixture model is trained with the observed normal events, then the likelihood estimation for testing events is computed to judge whether they are abnormal. Meanwhile, the proposed model can be learned online by updating the parameters incrementally. The proposed approach is evaluated on several publicly available datasets and outperforms several other methods proposed before, which is shown that the structural spatio-temporal information added in motion representation helps increasing the anomalies detection rate.

Learning Discriminative Hidden Structural Parts for Visual Tracking

Part-based visual tracking is attractive in recent years due to its robustness to occlusion and non-rigid motion. However, how to automatically generate the discriminative structural parts and consider their interactions jointly to construct a more robust tracker still remains unsolved. This paper proposes a discriminative structural part learning method while integrating the structure information, to address the visual tracking problem. Particulary, the state (e.g. position, width and height) of each part is regarded as a hidden variable and inferred automatically by considering the inner structure information of the target and the appearance difference between the target and the background. The inner structure information considering the relationship between neighboring parts, is integrated using a graph model based on a dynamically constructed pair-wise Markov Random Field. Finally, we adopt Metropolis-Hastings algorithm integrated with the online Support Vector Machine to complete the hidden variable inference task. The experimental results on various challenging sequences demonstrate the favorable performance of the proposed tracker over the state-of-the-art ones.

Iterative Graph Seeking for Object Tracking

To effectively solve the challenges in object tracking, such as large deformation and severe occlusion, many existing methods use graph-based models to capture target part relations, and adopt a sequential scheme of target part selection, part matching, and state estimation. However, such methods have two major drawbacks: 1) inaccurate part selection leads to performance deterioration of part matching and state estimation and 2) there are insufficient effective global constraints for local part selection and matching. In this paper, we propose a new object tracking method based on iterative graph seeking, which integrate target part selection, part matching, and state estimation using a unified energy minimization framework. Our method also incorporates structural information in local parts variations using the global constraint. We devise an alternative iteration scheme to minimize the energy function for searching the most plausible target geometric graph. Experimental results on several challenging benchmarks (i.e., VOT2015, OTB2013, and OTB2015) demonstrate improved performance and robustness in comparison with existing algorithms.

UA-DETRAC 2017: Report of AVSS2017 & IWT4S Challenge on Advanced Traffic Monitoring

The rapid advances of transportation infrastructure have led to a dramatic increase in the demand for smart systems capable of monitoring traffic and street safety. Fundamental to these applications are a community-based evaluation platform and benchmark for object detection and multi-object tracking. To this end, we organize the AVSS2017 Challenge on Advanced Traffic Monitoring, in conjunction with the International Workshop on Traffic and Street Surveillance for Safety and Security (IWT4S), to evaluate the state-of-the-art object detection and multi-object tracking algorithms in the relevance of traffic surveillance. Submitted algorithms are evaluated using the large-scale UA-DETRAC benchmark and evaluation protocol. The benchmark, the evaluation toolkit and the algorithm performance are publicly available from the website http://detrac-db.rit.albany.edu.

Recover image details from LDR photographs

In this paper, a novel self-adaptive curve, based on human visual model (HVM), is proposed for recovering details from low dynamic range (LDR) digital photographs, which are under-exposed or over-exposed or both. In order to improve the perceptual visibility, we utilize HVM to construct our method, which is able to take advantage of entire dynamic range to enhance the contrast of images. Extensive experiments demonstrate that our method consistently achieves satisfying results for unwell-exposed LDR photographs.