Matej Kristan

The Visual Object Tracking VOT2014 Challenge Results

Visual tracking has attracted a significant attention in the last few decades. The recent surge in the number of publications on tracking-related problems have made it almost impossible to follow the developments in the field. One of the reasons is that there is a lack of commonly accepted annotated data-sets and standardized evaluation protocols that would allow objective comparison of different tracking methods. To address this issue, the Visual Object Tracking (VOT) workshop was organized in conjunction with ICCV2013. Researchers from academia as well as industry were invited to participate in the first VOT2013 challenge which aimed at single-object visual trackers that do not apply pre-learned models of object appearance (model-free). Presented here is the VOT2013 benchmark dataset for evaluation of single-object visual trackers as well as the results obtained by the trackers competing in the challenge. In contrast to related attempts in tracker benchmarking, the dataset is labeled per-frame by visual attributes that indicate occlusion, illumination change, motion change, size change and camera motion, offering a more systematic comparison of the trackers. Furthermore, we have designed an automated system for performing and evaluating the experiments. We present the evaluation protocol of the VOT2013 challenge and the results of a comparison of 27 trackers on the benchmark dataset. The dataset, the evaluation tools and the tracker rankings are publicly available from the challenge website (http://votchallenge.net).

Robust Visual Tracking Using an Adaptive Coupled-Layer Visual Model

This paper addresses the problem of tracking objects which undergo rapid and significant appearance changes. We propose a novel coupled-layer visual model that combines the targets global and local appearance by interlacing two layers. The local layer in this model is a set of local patches that geometrically constrain the changes in the targets appearance. This layer probabilistically adapts to the targets geometric deformation, while its structure is updated by removing and adding the local patches. The addition of these patches is constrained by the global layer that probabilistically models the targets global visual properties, such as color, shape, and apparent local motion. The global visual properties are updated during tracking using the stable patches from the local layer. By this coupled constraint paradigm between the adaptation of the global and the local layer, we achieve a more robust tracking through significant appearance changes. We experimentally compare our tracker to 11 state-of-the-art trackers. The experimental results on challenging sequences confirm that our tracker outperforms the related trackers in many cases by having a smaller failure rate as well as better accuracy. Furthermore, the parameter analysis shows that our tracker is stable over a range of parameter values.

A trajectory-based analysis of coordinated team activity in a basketball game

This paper proposes a novel, trajectory-based approach to the automatic recognition of complex multi-player behavior in a basketball game. First, a probabilistic play model is applied to the player-trajectory data in order to segment the play into game phases (offense, defense, time out). In this way, both the temporal boundaries of the observed activity and its broader context are obtained. Next, the teams activity is analyzed in more detail by detecting the key elements of basketball play. Following basketball theory, these key elements (starting formation, screen, and move) are the building blocks of basketball play, and therefore their temporal order is used to produce a semantic description of the observed activity. Finally, the activity is recognized by comparing its semantic description with the descriptions of manually defined templates, stored in a database. The effectiveness and robustness of the proposed approach is demonstrated on two championship games and 71 examples of three types of basketball offense.

A Novel Performance Evaluation Methodology for Single-Target Trackers

This paper addresses the problem of single-target tracker performance evaluation. We consider the performance measures, the dataset and the evaluation system to be the most important components of tracker evaluation and propose requirements for each of them. The requirements are the basis of a new evaluation methodology that aims at a simple and easily interpretable tracker comparison. The ranking-based methodology addresses tracker equivalence in terms of statistical significance and practical differences. A fully-annotated dataset with per-frame annotations with several visual attributes is introduced. The diversity of its visual properties is maximized in a novel way by clustering a large number of videos according to their visual attributes. This makes it the most sophistically constructed and annotated dataset to date. A multi-platform evaluation system allowing easy integration of third-party trackers is presented as well. The proposed evaluation methodology was tested on the VOT2014 challenge on the new dataset and 38 trackers, making it the largest benchmark to date. Most of the tested trackers are indeed state-of-the-art since they outperform the standard baselines, resulting in a highly-challenging benchmark. An exhaustive analysis of the dataset from the perspective of tracking difficulty is carried out. To facilitate tracker comparison a new performance visualization technique is proposed.

A Bayes-spectral-entropy-based measure of camera focus using a discrete cosine transform

In this paper we present a novel measure of camera focus based on the Bayes spectral entropy of an image spectrum. In order to estimate the degree of focus, the image is divided into non-overlapping sub-images of 8x8 pixels. Next, sharpness values are calculated separately for each sub-image and their mean is taken as a measure of the overall focus. The sub-image spectra are obtained by an 8x8 discrete cosine transform (DCT). Comparisons were made against four well-known measures that were chosen as reference, on images captured with a standard visible-light camera and a thermal camera. The proposed measure outperformed the reference measures by exhibiting a wider working range and a smaller failure rate. To assess its robustness to noise, additional tests were conducted with noisy images.

An adaptive coupled-layer visual model for robust visual tracking

This paper addresses the problem of tracking objects which undergo rapid and significant appearance changes. We propose a novel coupled-layer visual model that combines the targets global and local appearance. The local layer in this model is a set of local patches that geometrically constrain the changes in the targets appearance. This layer probabilistically adapts to the targets geometric deformation, while its structure is updated by removing and adding the local patches. The addition of the patches is constrained by the global layer that probabilistically models targets global visual properties such as color, shape and apparent local motion. The global visual properties are updated during tracking using the stable patches from the local layer. By this coupled constraint paradigm between the adaptation of the global and the local layer, we achieve a more robust tracking through significant appearance changes. Indeed, the experimental results on challenging sequences confirm that our tracker outperforms the related state-of-the-art trackers by having smaller failure rate as well as better accuracy.

Closed-world tracking of multiple interacting targets for indoor-sports applications

In this paper we present an efficient algorithm for tracking multiple players during indoor sports matches. A sports match can be considered as a semi-controlled environment for which a set of closed-world assumptions regarding the visual as well as the dynamical properties of the players and the court can be derived. These assumptions are then used in the context of particle filtering to arrive at a computationally fast, closed-world, multi-player tracker. The proposed tracker is based on multiple, single-player trackers, which are combined using a closed-world assumption about the interactions among players. With regard to the visual properties, the robustness of the tracker is achieved by deriving a novel sports-domain-specific likelihood function and employing a novel background-elimination scheme. The restrictions on the players dynamics are enforced by employing a novel form of local smoothing. This smoothing renders the tracking more robust and reduces the computational complexity of the tracker. We evaluated the proposed closed-world, multi-player tracker on a challenging data set. In comparison with several similar trackers that did not utilize all of the closed-world assumptions, the proposed tracker produced better estimates of position and prediction as well as reducing the number of failures.

Visual Object Tracking Performance Measures Revisited

The problem of visual tracking evaluation is sporting a large variety of performance measures, and largely suffers from lack of consensus about which measures should be used in experiments. This makes the cross-paper tracker comparison difficult. Furthermore, as some measures may be less effective than others, the tracking results may be skewed or biased toward particular tracking aspects. In this paper, we revisit the popular performance measures and tracker performance visualizations and analyze them theoretically and experimentally. We show that several measures are equivalent from the point of information they provide for tracker comparison and, crucially, that some are more brittle than the others. Based on our analysis, we narrow down the set of potential measures to only two complementary ones, describing accuracy and robustness, thus pushing toward homogenization of the tracker evaluation methodology. These two measures can be intuitively interpreted and visualized and have been employed by the recent visual object tracking challenges as the foundation for the evaluation methodology.

A Two-Stage Dynamic Model for Visual Tracking

We propose a new dynamic model which can be used within blob trackers to track the targets center of gravity. A strong point of the model is that it is designed to track a variety of motions which are usually encountered in applications such as pedestrian tracking, hand tracking, and sports. We call the dynamic model a two-stage dynamic model due to its particular structure, which is a composition of two models: a liberal model and a conservative model. The liberal model allows larger perturbations in the targets dynamics and is able to account for motions in between the random-walk dynamics and the nearly constant-velocity dynamics. On the other hand, the conservative model assumes smaller perturbations and is used to further constrain the liberal model to the targets current dynamics. We implement the two-stage dynamic model in a two-stage probabilistic tracker based on the particle filter and apply it to two separate examples of blob tracking: 1) tracking entire persons and 2) tracking of a persons hands. Experiments show that, in comparison to the widely used models, the proposed two-stage dynamic model allows tracking with smaller number of particles in the particle filter (e.g., 25 particles), while achieving smaller errors in the state estimation and a smaller failure rate. The results suggest that the improved performance comes from the models ability to actively adapt to the targets motion during tracking.

Histograms of optical flow for efficient representation of body motion

A novel method for efficient encoding of human body motion, extracted from image sequences is presented. Optical flow field is calculated from sequential images, and the part of the flow field containing a person is subdivided into six segments. For each of the segments, a two dimensional, eight-bin histogram of optical flow is calculated. A symbol is generated, corresponding to the bin with the maximum sample count. Since the optical flow sequences before and after the temporal reference point are processed separately, twelve symbol sequences are obtained from the whole image sequence. Symbol sequences are purged of all symbol repetitions. To establish the similarity between two motion sequences, two sets of symbol sequences are compared. In our case, this is done by the means of normalized Levenshtein distance. Due to use of symbol sequences, the method is extremely storage efficient. It is also performance efficient, as it could be performed in near-realtime using the motion vectors from MPEG4 encoded video sequences. The approach has been tested on video sequences of persons entering restricted area using keycard and fingerprint reader. We show that it could be applied both to verification of person identities due to minuscule differences in their motion, and to detection of unusual behavior, such as tailgating.