Andrew G. Backhouse

Robust Visual Object Tracking Using Multi-Mode Anisotropic Mean Shift and Particle Filters

This paper addresses issues in object tracking where videos contain complex scenarios. We propose a novel tracking scheme that jointly employs particle filters and multi-mode anisotropic mean shift. The tracker estimates the dynamic shape and appearance of objects, and also performs online learning of reference object. Several partition prototypes and fully tunable parameters are applied to the rectangular object bounding box for improving the estimates of shape and multiple appearance modes in the object. The main contributions of the proposed scheme include: 1) use a novel approach for online learning of reference object distributions; 2) use a five parameter set (2-D central location, width, height, and orientation) of rectangular bounding box as tunable variables in the joint tracking scheme; 3) derive the multi-mode anisotropic mean shift related to a partitioned rectangular bounding box and several partition prototypes; and 4) relate the bounding box parameter computation with the multi-mode mean shift estimates by combining eigen decomposition, geometry of subareas, and weighted average. This has led to more accurate and efficient tracking where only small number of particles (<;20) is required. Experiments have been conducted for a range of videos captured by a dynamic or stationary camera, where the target object may experience long-term partial occlusions, intersections with other objects with similar color distributions, deformable object accompanied with shape, pose or abrupt motion speed changes, and cluttered background. Comparisons with existing methods and performance evaluations are also performed. Test results have shown marked improvement of the proposed method in terms of robustness to occlusions, tracking drifts and tightness and accuracy of tracked bounding box. Limitations of the method are also mentioned.

Joint anisotropic mean shift and consensus point feature correspondences for object tracking in video

We propose a novel tracking scheme that jointly employs point feature correspondences and object appearance similarity. For selecting point correspondences, we use a subset of scaleinvariant point features from SIFT that agree with a pre-defined affine transformation. The selected consensus points are then used for pre-selecting candidate regions. For appearance similarity based tracking, we employ an existing anisotropic mean shift, from which the formula for estimating bounding box parameters (width, height, orientation and center) are derived. A switching criterion is utilized to handle the situation where only a small number of point correspondences is found. Experiments and evaluation are performed on tracking moving objects on videos where objects may contain partial occlusions, intersection, deformation and pose changes among other transforms. Our comparisons with two existing methods have shown that the proposed scheme has yielded marked improvement, especially in terms of reducing tracking drifts, of robustness to occlusions, and of tightness and accuracy of tracked bounding box.

Joint particle filters and multi-mode anisotropic mean shift for robust tracking of video objects with partitioned areas

We propose a novel scheme that jointly employs anisotropic mean shift and particle filters for tracking moving objects from video. The proposed anisotropic mean shift, that is applied to partitioned areas in a candidate object bounding box whose parameters (center, width, height and orientation) are adjusted during the mean shift iterations, seeks multiple local modes in spatial-kernel weighted color histograms. By using a Gaussian distributed Bhattacharyya distance as the likelihood and mean shift updated parameters as the state vector, particle filters become more efficient in terms of tracking using a small number of particles (≪20). The combined scheme is able to maintain the merits of both methods. Experiments conducted on videos containing deformable objects with long-term partial occlusions and intersections have shown robust tracking performance. Comparisons with two existing methods have been made which showed marked improvement in terms of robustness to occlusions, tightness and accuracy of tracked box, and tracking drift.

Face tracking using Rao-Blackwellized particle filter and pose-dependent probabilistic PCA

This paper deals with tracking of face blobs containing pose changes. We propose a novel tracking method to deal with face pose changes during the tracking. In the method, tracking is formulated as an approximate solution to the MAP estimate of the state vector, consisting of a linear and a nonlinear part. Multi-pose face appearances are described by local linear models, each being related to a single pose and estimated by probabilistic PCA (PPCA). A Markov model with pose indices as its states is used to model the transitions between poses. Shape and locations of face blobs and the associated pose indices are assumed to be nonlinear, and are estimated by a Rao-Blackwellized particle filter (RBPF). This enables a separate estimation of the linear state vector through marginalizing the joint probability. The proposed method has been tested for videos containing frequent face pose changes and large illumination variations, where 5 poses (left, frontal, right, up, down) were modeled. The tracking results are shown to be robust to variable speed of pose changes and with relatively tight boxes.

Moving Object Tracking from Videos Based on Enhanced Space-Time-Range Mean Shift and Motion Consistency

Video surveillance and object tracking have drawn increased interests in recent years. This paper addresses the problem of moving object tracking from image sequences captured from stationary cameras. Based on the previous work on video segmentation using joint space-time-range mean shift, we extend the scheme to enable the tracking of moving objects. Large displacements of pdf modes in consecutive image frames are exploited for tracking. We also improve the above mean shift-based video segmentation by introducing edge-guided merging of over-segmented regions. This can be viewed as an extension of the enhanced mean shift 2D image segmentation in [2] to the enhanced space-time-range mean shift video segmentation. Experiments have been conducted on several indoor and outdoor videos. Our preliminary results and performance evaluation have indicated the effectiveness of the proposed scheme.

A Bayesian framework-based end-to-end packet loss prediction in IP networks

Channel modelling in a network path is of major importance in designing delay sensitive applications. It is often not possible for these applications to retransmit packets due to delay constraints and they must therefore be resilient to packet losses. In this paper, we first establish an association between traffic delays and the queue size at a network gateway. A novel method for predicting packet losses is then proposed that is based on the correlation between the packet losses and the variations in the end-to-end time delay observed during transmission. We show that this makes it possible to predict packet losses before they occur. The transmission of multimedia streams can then be dynamically adjusted to account for the predicted losses. As a result, better error-resilience can be provided for multimedia streams transmitting through a dynamic network channel. This means that they can provide an improved quality of transmission under the same network budget constraint. Experiments have been performed and preliminary results have shown that the method can provide a much smoother and more reliable transmission of data.

Robust Object Tracking Using Particle Filters and Multi-region Mean Shift

In this paper, we introduce a novel algorithm which builds upon the combined anisotropic mean-shift and particle filter framework. The anisotropic mean-shift [4] with 5 degrees of freedom, is extended to work on a partition of the object into concentric rings. This adds spatial information to the description of the object which makes the algorithm more resilient to occlusion and less likely to mistake the object with other objects having similar color densities. Experiments conducted on videos containing deformable objects with long-term partial occlusion (or, short-term full occlusion) and intersection have shown robust tracking performance, especially in tracking objects with long term partial occlusion, short term full occlusion, close color background clutter, severe object deformation and fast changing motion. Comparisons with two existing methods have shown marked improvement in terms of robustness to occlusions, tightness and accuracy of tracked box, and tracking drifts.

Bayesian traffic dynamics and packet loss prediction for video over IP networks

Designing good network-adaptive, error resilient video coders for IP networks is a challenging task. Video data packets can be lost as a consequence of congestion in the network, causing a degradation in video quality at the receiver side. Predicting the packet loss probability is therefore an important step in the design of an efficient network-adaptive video coder.In this paper, we present a novel Bayesian-based approach that can dynamically predict the instantaneous packet loss probability as well as estimate the probability distribution of the cross-traffic rate. The method utilises observed losses and end-to-end measurements of the packet delays to predict the time-varying cross-traffic rate in the network. The posterior probability for the cross-traffic rate is established by using the Bayes formulation. To obtain this, the probability of delays conditioned on the cross-traffic are derived by decomposing the delays with a multi-scale wavelet transform. The probability of the observed losses conditioned on the cross-traffic are derived by studying the queueing properties in the network. The predicted packet loss probability is then derived from the posteriori distribution of the cross-traffic rate and the expected queueing behaviour. An algorithm for obtaining the numerical solution is also given.Two sets of simulations are performed to demonstrate the effectiveness of the proposed method. The first set is designed to test the performance of the proposed Bayesian method for dynamically predicting the packet loss probabilities. The results of the predicted packet loss probabilities are then compared with the ground truth of packet loss values for three different network traffic generators. In the 2nd set of simulations, the proposed method and the two-state Markov model are incorporated separately into a network-adaptive video codec whose coding rate is adapted to the predicted packet loss probabilities. The reconstructed videos using these two methods are then compared.

ML Nonlinear Smoothing for Image Segmentation and its Relationship to the Mean Shift

This paper addresses the issues of nonlinear edge-preserving image smoothing and segmentation. A ML-based approach is proposed which uses an iterative algorithm to solve the problem. First, assumptions about segments are made by describing the joint probability distribution of pixel positions and colours within segments. Based on these assumptions, an optimal smoothing algorithm is derived under the ML condition. By studying the derived algorithm, we show that the solution is related to a two-stage mean shift which is separated in space and range. This novel ML-based approach takes a new kernel function. Experiments have been conducted on a range of images to smooth and segment them. Visual results and evaluations with 2 objective criteria have shown that the proposed method has led to improved results which suffer from less over-segmentation than the standard mean-shift.

Error-resilient packet video coding using harmonic frame-expansions and temporal prediction

This paper proposes a novel error-resilient packet video codec through the use of harmonic frame-expansions. It provides robustness to packet video in erasure channels in an integrated framework without requiring inter/intra coding modes. In the proposed method, spatial redundancy is added by applying a frame-expansion to DCT coefficients while temporal redundancy is added by filtering the motion compensated reference blocks. The frame expansion provides excellent error-resilience at low frequencies, while filtering of motion-reference blocks prevents the propagation of errors at high frequencies. Mathematical expressions of the PSNR for packet video streams with arbitrary losses are derived using piecewise linear approximations. Preliminary simulation results have shown that the proposed codec has generated high quality video at low bit-rates with significant packet losses, and has much less visual artifact than the conventional codecs with similar PSNRs.