Sylvain Boltz

High-dimensional statistical distance for region-of-interest tracking: Application to combining a soft geometric constraint with radiometry

This paper deals with region-of-interest (ROI) tracking in video sequences. The goal is to determine in successive frames the region which best matches, in terms of a similarity measure, an ROI defined in a reference frame. Two aspects of a similarity measure between a reference region and a candidate region can be distinguished: radiometry which checks if the regions have similar colors and geometry which checks if these colors appear at the same location in the regions. Measures based solely on radiometry include distances between probability density functions (PDF) of color. The absence of geometric constraint increases the number of potential matches. A soft geometric constraint can be added to a PDF-based measure by enriching the color information with location, thus increasing the dimension of the domain of definition of the PDFs. However, high-dimensional PDF estimation is not trivial. Instead, we propose to compute the Kullback-Leibler distance between high-dimensional PDFs without explicitly estimating the PDFs. The distance is expressed directly from the samples using the k-th nearest neighbor framework. Tracking experiments were performed on several standard sequences.

A Minimum-Entropy Procedure for Robust Motion Estimation

We focus on motion estimation using a block matching approach and suggest using a minimum-entropy criterion. Many entropy-based estimation procedures exist, such as plug-in estimators based on Parzen windowing. We consider here an alternative that is applicable to data of any dimension and that circumvents the critical issues raised by kernel-based methods. To the best of our knowledge, this criterion has not yet been considered for image processing problems. The inherent robustness property of entropy is expected to provide a robust and efficient estimation of the motion vector of a block of a video sequence. In particular, the minimum-entropy estimator should be robust to occlusions and variations of luminance, for which standard approaches like SSD usually meet their limitations.

kNN-based high-dimensional Kullback-Leibler distance for tracking

This paper deals with region-of-interest (ROI) tracking in video sequences. The goal is to determine in successive frames the region which best matches, in terms of a similarity measure, a ROI defined in a reference frame. Two aspects of such a measure between the reference region and a candidate region can be distinguished: radiometry which indicates if the regions have similar colors and geometry which correlates where these colors are present in the regions. If not using geometry, the number of potential matches increases. A soft geometric constraint can be added in the form of a joint radiometric-geometric PDF. High-dimensional PDF estimation being a difficult problem, measures based on these PDF distances may lead to an incorrect match. Instead, we propose to compute the Kullback-Leibler distance between high-dimensional PDFs without explicit estimation of the PDFs, i.e., directly from the samples using the kth-nearest neighbor (kNN) framework. Results showed accurate tracking.

Motion and Appearance Nonparametric Joint Entropy for Video Segmentation

This paper deals with video segmentation based on motion and spatial information. Classically, the motion term is based on a motion compensation error (MCE) between two consecutive frames. Defining a motion-based energy as the integral of a function of the MCE over the object domain implicitly results in making an assumption on the MCE distribution: Gaussian for the square function and, more generally, parametric distributions for functions used in robust estimation. However, these assumptions are not necessarily appropriate. Instead, we propose to define the energy as a function of (an estimation of) the MCE distribution. This function was chosen to be a continuous version of the Ahmad-Lin entropy approximation, the purpose being to be more robust to outliers inherently present in the MCE. Since a motion-only constraint can fail with homogeneous objects, the motion-based energy is enriched with spatial information using a joint entropy formulation. The resulting energy is minimized iteratively using active contours. This approach provides a general framework which consists in defining a statistical energy as a function of a multivariate distribution, independently of the features associated with the object of interest. The link between the energy and the features observed or computed on the video sequence is then made through a nonparametric, kernel-based distribution estimation. It allows for example to keep the same energy definition while using different features or different assumptions on the features.

Outer-Layer Based Tracking using Entropy as a Similarity Measure

Tracking can be achieved using region active contours based on homogeneity models (intensity, motion...). However the model complexity necessary to achieve a given accuracy might be prohibitive. Methods based on salient points may not extract enough of these for reliable motion estimation if the object is too homogeneous. Here we propose to compute the contour deformation based on its neighborhood. Motion estimation is performed at contour samples using a block matching approach. First, partial background masking is applied. Since outliers may then bias the motion estimation, a robust, nonparametric estimation using entropy as a similarity measure between blocks is proposed. Tracking results on synthetic and natural sequences are presented.

Robust Motion-Based Segmentation in Video Sequences using Entropy Estimator

This paper deals with motion estimation and segmentation in video sequences. Some methods of motion computation between two consecutive frames of a video sequence are based on the minimization of the square error of the prediction error. More robust estimators such as absolute value or M-estimators were proposed but these estimators loose their efficiency when the data do not have parametric distributions. We relax the parametric assumption on the prediction error distribution and propose to use a nonparametric estimator for the motion estimation : the entropy of the prediction error. We use the same criterion to perform a spatio-temporal segmentation of the sequence using an active contour algorithm. Segmentation and tracking tests on a textured synthetic and a real sequence, compared to a standard method in motion segmentation, tends to show that our method is more stable and accurate.

Joint appearance and deformable shape for nonparametric segmentation

This paper deals with region-of-interest (ROI) segmentation in video sequences. The goal is to determine in one frame the region which best matches, in terms of a similarity measure, a ROI defined in a reference frame. A similarity measure can combine color histograms and geometry information into a joint PDF. Geometric information are basically interior region coordinates. We propose a system of shape coordinates constant under shape deformations. High-dimensional color-geometry PDF estimation being a difficult problem, measures based on these PDF distances may lead to an incorrect match. Instead, we use an estimator for Kullback-Leibler divergence efficient for high dimensional PDFs. The distance is expressed from the samples using the kth-nearest neighbor framework (kNN). We plugged this distance into active contour framework using shape derivative. Segmentation results on both rigid and articulated objects showed promising results.

Space-time segmentation based on a joint entropy with estimation of nonparametric distributions

This paper deals with video segmentation based on motion and spatial information. Classically, the nucleus of the motion term is the motion compensation error (MCE) between two consecutive frames. Defining a motion-based energy as the integral of a function of the MCE over the object domain implicitly results in making an assumption on the MCE distribution: Gaussian for the square function, Laplacian for the absolute value, or other parametric distributions for functions used in robust estimation. However, these assumptions are generally false. Instead, it is proposed to integrate a function of (an estimation of) the MCE distribution. The function is taken such that the integral is the Ahmad-Lin entropy of the MCE, the purpose being to be more robust to outliers. Since a motion-only approach can fail in homogeneous areas, the proposed energy is the joint entropy of the MCE and the object color. It is minimized using active contours.

A High Dimensional Framework for Joint Color-Spatial Segmentation

This paper deals with region-of-interest (ROI) segmentation in video sequences. The goal is to determine in successive frames the region which best matches, in terms of a similarity measure, a ROI defined in a reference frame. Color and geometry can be combined in a joint PDF. However such high-dimensional PDFs being hard to estimate, measures based on PDF distances may lead to incorrect segmentations. Here, we propose to use an estimate of the Kullback-Leibler divergence adapted to high-dimensional PDFs. It is defined from the samples using the kth-nearest neighbor (kNN) framework and it is differentiated for active contour implementation and expressed in both the continuous form and a kNN form. Results are presented on standard sequences.