Andrew Naftel

Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space

This paper proposes a novel technique for clustering and classification of object trajectory-based video motion clips using spatiotemporal function approximations. Assuming the clusters of trajectory points are distributed normally in the coefficient feature space, we propose a Mahalanobis classifier for the detection of anomalous trajectories. Motion trajectories are considered as time series and modelled using orthogonal basis function representations. We have compared three different function approximations – least squares polynomials, Chebyshev polynomials and Fourier series obtained by Discrete Fourier Transform (DFT). Trajectory clustering is then carried out in the chosen coefficient feature space to discover patterns of similar object motions. The coefficients of the basis functions are used as input feature vectors to a Self- Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Our proposed techniques are validated on three different datasets – Australian sign language, hand-labelled object trajectories from video surveillance footage and real-time tracking data obtained in the laboratory. Applications to event detection and motion data mining for multimedia video surveillance systems are envisaged.

Detection and classification of highway lanes using vehicle motion trajectories

Intelligent vision-based traffic surveillance systems are assuming an increasingly important role in highway monitoring and road management schemes. This paper describes a low-level object tracking system that produces accurate vehicle motion trajectories that can be further analyzed to detect lane centers and classify lane types. Accompanying techniques for indexing and retrieval of anomalous trajectories are also derived. The predictive trajectory merge-and-split algorithm is used to detect partial or complete occlusions during object motion and incorporates a Kalman filter that is used to perform vehicle tracking. The resulting motion trajectories are modeled using variable low-degree polynomials. A K-means clustering technique on the coefficient space can be used to obtain approximate lane centers. Estimation bias due to vehicle lane changes can be removed using robust estimation techniques based on Random Sample Consensus (RANSAC). Through the use of nonmetric distance functions and a simple directional indicator, highway lanes can be classified into one of the following categories: entry, exit, primary, or secondary. Experimental results are presented to show the real-time application of this approach to multiple views obtained by an uncalibrated pan-tilt-zoom traffic camera monitoring the junction of two busy intersecting highways.

Classifying spatiotemporal object trajectories using unsupervised learning of basis function coefficients

This paper proposes a novel technique for clustering and classification of object trajectory-based video motion clips using spatiotemporal functional approximations. A Mahalanobis classifier is then used for the detection of anomalous trajectories. Motion trajectories are considered as time series and modeled using the leading Fourier coefficients obtained by a Discrete Fourier Transform. Trajectory clustering is then carried out in the Fourier coefficient feature space to discover patterns of similar object motions. The coefficients of the basis functions are used as input feature vectors to a Self-Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Experiments are performed on two different datasets -- synthetic and pedestrian object tracking - to demonstrate the effectiveness of our approach. Applications to motion data mining in video surveillance databases are envisaged.

Motion Trajectory Learning in the DFT-Coefficient Feature Space

Techniques for understanding video object motion activity are becoming increasingly important with the widespread adoption of CCTV surveillance systems. In this paper we propose a novel vision system for clustering and classification of object-based video motion clips using spatiotemporal models. Object trajectories are modeled as motion time series using the lowest order Fourier coefficients obtained by Discrete Fourier Transform. Trajectory clustering is then carried out in the DFT-coefficient feature space to discover patterns of similar object motion activity. The DFT coefficients are used as input feature vectors to a Self-Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Assuming the clusters of trajectory points are distributed normally in the coefficient feature space, we propose a simple Mahalanobis classifier for the detection of anomalous trajectories. Our proposed techniques are validated on three different datasets - Australian sign language, handlabelled object trajectories from video surveillance footage and real-time tracking data obtained in the laboratory. Applications to event detection and motion data mining for visual surveillance systems are envisaged.

Viewpoint Independent Detection of Vehicle Trajectories and Lane Geometry from Uncalibrated Traffic Surveillance Cameras

In this paper, we present a low-level object tracking system that produces accurate vehicle trajectories and estimates the lane geometry using uncalibrated traffic surveillance cameras. A novel algorithm known as Predictive Trajectory Merge-and-Split (PTMS) has been developed to detect partial or complete occlusions during object motion and hence update the number of objects in each tracked blob. This hybrid algorithm is based on the Kalman filter and a set of simple heuristics for temporal analysis. Some preliminary results are presented on the estimation of lane geometry through aggregation and K-means clustering of many individual vehicle trajectories modelled by polynomials of varying degree. We show how this process can be made insensitive to the presence of vehicle lane changes inherent in the data. An advantage of this approach is that estimation of lane geometry can be performed with non-stationary uncalibrated cameras.

Evaluation of Matching Metrics for Trajectory-Based Indexing and Retrieval of Video Clips

This paper describes a comparative evaluation of three different similarity metrics for trajectory-based indexing and retrieval of video motion clips. The motion paths are generated using a low-level tracking algorithm incorporating first-order Kalman filter and colour appearance models. For simple motion paths, a RANSAC approach can be used to generate smooth trajectories for each tracked object described by low-order polynomials. This allows us to obtain a representative trajectory model even in the case of high numbers of outlier points caused by target mis-detection and multiple occlusions. We show that more complex trajectories including stop-start motions, can be modelled as time series using high order Chebyshev polynomials. Similarity metrics based on coefficient descriptors are shown to have comparable performance to a Hausdorff distance measure when retrieving trajectory-based motion clips but at substantially reduced computational cost. Experimental results are presented to illustrate the comparative performance of different matching metrics on real-world trajectory data collected by a retail store CCTV installation.

Stereo-assisted landmark detection for the analysis of changes in 3-D facial shape.

In this paper, a semi-automated approach to 3-D landmark digitization of the face is described which uses a combination of active shape model-driven feature detection and stereophotogrammetric analysis. The study aims to assess whether the proposed method is capable of detecting statistically significant changes in facial soft tissue shape due to mandibular repositioning in a cross-sectional patient sample. A hybrid stereophotogrammetric and structured-light imaging system is used for acquiring 3-D face models in the first instance. A landmark-based statistical analysis of facial shape change is then carried out using procrustes registration, principal component analysis and thin plate spline warping on the 2-D facial midline profiles and automatically digitized 3-D landmarks. The proposed method is validated both statistically and visually by characterizing shape changes induced by mandibular repositioning in a heterogeneous cross-sample of 20 orthodontic patients. It is shown that the method is capable of distinguishing between changes in facial morphology due to simulated surgical correction and changes due to other factors such as growth and normal variation within the patient sample. The study shows that the proposed method may be useful for auditing outcomes of clinical treatment or surgical intervention which result in changes to facial soft tissue morphology.

Motion trajectory clustering for video retrieval using spatio-temporal approximations

A new technique is proposed for clustering and similarity retrieval of video motion clips based on spatio-temporal object trajectories. The trajectories are treated as motion time series and modelled using orthogonal basis polynomial approximations. Trajectory clustering is then carried out to discover patterns of similar object motion behaviour. The coefficients of the basis functions are used as input feature vectors to a Self-Organising Map which can learn similarities between object trajectories in an unsupervised manner. Clustering in the basis coefficient space leads to efficiency gains over existing approaches that encode trajectories as point-based flow vectors. Experiments on pedestrian motion data gathered from video surveillance demonstrate the effectiveness of our approach. Applications to motion data mining in video surveillance databases are envisaged.

Automatic Motion Learning in the Presence of Anomalies Using Coefficient Feature Space Representation of Trajectories

Abstract Techniques for understanding video object motion activity are becoming increasingly important with the widespread adoption of CCTV surveillance systems. Motion trajectories provide rich spatiotemporal information about an objects activity. This paper presents a novel technique for clustering of object trajectory-based video motion clips using basis function approximations. Motion cues can be extracted using a tracking algorithm on video streams from video cameras. In the proposed system, trajectories are treated as time series and modelled using orthogonal basis function representation. Various function approximations have been compared including least squares polynomial, Chebyshev polynomials, piecewise aggregate approximation, discrete Fourier transform (DFT), and modified DFT (DFT-MOD). A novel framework, namely iterative hierarchical semi-agglomerative clustering using learning vector quantization (Iterative HSACT-LVQ), is proposed for learning of patterns in the presence of significant number of anomalies in training data. In this context, anomalies are defined as atypical behavior patterns that are not represented by sufficient samples in training data and are infrequently occurring or unusual. The proposed algorithm does not require any prior knowledge about the number of patterns hidden in unclassified dataset. Experiments using complex real-life trajectory datasets demonstrate the superiority of our proposed Iterative HSACT-LVQ-based motion learning technique compared to other recent approaches.

Visual Recognition of Manual Tasks Using Object Motion Trajectories

Motion trajectories are powerful cues for event detection and recognition. In this paper we present a system for manual task analysis that distinguishes between skin and object motion and learns activity patterns through analysing object trajectories. It is particularly suited to the recognition of common object handling tasks. Our vision system performs hand skin detection and object segmentation for each frame in a sequence. The object trajectories are then modelled as motion time series. We have compared the performance of several different time series indexing schemes: symbolic, polynomial and orthonormal basis functions used for trajectory similarity retrieval and classification. We then attempt to cluster objectcentred motion patterns in the coefficient feature space. The proposed technique is validated on two different datasets, Australian Sign Language and object handling data obtained in the laboratory. Applications to task recognition and motion data mining in industrial surveillance applications are envisaged.