Brendan Morris

A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance

This paper presents a survey of trajectory-based activity analysis for visual surveillance. It describes techniques that use trajectory data to define a general set of activities that are applicable to a wide range of scenes and environments. Events of interest are detected by building a generic topographical scene description from underlying motion structure as observed over time. The scene topology is automatically learned and is distinguished by points of interest and motion characterized by activity paths. The methods we review are intended for real-time surveillance through definition of a diverse set of events for further analysis triggering, including virtual fencing, speed profiling, behavior classification, anomaly detection, and object interaction.

Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach

Society is rapidly accepting the use of video cameras in many new and varied locations, but effective methods to utilize and manage the massive resulting amounts of visual data are only slowly developing. This paper presents a framework for live video analysis in which the behaviors of surveillance subjects are described using a vocabulary learned from recurrent motion patterns, for real-time characterization and prediction of future activities, as well as the detection of abnormalities. The repetitive nature of object trajectories is utilized to automatically build activity models in a 3-stage hierarchical learning process. Interesting nodes are learned through Gaussian mixture modeling, connecting routes formed through trajectory clustering, and spatio-temporal dynamics of activities probabilistically encoded using hidden Markov models. Activity models are adapted to small temporal variations in an online fashion using maximum likelihood regression and new behaviors are discovered from a periodic retraining for long-term monitoring. Extensive evaluation on various data sets, typically missing from other work, demonstrates the efficacy and generality of the proposed framework for surveillance-based activity analysis.

Learning, Modeling, and Classification of Vehicle Track Patterns from Live Video

This paper presents two different types of visual activity analysis modules based on vehicle tracking. The highway monitoring module accurately classifies vehicles into eight different types and collects traffic flow statistics by leveraging tracking information. These statistics are continuously accumulated to maintain daily highway models that are used to categorize traffic flow in real time. The path modeling block is a more general analysis tool that learns the normal motions encountered in a scene in an unsupervised fashion. The spatiotemporal motion characteristics of these motion paths are encoded by a hidden Markov model. With the path definitions, abnormal trajectories are detected and future intent is predicted. These modules add realtime situational awareness to highway monitoring for high-level activity and behavior analysis.

Learning trajectory patterns by clustering: Experimental studies and comparative evaluation

Recently a large amount of research has been devoted to automatic activity analysis. Typically, activities have been defined by their motion characteristics and represented by trajectories. These trajectories are collected and clustered to determine typical behaviors. This paper evaluates different similarity measures and clustering methodologies to catalog their strengths and weaknesses when utilized for the trajectory learning problem. The clustering performance is measured by evaluating the correct clustering rate on different datasets with varying characteristics.

On-road prediction of driver's intent with multimodal sensory cues

By predicting a drivers maneuvers before they occur, a driver-assistance system can prepare for or avoid dangerous situations. This article describes a real-time, on-road lane-change-intent detector that can enhance driver safety.

Lane change intent prediction for driver assistance: On-road design and evaluation

Automobiles are quickly becoming more complex as new sensors and support systems are being added to improve safety and comfort. The next generation of intelligent driver assistance systems will need to utilize this wide array of sensors to fully understand the driving context and situation. Effective interaction requires these systems to examine the intentions, desires, and needs of the driver for preemptive actions which can help prepare for or avoid dangerous situations. This manuscript develops a real-time on-road prediction system able to detect a drivers intention to change lanes seconds before it occurs. In-depth analysis highlights the challenges when moving intent prediction from the laboratory to the road and provides detailed characterization of on-road performance.

Robust classification and tracking of vehicles in traffic video streams

The widespread use of cameras for traffic monitoring coupled with the availability of robust tracking algorithms has led to volumes of data. It is necessary to process this data for higher level tasks. One of these processing tasks is vehicle type classification, which can be used in a query based management system. This paper presents a tracking system with the ability to classify vehicles into three classes {sedan, semi, truck+SUV+van}. This system was developed after comparing classification schemes using both vehicle images and measurements. The most accurate of these learned classifiers was integrated into tracking software. This merging of classification and tracking greatly improved the accuracy on low resolution traffic video

Learning and Classification of Trajectories in Dynamic Scenes: A General Framework for Live Video Analysis

This paper presents a general framework for live video analysis. The activities of surveillance subjects are described using a spatio-temporal vocabulary learned from recurrent motion patterns. The repetitive nature of object trajectories is used to build a topographical scene description where nodes are points of interest (POT) and the edges correspond to activity paths (AP). The POI are learned through as a mixture of Gaussians and AP by clustering trajectories. The paths are probabilistically represented by hidden Markov models and adapt to temporal variations using online maximum likelihood regression (MLLR) and through a periodic batch update. Using the scene graph, new trajectories can be analyzed in online fashion to categorize past and present activity, predict future behavior, and detect abnormalities.

Hierarchical and Networked Vehicle Surveillance in ITS: A Survey

Traffic surveillance has become an important topic in intelligent transportation systems (ITSs), which is aimed at monitoring and managing traffic flow. With the progress in computer vision, video-based surveillance systems have made great advances on traffic surveillance in ITSs. However, the performance of most existing surveillance systems is susceptible to challenging complex traffic scenes (e.g., object occlusion, pose variation, and cluttered background). Moreover, existing related research is mainly on a single video sensor node, which is incapable of addressing the surveillance of traffic road networks. Accordingly, we present a review of the literature on the video-based vehicle surveillance systems in ITSs. We analyze the existing challenges in video-based surveillance systems for the vehicle and present a general architecture for video surveillance systems, i.e., the hierarchical and networked vehicle surveillance, to survey the different existing and potential techniques. Then, different methods are reviewed and discussed with respect to each module. Applications and future developments are discussed to provide future needs of ITS services.

Improved Vehicle Classification in Long Traffic Video by Cooperating Tracker and Classifier Modules

Visual surveillance systems intend to extract meaning from a scene. Two initial steps for this extraction are the detection and tracking of objects followed by the classification of these objects. Often times these are viewed as separate problems where each is solved by an individual module. These tasks should not be done individually because they can help one another. This paper demonstrates the benefit gained both in tracking and classification through the communication between these individual modules. This is shown on a real-time system monitoring highway traffic. The system retreives online video at 10 frames/sec and conducts tracking and classification simultaneously. Results show an improvement from 74% to 88% accuracy in classification results.