Chiao-fe Shu

Knowledge-guided parsing in video databases

Visual information systems require a new insertion process. Prior to storage within the database, the system must first identify the desired objects (shots and episodes), and then calculate a descriptive representation of these objects. This paper discusses the steps in the insertion process, and some of the tools we have developed to semi-automatically segment the data into domain objects which are meaningful to the user. Image processing routines are necessary to derive features of the video frames. Models are required to represent the desired domain, and similarity measures must compare the models to the derived features.

Virage video engine

The temporal and multi-modal nature of video increases the dimensionality of content based retrieval problem. This places new demands on the indexing and retrieval tools required. The Virage Video Engine (VVE) with the default set of primitives provide the necessary frame work and basic tools for video content based retrieval. The video engine is a flexible platform independent architecture which provides support for processing multiple synchronized data streams like image sequences, audio and closed captions. The architecture allows for multi-modal indexing and retrieval of video through the use of media specific primitives. This paper presents the use of the VVE framework for content based video retrieval.

IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework

The increasing need for sophisticated surveillance systems and the move to a digital infrastructure has transformed surveillance into a large scale data analysis and management challenge. Smart surveillance systems use automatic image understanding techniques to extract information from the surveillance data. While the majority of the research and commercial systems have focused on the information extraction aspect of the challenge, very few systems have explored the use of extracted information in the search, retrieval, data management and investigation context. The IBM smart surveillance system (S3) is one of the few advanced surveillance systems which provides not only the capability to automatically monitor a scene but also the capability to manage the surveillance data, perform event based retrieval, receive real time event alerts thru standard web infrastructure and extract long term statistical patterns of activity. The IBM S3 is easily customized to fit the requirements of different applications by using an open-standards based architecture for surveillance.

IBM smart surveillance system (S3): a open and extensible framework for event based surveillance

As smart surveillance technology becomes a critical component in security infrastructures, the system architecture assumes a critical importance. This paper considers the example of smart surveillance in an airport environment. We start with a threat model for airports and use this to derive the security requirements. These requirements are used to motivate an open-standards based architecture for surveillance. We discuss the critical aspects of this architecture and its implementation in the IBM S3 smart surveillance system. Demo results from a pilot deployment in Hawthorne, NY are presented.

Vector field analysis for oriented patterns

Presents a method, based on the properties of vector fields, for the estimation of a set of symbolic descriptors (node, saddle, star-node, improper-node, center, and spiral) from linear orientation fields. Planar first-order phase portraits are used to model the linear orientation fields. A weighted linear estimator is developed to estimate linear phase portraits, using only the flow orientation. A classification scheme for planar first-order phase portraits, based on their local properties: curl, divergence, and deformation is developed. The authors present results of experiments on noise-added synthetic flow patterns and real oriented textures. >

Video analytics for retail

We describe a set of tools for retail analytics based on a combination of video understanding and transaction-log. Tools are provided for loss prevention (returns fraud and cashier fraud), store operations (customer counting) and merchandising (display effectiveness). Results are presented on returns fraud and customer counting.

S3-R1: the IBM smart surveillance system-release 1

Summary form only given. One of the key components of tele-presence systems is automatic awareness of the remote environment. This very same capability of automatic situation awareness is currently being developed and deployed in the context of the next generation smart surveillance systems. Smart surveillance systems use a number of automatic video analysis techniques like object detection, tracking and classification in conjunction with database and Web application servers to provide users with the capability of distributed smart surveillance. The IBM smart surveillance system is one of the few advanced surveillance systems which provides not only the capability to automatically monitor a scene but also the capability to manage the surveillance data, perform event based retrieval, receive real time event alerts thru standard web infrastructure and extract long term statistical patterns of activity.

Video Analytics in Urban Environments

Urban environments present unique challenges from the perspective of surveillance and security. Threat activity in urban environments tends to be very similar to background activity, while the volume of activity is often very high. The widespread geographical area presents issues from the perspective of response. These characteristics of urban environments create challenges to traditional applications of video analytics technologies and opens up opportunities for novel approaches. This paper explores the applicability of video analytics in various scenarios presented in urban surveillance situations. We also describe novel technical solutions to some of the challenges of urban surveillance.

S3: The IBM Smart Surveillance System: From Transactional Systems to Observational Systems

Pervasive sensor based systems are transforming Information Technology systems from being transactional in nature to being observational in nature. Observational systems are inherently distributed and capture information at a much finer grain of space and time. Enabling and building such systems also poses many technology challenges, extracting information from sensor signals, indexing and searching sensor meta-data, data mining and scalability. In this paper we use S3: the IBM smart surveillance system as an example of an observational system to explore several of these issues through real world deployment examples.

A linear algorithm for computing the phase portraits of oriented textures

Phase portraits are a powerful mathematical model for describing oriented textures. An isotangent-based approach is presented which is a linear formulation to the problem, to locate the critical points and compute the parameter sets of this model for the nonsingular two-dimensional first-order phase portraits. The authors classify flow patterns by Jordan canonical forms of the characteristic matrix made up of the estimated parameters. For these systems, they prove that all the isotangent curves are straight lines which intersect at a critical point. They also apply least median of squares (LMS) estimators to find the isotangent lines and locate the critical point. A linear regression technique is used to estimate the parameters of the two-dimensional first-order phase portrait of a given flow pattern. Results of applying the algorithm to synthetic and real images are presented.<<ETX>>