Russell P. Bobbitt

Recognition of repetitive sequential human activity

We present a novel framework for recognizing repetitive sequential events performed by human actors with strong temporal dependencies and potential parallel overlap. Our solution incorporates sub-event (or primitive) detectors and a spatiotemporal model for sequential event changes. We develop an effective and efficient method to integrate primitives into a set of sequential events where strong temporal constraints are imposed on the ordering of the primitives. In particular, the combination process is approached as an optimization problem. A specialized Viterbi algorithm is designed to learn and infer the target sequential events and handle the event overlap simultaneously. To demonstrate the effectiveness of the proposed framework, we report detailed quantitative analysis on a large set of cashier checkout activities in a retail store.

Video Analytics in Urban Environments

Urban environments present unique challenges from the perspective of surveillance and security. Threat activity in urban environments tends to be very similar to background activity, while the volume of activity is often very high. The widespread geographical area presents issues from the perspective of response. These characteristics of urban environments create challenges to traditional applications of video analytics technologies and opens up opportunities for novel approaches. This paper explores the applicability of video analytics in various scenarios presented in urban surveillance situations. We also describe novel technical solutions to some of the challenges of urban surveillance.

Detecting sweethearting in retail surveillance videos

A significant portion of retail shrink is attributed to employees and occurs around the point of sale (POS). In this paper, we target a major type of retail fraud in surveillance videos, known as sweethearting (or fake scan), where a cashier intentionally fails to enter one or more items into the transaction in an attempt to get free merchandise for the customer. We first develop a motion-based algorithm to identify video segments as candidates for primitive events at the POS. We then apply spatio-temporal features to recognize true primitive events from the candidates and prune those falsely alarmed. In particular, we learn location-aware event models by Multiple-Instance Learning to address the location-sensitive issues that appear in our problem. Finally, we validate the entire transaction by combining primitive events according to temporal ordering constraints. We demonstrate the effectiveness of our approach on data captured from a real grocery store.

Fast detection of retail fraud using polar touch buttons

Video analytics have recently emerged as a promising technique of retail fraud detection for loss prevention. Efficient video analytic algorithms are highly desired for a practical fraud detection system. In this paper, we present a real-time algorithm for recognizing a cashiers actions at the Point of Sale (POS), which can be further used to analyze cashier behaviors for identifying fraudulent incidents. The algorithm uses a set of simple but effective features derived from a global representation of motion energy called Polar Motion Map (PMM). These features capture the motion patterns exhibited in a cashiers actions as a focused beam of motion energy, characterizing the actions as the extension and retraction movement of the cashiers arm with respect to a prespecified region. Our algorithm demonstrates comparable accuracy against one of the state-of-the-art event recognition techniques [1] while running significantly faster.

Multi-media compliance: A practical paradigm for managing business integrity

In virtually every business context there is a need to establish some form of monitoring system to ensure that employees comply with business processes and policies. Compliance failures range from organized theft to gaps in procedure that can be easily remedied through retraining. It is clearly important for businesses to capture and record these deviations to minimize loss prevention and maximize workplace safety and efficiency. In this workshop, we discuss the growing problem of compliance failure and how our system addresses this problem in a retail context to detect checkout-related fraud through the integration of visual and non-visual data.

Efficient 24/7 object detection in surveillance videos

We address the problem of 24/7 object detection in urban surveillance videos, which presents unique challenges due to significant object appearance variations caused by lighting effects such as shadows and specular reflections, object pose variation, multiple weather conditions, and different times of the day. Rather than training a generic detector and adapting its parameters over time to handle all these variations, we rely on a large set of complementary and extremely efficient detector models, covering multiple overlapping appearance subspaces. At run time, our method continuously selects the most suitable detectors for a given scene and condition, using a novel approach inspired by parametric background modeling algorithms. We provide a comprehensive experimental analysis to show the effectiveness of our approach, considering traffic monitoring as our application domain. Our system runs at 100 frames per second on a standard laptop computer.

Composite Event Detection in Multi-Camera and Multi-Sensor Surveillance Networks

Given the rapid development of digital video technologies, large-scale multi-camera networks are now more prevalent than ever. There is an increasing demand for automated multi-camera/sensor event-modeling technologies that can efficiently and effectively extract events and activities occurring in the surveillance network. In this chapter, we present a composite event detection system for multicamera networks. The proposed framework is capable of handling relationships between primitive events generated from (1) a single camera view, (2) multiple camera views, and (3) nonvideo sensors with spatial and temporal variations. Composite events are represented in the form of full binary trees, where the leaf nodes represent primitive events, the root node represents the target composite event, and the middle nodes represent rule definitions. The multi-layer design of composite events provides great extensibility and flexibility in different applications. A standardized XML-style event language is designed to describe the composite events such that inter-agent communication and event detection module construction can be conveniently achieved. In our system, a set of graphical interfaces is also developed for users to easily define both primitive and high-level composite events. The proposed system is designed in distributed form, where the system components can be deployed on separate processors, communicating with each other over a network. The capabilities and effectiveness of our system have been demonstrated in several real-life applications, including retail loss prevention, indoor tailgating detection, and false positive reduction.