Grzegorz Szwoch

Multi-Stage Video Analysis Framework

The chapter is organized as follows. Section 2 presents the general structure of the proposed framework and a method of data exchange between system elements. Section 3 is describing the low-level analysis modules for detection and tracking of moving objects. In Section 4 we present the object classification module. Sections 5 and 6 describe specialized modules for detection and recognition of faces and license plates, respectively. In section 7 we discuss how video analysis results provided by other modules may be used for automatic detection of events related to possible security threats. The chapter ends with conclusions and discussion of future framework development.

Parallel implementation of background subtraction algorithms for real-time video processing on a supercomputer platform

Results of evaluation of the background subtraction algorithms implemented on a supercomputer platform in a parallel manner are presented in the article. The aim of the work is to chose an algorithm, a number of threads and a task scheduling method, that together provide satisfactory accuracy and efficiency of a real-time processing of high-resolution camera images, maintaining the cost of resources usage at a reasonable level. Two selected algorithms: the Gaussian mixture models and the Codebook, are presented and their computational complexity is discussed. Various approaches to the parallel implementation, including assigning the image pixels to threads, the task scheduling methods and the thread management systems, are presented. The experiments were performed on a supercomputer cluster, using a single machine with 12 physical cores. The accuracy and performance of the implemented algorithms were evaluated for varying image resolutions and numbers of concurrent processing threads. On a basis of the evaluation results, an optimal configuration for the parallel implementation of the system for real-time video content analysis on a supercomputer platform was proposed.

Extraction of stable foreground image regions for unattended luggage detection

A novel approach to detection of stationary objects in the video stream is presented. Stationary objects are these separated from the static background, but remaining motionless for a prolonged time. Extraction of stationary objects from images is useful in automatic detection of unattended luggage. The proposed algorithm is based on detection of image regions containing foreground image pixels having stable values in time and checking their correspondence with the detected moving objects. In the first stage of the algorithm, stability of individual pixels belonging to moving objects is tested using a model constructed from vectors. Next, clusters of pixels with stable color and brightness are extracted from the image and related to contours of the detected moving objects. This way, stationary (previously moving) objects are detected. False contours of objects removed from the background are also found and discarded from the analysis. The results of the algorithm may be analyzed further by the classifier, separating luggage from other objects, and the decision system for unattended luggage detection. The main focus of the paper is on the algorithm for extraction of stable image regions. However, a complete framework for unattended luggage detection is also presented in order to show that the proposed approach provides data for successful event detection. The results of experiments in which the proposed algorithm was validated using both standard datasets and video recordings from a real airport security system are presented and discussed.

Video Content Analysis in the Urban Area Telemonitoring System

Monitoring systems of urban areas comprise a large number of video cameras, occasionally with an inclusion of other sensors, such as: RFID readers, IR motion detectors, access cards and others. Dangerous events may also be accompanied with a sound cues, that, if analyzed by the monitoring system, may reveal more information than only video analysis. Therefore utilization of cameras with microphones attached is a current trend in evolution of monitoring systems. Sound analysis may provide an effective detection of screams, calling for help, breaking of a glass, explosions, gun shots, etc. Methods are developed for estimation of direction of sound arrival, therefore allowing to point a moving video monitoring camera on the sound source. Combining both techniques, audio analysis and video analysis in single monitoring system results in a great increase of the threat detection efficiency. The chapter presents various aspects of audio signals processing dedicated to detection and monitoring of threats in urban areas. The audio analysis is described in Sec. 2. It comprises detection of sounds and classification of them into 4 groups: broken glass, explosion, scream, and environmental sounds non-related to threats. Classification algorithm and results are presented in Sec. 2.1. Once in an audio stream an important event is detected, it is crucial to estimate localization of the sound source. Sec. 2.2 describes techniques for the localization of sound source utilizing set of microphones. Outcome data of localization module can be utilized to drive moving PTZ cameras correctly, as is discussed in Sec. 3.

Fast communication: A low complexity double-talk detector based on the signal envelope

A new algorithm for double-talk detection, intended for use in the acoustic echo canceller for voice communication applications, is proposed. The communication system developed by the authors required the use of a double-talk detection algorithm with low complexity and good accuracy. The authors propose an approach to double-talk detection based on the signal envelopes. For each of three signals: the far-end speech, the microphone signal and the echo estimate, an envelope is detected. Next, using these envelopes, a detection function is determined and compared to the threshold. Additionally, a dynamic threshold is introduced in order to improve the accuracy of the algorithm. The results of the simulations presented in the paper proved that the accuracy of double-talk detection obtained using the proposed algorithm is higher than in the Geigel algorithm and comparable to the correlation-based methods, while the computational complexity of the proposed method remains at an acceptable level. The double-talk detection algorithm presented here may be used in voice communication systems having limited resources, allowing for accurate double-talk detection and, as a consequence, efficient acoustic echo cancellation.

Performance Evaluation of the Parallel Codebook Algorithm for Background Subtraction in Video Stream

A background subtraction algorithm based on the codebook approach was implemented on a multi-core processor in a parallel form, using the OpenMP system. The aim of the experiments was to evaluate performance of the multithreaded algorithm in processing video streams recorded from monitoring cameras, depending on a number of computer cores used, method of task scheduling, image resolution and degree of image content variability. The results of the tests are presented and discussed. The main purpose of the research is application of the tested algorithm in a real-time video content analysis system, e.g. for automatic detection of important security threats.

Identification of regions of interest in video for a traffic monitoring system

A system for automatic event detection in the camera image is presented in this paper. A method of limiting a region of interest to relevant parts of the image using a set of processing procedures is proposed. Image processing includes object detection with shadow removal performed in the modified YCbCr color space instead of RGB. The proposed procedures help to reduce the complexity of image processing algorithm and result in extraction of objects in the image that fulfill the imposed criteria. The selected objects may be then processed by the rule-based event detection system.

Distributed Framework for Visual Event Detection in Parking Lot Area

The paper presents the framework for automatic detection of various events occurring in a parking lot basing on multiple camera video analysis. The framework is massively distributed, both in the logical and physical sense. It consists of several entities called node stations that use XMPP protocol for internal communication and SRTP protocol with Jingle extension for video streaming. Recognized events include detecting parking vehicles supplemented with parking place and vehicle identification. Event detection is based on low level image processing consisting of moving object detection, tracking and classification. Front-end of the framework is formed by the operator console that presents results of the framework accompanied with video streams from selected cameras in the real-time. Additionally, the operator console may be used to automatically aim any PTZ camera in the framework at the selected spot in video frames from fixed cameras and track moving objects as long as they move within the monitored area. The paper is concluded with the framework performance evaluation during real condition experiments.

Detection of Moving Objects in Images Combined from Video and Thermal Cameras

An algorithm for detection of moving objects in video streams from the monitoring cameras is presented. A system composed of a standard video camera and a thermal camera, mounted in close proximity to each other, is used for object detection. First, a background subtraction is performed in both video streams separately, using the popular Gaussian Mixture Models method. For the next processing stage, the authors propose an algorithm which synchronizes the video streams and performs a projective transformation of the images so that they are properly aligned. Finally, the algorithm processes the partial background subtraction results from both cameras in order to obtain a combined result, from which connected components representing moving objects may be extracted. The tests of the proposed algorithm confirm that employing the dual camera system for moving object detection improves its accuracy in difficult lighting conditions.

A Framework for Automatic Detection of Abandoned Luggage in Airport Terminal

A framework for automatic detection of events in a video stream transmitted from a monitoring system is presented. The framework is based on the widely used background subtraction and object tracking algorithms. The authors elaborated an algorithm for detection of left and removed objects based on morphological processing and edge detection. The event detection algorithm collects and analyzes data of all the moving objects in order to detect events defined by rules. A system was installed at the airport for detecting abandoned luggage. The results of the tests indicate that the system generally works as expected, but the low-level modules currently limit the system performance in some problematic conditions. The proposed solution may supplement the existing monitoring systems in order to improve the detection of security threats.