Piotr Szczuko

Multi-Stage Video Analysis Framework

The chapter is organized as follows. Section 2 presents the general structure of the proposed framework and a method of data exchange between system elements. Section 3 is describing the low-level analysis modules for detection and tracking of moving objects. In Section 4 we present the object classification module. Sections 5 and 6 describe specialized modules for detection and recognition of faces and license plates, respectively. In section 7 we discuss how video analysis results provided by other modules may be used for automatic detection of events related to possible security threats. The chapter ends with conclusions and discussion of future framework development.

Processing of musical data employing rough sets and artificial neural networks

This article presents experiments aiming at testing the effectiveness of the implemented low-level descriptors for automatic recognition of musical instruments and musical styles. The paper discusses first some problems in audio information analysis related to MPEG-7-based applications. A short overview of the MPEG-7 standard focused on audio information description is also given. System assumptions for automatic identification of music and musical instrument sounds are presented. A discussion on the influence of descriptor selection process on the classification accuracy is included. Experiments are carried out basing on a decision system employing Rough Sets (RS) and Artificial Neural Networks (ANNs).

Gaze-tracking-based audio-visual correlation analysis employing quality of experience methodology

This paper investigates a new approach to audio-visual correlation assessment based on the gaze-tracking system developed at the Multimedia Systems Department (MSD) of Gdansk University of Technology (GUT). The gaze-tracking methodology, having roots in Human-Computer Interaction borrows the relevance feedback through gaze-tracking and applies it to the new area of interests, which is Quality of Experience (QoE). Results of subjective tests carried out at the MSD showed a strong dependency between video presented in the screen and the perceived audio. It has also been shown that the application of gaze-tracking to the audio-visual correlation analysis allows for the objectivization of results obtained in subjective tests. Therefore this research study concentrates on the possibility to apply this methodology to the area of Quality of Experience.

Video Content Analysis in the Urban Area Telemonitoring System

Monitoring systems of urban areas comprise a large number of video cameras, occasionally with an inclusion of other sensors, such as: RFID readers, IR motion detectors, access cards and others. Dangerous events may also be accompanied with a sound cues, that, if analyzed by the monitoring system, may reveal more information than only video analysis. Therefore utilization of cameras with microphones attached is a current trend in evolution of monitoring systems. Sound analysis may provide an effective detection of screams, calling for help, breaking of a glass, explosions, gun shots, etc. Methods are developed for estimation of direction of sound arrival, therefore allowing to point a moving video monitoring camera on the sound source. Combining both techniques, audio analysis and video analysis in single monitoring system results in a great increase of the threat detection efficiency. The chapter presents various aspects of audio signals processing dedicated to detection and monitoring of threats in urban areas. The audio analysis is described in Sec. 2. It comprises detection of sounds and classification of them into 4 groups: broken glass, explosion, scream, and environmental sounds non-related to threats. Classification algorithm and results are presented in Sec. 2.1. Once in an audio stream an important event is detected, it is crucial to estimate localization of the sound source. Sec. 2.2 describes techniques for the localization of sound source utilizing set of microphones. Outcome data of localization module can be utilized to drive moving PTZ cameras correctly, as is discussed in Sec. 3.

New Method for Personalization of Avatar Animation

The paper presents a method for creating a personalized animation of avatar utilizing fuzzy inference. First the user designs a prototype version of animation, with keyframes only for important poses, roughly describing the action. Then, animation is enriched with new motion phases calculated by the fuzzy inference system using descriptors given by the user. Various degrees of motion fluency and naturalness are possible to achieve. The proposed algorithm of the animation enrichment based on fuzzy description is thoroughly presented. The first part consists of creating fuzzy rules for the algorithm using results of subjective evaluation of the animated movement, the second one utilizes input descriptors for new motion phases calculation, which are finally added to the animation. Results of subjective evaluation of obtained animations are presented.

Gaze-tracking and Acoustic Vector Sensors Technologies for PTZ Camera Steering and Acoustic Event Detection

An innovative application of gaze-tracking and acoustic vector sensors (AVS) technologies for guidance of moving pan-tilt-zoom (PTZ) monitoring camera is presented. Gaze-tracking is used to steer and to zoom the camera to the gaze focus area. Additionally, it is combined with audio processing in two scenarios. First is called “audio slave”: directional acoustic monitoring is adjusted automatically to the camera direction. Second is called “audio master”: automatic detection of sound events directions is performed to take priority over user control and steer the camera towards sound source. An approach to gaze tracking is presented, utilizing new algorithmic methods for both image processing and PTZ camera steering. Then application of AVS for directional filtering of sound, and for detection of acoustic events direction is discussed. The implemented application is described, and user experience is reported. Finally, future work is discussed.

Genetic programming extension to APF-based monocular human body pose estimation

New method of the human body pose estimation based on a single camera 2D observation is presented, aimed at smart surveillance related video analysis and action recognition. It employs 3D model of the human body, and genetic algorithm combined with annealed particle filter for searching the global optimum of model state, best matching the object’s 2D observation. Additionally, new motion cost metric is employed, considering current pose and history of the body movement, favouring the estimates with the lowest changes of motion speed comparing to previous poses. The “genetic memory” concept is introduced for the genetic processing of both current and past states of 3D model. State-of-the-art in the field of human body tracking is presented and discussed. Details of implemented method are described. Results of experimental evaluation of developed algorithm are included and discussed.

Camera sabotage detection for surveillance systems

Camera dysfunction detection algorithms and their utilization in realtime video surveillance systems are described. The purpose of using the proposed analysis is explained. Regarding image tampering three algorithms for focus loss, scene obstruction and camera displacement detection are implemented and presented. Features of each module are described and certain scenarios for best performance are depicted. Implemented solutions are evaluated as independent events and final results are discussed. A detection efficiency improvement method is proposed.

Real and imaginary motion classification based on rough set analysis of EEG signals for multimedia applications

Rough set-based approach to the classification of EEG signals of real and imaginary motion is presented. The pre-processing and signal parametrization procedures are described, the rough set theory is briefly introduced, and several classification scenarios and parameters selection methods are proposed. Classification results are provided and discussed with their potential utilization for multimedia applications controlled by the motion intent. Accuracy metrics of classification for real and imaginary motion obtained with different parameter sets are compared. Results of experiments employing recorded EEG signals are commented and further research directions are proposed.

Surveillance Camera Tracking of Geo positioned Objects

A system for tracking moving objects with a known GPS position using a set of PTZ cameras with limited areas of coverage is presented. The basic idea of the system and its possible applications are discussed. The proposed camera calibration technique is used to transform the GPS position to camera settings. The current position of the tracked object is predicted in order to compensate the transmission and processing delays. The distributed client-server system using mobile terminals and its application to tracking objects movement, with additional functionality such as showing the position of the objects on the map, is presented. The results of the tests performed in real-life conditions, as well as perspectives of a future development of the system, are discussed.