Kuba Lopatka

Detection and localization of selected acoustic events in acoustic field for smart surveillance applications

A method for automatic determination of position of chosen sound events such as speech signals and impulse sounds in 3-dimensional space is presented. The events are localized in the presence of sound reflections employing acoustic vector sensors. Human voice and impulsive sounds are detected using adaptive detectors based on modified peak-valley difference (PVD) parameter and sound pressure level. Localization based on signals from the multichannel acoustic vector probe is performed upon the detection. The described algorithms can be employed in surveillance systems to monitor behavior of public events participants. The results can be used to detect sound source position in real time or to calculate the spatial distribution of sound energy in the environment. Moreover, the spatial filtration can be performed to separate sounds arriving from a chosen direction.

Behavior Analysis and Dynamic Crowd Management in Video Surveillance System

A concept and practical implementation of a crowd management system which acquires input data by the set of monitoring cameras is presented. Two leading threads are considered. First concerns the crowd behavior analysis. Second thread focuses on detection of a hold-ups in the doorway. The optical flow combined with soft computing methods (neural network) is employed to evaluate the type of crowd behavior, and fuzzy logic aids detection of the hold-ups. The experiments with the behavior classification algorithm were performed employing prepared repository of typical and untypical behavior recordings. The effectiveness of the analysis was assessed by comparing algorithmic processing results to a set of prepared reference data, which provides a description of behavior type occurring in each video frame. Application of parallel image processing and influence of parallelization on achieved performance is explained. Apart from the crowd management the behavior analysis may be used in automatic surveillance system deployed in a city area.

Detection, classification and localization of acoustic events in the presence of background noise for acoustic surveillance of hazardous situations

Evaluation of sound event detection, classification and localization of hazardous acoustic events in the presence of background noise of different types and changing intensities is presented. The methods for discerning between the events being in focus and the acoustic background are introduced. The classifier, based on a Support Vector Machine algorithm, is described. The set of features and samples used for the training of the classifier are introduced. The sound source localization algorithm based on the analysis of multichannel signals from the Acoustic Vector Sensor is presented. The methods are evaluated in an experiment conducted in the anechoic chamber, in which the representative events are played together with noise of differing intensity. The results of detection, classification and localization accuracy with respect to the Signal to Noise Ratio are discussed. The results show that the recognition and localization accuracy are strongly dependent on the acoustic conditions. We also found that the engineered algorithms provide a sufficient robustness in moderately intense noise in order to be applied to practical audio-visual surveillance systems.

Acceleration of decision making in sound event recognition employing supercomputing cluster

A sound event recognition engine on a supercomputing cluster was implemented.Dedicated parallel processing framework on the supercomputer was employed.The implemented parallel processing approaches was evaluated and compared.The decision-making time of sound event recognition was assessed.It was proven that parallel processing speeds up the computations. Parallel processing of audio data streams is introduced to shorten the decision making time in hazardous sound event recognition. A supercomputing cluster environment with a framework dedicated to processing multimedia data streams in real time is used. The sound event recognition algorithms employed are based on detecting foreground events, calculating their features in short time frames, and classifying the events with Support Vector Machine. Different strategies for improving the decision time are introduced. The experiments with the presented strategies are conducted and the results are presented.

Audio-Visual Surveillance System for Application in Bank Operating Room

An audio-visual surveillance system able to detect, classify and to localize acoustic events in a bank operating room is presented. Algorithms for detection and classification of abnormal acoustic events, such as screams or gunshots are introduced. Two types of detectors are employed to detect impulsive sounds and vocal activity. A Support Vector Machine (SVM) classifier is used to discern between the different classes of acoustic events. The methods for calculating the direction of coming sound employing an acoustic vector sensor are presented. The localization is achieved by calculating the DOA (Direction of Arrival) histogram. The evaluation of the system based on experiments conducted in a real bank operating room is given. Results of sound event detection, classification and localization are given and discussed. The system proves efficient for the task of automatic surveillance of the bank operating room.

Enhanced voice user interface employing spatial filtration of signals from acoustic vector sensor

Spatial filtration of sound is introduced to enhance speech recognition accuracy in noisy conditions. An acoustic vector sensor (AVS) is employed. The signals from the AVS probe are processed in order to attenuate the surrounding noise. As a result the signal to noise ratio is increased. An experiment is featured in which speech signals are disturbed by babble noise. The signals before and after spatial filtration are processed by an automatic speech recognition (ASR) engine. It is shown that employing spatial filtration of signals from the AVS probe leads to a significant reduction in word error rate (WER) for a dictionary of 184 words.

Multimodal Audio-Visual Recognition of Traffic Events

A demonstrator of traffic events detector based on the multimodal analysis of audio and video signals is described. The subsystem is a part of smart surveillance applications. It uses surveillance cameras and microphones as the data source. The algorithms employed to the analysis of data - sound event recognition and video analytics, are explained. Results of the multimodal analysis of recordings of dangerous and passive traffic situations are presented and discussed.