Piotr Dalka

Estimation of musical sound separation algorithm effectiveness employing neural networks

Blind separation of musical sounds contained in sound mixtures is a challenging and difficult task. It is due to the fact that in Western music, mixed harmonic sources may be correlated with each other, i.e. their harmonic partials might be overlapping in the frequency domain if the signals remain in harmonic relation. Evaluation of the separation results is also problematic, since analysis of the energy-based error between the original signals used for mixing and the separated ones, in some cases, do not correspond with perceptual evaluation results. In this paper, four separation algorithms, engineered by the Authors, are presented. Then, musical instrument sound identification based on artificial neural networks is performed as a means of evaluating the performance of the separation algorithms. Results are discussed and conclusions are derived.

Multi-Stage Video Analysis Framework

The chapter is organized as follows. Section 2 presents the general structure of the proposed framework and a method of data exchange between system elements. Section 3 is describing the low-level analysis modules for detection and tracking of moving objects. In Section 4 we present the object classification module. Sections 5 and 6 describe specialized modules for detection and recognition of faces and license plates, respectively. In section 7 we discuss how video analysis results provided by other modules may be used for automatic detection of events related to possible security threats. The chapter ends with conclusions and discussion of future framework development.

Examining Kalman Filters Applied to Tracking Objects in Motion

Kalman filters were used for establishing relations between objects moving in video frames to the real moving objects under analysis. As a result of applying some popular methods of moving objects detection, the objects were represented by rectangles. A two-dimensional colour histogram based on a chromatic space was used for each object in experiments. The objects coupling with adequate regions including the relation of many-to-many was studied experimentally employing Kalman filters. The implemented algorithm provides a part of an advanced audio-video surveillance system for security applications.

Moving Object Detection and Tracking for the Purpose of Multimodal Surveillance System in Urban Areas

Background subtraction method based on mixture of Gaussians was employed to detect all regions in a video frame denoting moving objects. Kalman filters were used for establishing relations between the regions and real moving objects in a scene and for tracking them continuously. The objects were represented by rectangles. The objects coupling with adequate regions including the relation of many-to-many was studied experimentally employing Kalman filters. The implemented algorithm provides a part of an advanced audio-video surveillance system for security applications which is described briefly in the paper.

Vehicle classification based on soft computing algorithms

Experiments and results regarding vehicle type classification are presented. Three classes of vehicles are recognized: sedans, vans and trucks. The system uses a non-calibrated traffic camera, therefore no direct vehicle dimensions are used. Various vehicle descriptors are tested, including those based on vehicle mask only and those based on vehicle images. The latter ones employ Speeded Up Robust Features (SURF) and gradient images convolved with Gabor filters. Vehicle type is recognized with various classifiers: artificial neural network, K-nearest neighbors algorithm, decision tree and random forest.

Processing of musical data employing rough sets and artificial neural networks

This article presents experiments aiming at testing the effectiveness of the implemented low-level descriptors for automatic recognition of musical instruments and musical styles. The paper discusses first some problems in audio information analysis related to MPEG-7-based applications. A short overview of the MPEG-7 standard focused on audio information description is also given. System assumptions for automatic identification of music and musical instrument sounds are presented. A discussion on the influence of descriptor selection process on the classification accuracy is included. Experiments are carried out basing on a decision system employing Rough Sets (RS) and Artificial Neural Networks (ANNs).

Video Content Analysis in the Urban Area Telemonitoring System

Monitoring systems of urban areas comprise a large number of video cameras, occasionally with an inclusion of other sensors, such as: RFID readers, IR motion detectors, access cards and others. Dangerous events may also be accompanied with a sound cues, that, if analyzed by the monitoring system, may reveal more information than only video analysis. Therefore utilization of cameras with microphones attached is a current trend in evolution of monitoring systems. Sound analysis may provide an effective detection of screams, calling for help, breaking of a glass, explosions, gun shots, etc. Methods are developed for estimation of direction of sound arrival, therefore allowing to point a moving video monitoring camera on the sound source. Combining both techniques, audio analysis and video analysis in single monitoring system results in a great increase of the threat detection efficiency. The chapter presents various aspects of audio signals processing dedicated to detection and monitoring of threats in urban areas. The audio analysis is described in Sec. 2. It comprises detection of sounds and classification of them into 4 groups: broken glass, explosion, scream, and environmental sounds non-related to threats. Classification algorithm and results are presented in Sec. 2.1. Once in an audio stream an important event is detected, it is crucial to estimate localization of the sound source. Sec. 2.2 describes techniques for the localization of sound source utilizing set of microphones. Outcome data of localization module can be utilized to drive moving PTZ cameras correctly, as is discussed in Sec. 3.

Visual lip contour detection for the purpose of speech recognition

A method for visual detection of lip contours in frontal recordings of speakers is described and evaluated. The purpose of the method is to facilitate speech recognition with visual features extracted from a mouth region. Different Active Appearance Models are employed for finding lips in video frames and for lip shape and texture statistical description. Search initialization procedure is proposed and error measure values are monitored in order to prevent the matching process from converging to a false local minimum. AAM-based visual features are applied in an experiment devoted to the static recognition of English vowels with SVM. Studies are carried out based on a database of recordings of 5 speakers of different skin colors. Results are thoroughly discussed and illustrated with figures.

Gaze-tracking and Acoustic Vector Sensors Technologies for PTZ Camera Steering and Acoustic Event Detection

An innovative application of gaze-tracking and acoustic vector sensors (AVS) technologies for guidance of moving pan-tilt-zoom (PTZ) monitoring camera is presented. Gaze-tracking is used to steer and to zoom the camera to the gaze focus area. Additionally, it is combined with audio processing in two scenarios. First is called “audio slave”: directional acoustic monitoring is adjusted automatically to the camera direction. Second is called “audio master”: automatic detection of sound events directions is performed to take priority over user control and steer the camera towards sound source. An approach to gaze tracking is presented, utilizing new algorithmic methods for both image processing and PTZ camera steering. Then application of AVS for directional filtering of sound, and for detection of acoustic events direction is discussed. The implemented application is described, and user experience is reported. Finally, future work is discussed.

Identification of regions of interest in video for a traffic monitoring system

A system for automatic event detection in the camera image is presented in this paper. A method of limiting a region of interest to relevant parts of the image using a set of processing procedures is proposed. Image processing includes object detection with shadow removal performed in the modified YCbCr color space instead of RGB. The proposed procedures help to reduce the complexity of image processing algorithm and result in extraction of objects in the image that fulfill the imposed criteria. The selected objects may be then processed by the rule-based event detection system.