Rob G. J. Wijnhoven

Color exploitation in hog-based traffic sign detection

We study traffic sign detection on a challenging large-scale real-world dataset of panoramic images. The core processing is based on the Histogram of Oriented Gradients (HOG) algorithm which is extended by incorporating color information in the feature vector. The choice of the color space has a large influence on the performance, where we have found that the CIELab and YCbCr color spaces give the best results. The use of color significantly improves the detection performance. We compare the performance of a specific and HOG algorithm, and show that HOG outperforms the specific algorithm by up to tens of percents in most cases. In addition, we propose a new iterative SVM training paradigm to deal with the large variation in background appearance. This reduces memory consumption and increases utilization of background information.

Flexible Surveillance System Architecture for Prototyping Video Content Analysis Algorithms

Many proposed video content analysis algorithms for surveillance applications are very computationally intensive, which limits the integration in a total system, running on one processing unit (e.g. PC). To build flexible prototyping systems of low cost, a distributed system with scalable processing power is therefore required. This paper discusses requirements for surveillance systems, considering two example applications. From these requirements, specifications for a prototyping architecture are derived. An implementation of the proposed architecture is presented, enabling mapping of multiple software modules onto a number of processing units (PCs). The architecture enables fast prototyping of new algorithms for complex surveillance applications without considering resource constraints.

Context modeling combined with motion analysis for moving ship detection in port surveillance

Abstract. In port surveillance, video-based monitoring is a valuable supplement to a radar system by helping to detect smaller ships in the shadow of a larger ship and with the possibility to detect nonmetal ships. Therefore, automatic video-based ship detection is an important research area for security control in port regions. An approach that automatically detects moving ships in port surveillance videos with robustness for occlusions is presented. In our approach, important elements from the visual, spatial, and temporal features of the scene are used to create a model of the contextual information and perform a motion saliency analysis. We model the context of the scene by first segmenting the video frame and contextually labeling the segments, such as water, vegetation, etc. Then, based on the assumption that each object has its own motion, labeled segments are merged into individual semantic regions even when occlusions occur. The context is finally modeled to help locating the candidate ships by exploring semantic relations between ships and context, spatial adjacency and size constraints of different regions. Additionally, we assume that the ship moves with a significant speed compared to its surroundings. As a result, ships are detected by checking motion saliency for candidate ships according to the predefined criteria. We compare this approach with the conventional technique for object classification based on support vector machine. Experiments are carried out with real-life surveillance videos, where the obtained results outperform two recent algorithms and show the accuracy and robustness of the proposed ship detection approach. The inherent simplicity of our algorithmic subsystems enables real-time operation of our proposal in embedded video surveillance, such as port surveillance systems based on moving, nonstatic cameras.

Performance evaluation of real-time video content analysis systems in the CANDELA project

The CANDELA project aims at realizing a system for real-time image processing in traffic and surveillance applications. The system performs segmentation, labels the extracted blobs and tracks their movements in the scene. Performance evaluation of such a system is a major challenge since no standard methods exist and the criteria for evaluation are highly subjective. This paper proposes a performance evaluation approach for video content analysis (VCA) systems and identifies the involved research areas. For these areas we give an overview of the state-of-the-art in performance evaluation and introduce a classification into different semantic levels. The proposed evaluation approach compares the results of the VCA algorithm with a ground-truth (GT) counterpart, which contains the desired results. Both the VCA results and the ground truth comprise description files that are formatted in MPEG-7. The evaluation is required to provide an objective performance measure and a mean to choose between competitive methods. In addition, it enables algorithm developers to measure the progress of their work at the different levels in the design process. From these requirements and the state-of-the-art overview we conclude that standardization is highly desirable for which many research topics still need to be addressed.

Robust automatic ship tracking in harbours using active cameras

Radar is commonly used to detect and track ships in maritime surveillance. Unfortunately the systems are costly and do not provide any visual information about the objects type. To complement the ship identity information given by a radar system, we propose a supplementary system using active visual cameras that can robustly detect and track ships in harbours. By combining a high-quality, non real-time robust object detector with a feature point tracker with low computational complexity, it is possible to track ships in real time over long intervals and large distances. In addition to controlling pan and tilt, we dynamically control camera zoom to provide a high resolution image of the tracked object over a large range of distances. The tracking system is improved by a special motion estimation model for the feature points, which also incorporates zooming of the camera. The system is robust and sustains tracking even under challenging conditions, such as multiple viewpoints, a large variety of ships and various weather conditions. During experiments, various types of ships were successfully tracked for up to 18 minutes, and over a distance of almost 1.5km in the port of Rotterdam. The proposed system is generic and can be utilized in various tracking applications, by training the detector for a different object class.

Water region detection supporting ship identification in port surveillance

In this paper, we present a robust and accurate water region detection technique developed for supporting ship identification. Due to the varying appearance of water body and frequent intrusion of ships, a region-based recognition is proposed. We segment the image into perceptually meaningful segments and find all water segments using a sampling-based Support Vector Machine (SVM). The algorithm is tested on 6 different port surveillance sequences and achieves a pixel classification recall of 97.5% and precision of 96.4%. We also apply our water region detection to support the task of multiple ship detection. Combined with our cabin detector, it successfully removes 74.6% false detections generated in the cabin detection process. A slight decrease of 5% in the recall value is compensated by a significant improvement of 15% in precision.

Comparing feature matching for object categorization in video surveillance

In this paper we consider an object categorization system using local HMAX features. Two feature matching techniques are compared: the MAX technique, originally proposed in the HMAX framework, and the histogram technique originating from Bag-of-Words literature. We have found that each of these techniques have their own field of operation. The histogram technique clearly outperforms the MAX technique with 5–15% for small dictionaries up to 500–1,000 features, favoring this technique for embedded (surveillance) applications. Additionally, we have evaluated the influence of interest point operators in the system. A first experiment analyzes the effect of dictionary creation and has showed that random dictionaries outperform dictionaries created from Hessian-Laplace points. Secondly, the effect of operators in the dictionary matching stage has been evaluated. Processing all image points outperforms the point selection from the Hessian-Laplace operator.

Gender classification in low-resolution surveillance video: in-depth comparison of random forests and SVMs

This research considers gender classification in surveillance environments, typically involving low-resolution images and a large amount of viewpoint variations and occlusions. Gender classification is inherently difficult due to the large intra-class variation and interclass correlation. We have developed a gender classification system, which is successfully evaluated on two novel datasets, which realistically consider the above conditions, typical for surveillance. The system reaches a mean accuracy of up to 90% and approaches our human baseline of 92.6%, proving a high-quality gender classification system. We also present an in-depth discussion of the fundamental differences between SVM and RF classifiers. We conclude that balancing the degree of randomization in any classifier is required for the highest classification accuracy. For our problem, an RF-SVM hybrid classifier exploiting the combination of HSV and LBP features results in the highest classification accuracy of 89.9 0.2%, while classification computation time is negligible compared to the detection time of pedestrians.

Multi-channel video streaming server for surveillance systems

In this paper we describe an architecture for a multi-client, multi-channel video streaming server targeted at the security market. To obtain scalability in bit-rate, multiple compressed video streams are available for each video channel. For transmission to the user, one stream has to be selected for each video channel. Parameters and cost functions to derive the optimal set of streams are defined and heuristic algorithms to find the best combination of video streams are introduced.