Michael Teutsch

Detection, Segmentation, and Tracking of Moving Objects in UAV Videos

Automatic processing of videos coming from small UAVs offers high potential for advanced surveillance applications but is also very challenging. These challenges include camera motion, high object distance, varying object background, multiple objects near to each other, weak signal-to-noise-ratio (SNR), or compression artifacts. In this paper, a video processing chain for detection, segmentation, and tracking of multiple moving objects is presented dealing with the mentioned challenges. The fundament is the detection of local image features, which are not stationary. By clustering these features and subsequent object segmentation, regions are generated representing object hypotheses. Multi-object tracking is introduced using a Kalman filter and considering the camera motion. Split or merged object regions are handled by fusion of the regions and the local features. Finally, a quantitative evaluation of object segmentation and tracking is provided.

Evaluation of binary keypoint descriptors

In this paper an evaluation of state-of-the-art binary keypoint descriptors, namely BRIEF, ORB, BRISK and FREAK, is presented. In contrast to previous evaluations we used the Stanford Mobile Visual Search (SMVS) data set because binary descriptors are mainly used in mobile applications. This large data set does provide a lot of characteristic transformations for mobile devices, but no ground truth data. The often used Oxford data set is used only for validation purposes. We use ratio-test and RANSAC (RANdom SAmple Consensus) for evaluation and present results for accuracy, precision and average number of best matches as performance metrics. The validity of the results is also checked by evaluating these binary keypoint descriptors on Oxford data set. The obtained results show that BRISK is the keypoint descriptor which gives highest percentage of precision and largest number of best matches among all the binary descriptors. Next to BRISK is FREAK, which offers comparably good result.

Classification of small boats in infrared images for maritime surveillance

Autonomous round-the-clock observation of wide critical maritime areas can be a powerful support for border protection agencies to avoid criminal acts like illegal immigration, piracy or drug trafficking. These criminal acts are often accomplished by using small boats to decrease the probability of being uncovered. In this paper, we present an image exploitation approach to detect and classify maritime objects in infrared image sequences recorded from an autonomous platform. We focus on high robustness and generality with respect to variations of boat appearance, image quality, and environmental condition. A fusion of three different detection algorithms is performed to create reliable alarm hypotheses. In the following, a set of well-investigated features is extracted from the alarm hypotheses and evaluated using a two-stage-classification with support vector machines (SVMs) in order to distinguish between three object classes: clutter, irrelevant objects and suspicious boats. On the given image data we achieve a rate of 97 % correct classifications.

Low Resolution Person Detection with a Moving Thermal Infrared Camera by Hot Spot Classification

In many visual surveillance applications the task of person detection and localization can be solved easier by using thermal long-wave infrared (LWIR) cameras which are less affected by changing illumination or background texture than visual-optical cameras. Especially in outdoor scenes where usually only few hot spots appear in thermal infrared imagery, humans can be detected more reliably due to their prominent infrared signature. We propose a two-stage person recognition approach for LWIR images: (1) the application of Maximally Stable Extremal Regions (MSER) to detect hot spots instead of background subtraction or sliding window and (2) the verification of the detected hot spots using a Discrete Cosine Transform (DCT) based descriptor and a modified Random Naïve Bayes (RNB) classifier. The main contributions are the novel modified RNB classifier and the generality of our method. We achieve high detection rates for several different LWIR datasets with low resolution videos in real-time. While many papers in this topic are dealing with strong constraints such as considering only one dataset, assuming a stationary camera, or detecting only moving persons, we aim at avoiding such constraints to make our approach applicable with moving platforms such as Unmanned Ground Vehicles (UGV).

Robust and fast detection of moving vehicles in aerial videos using sliding windows

The detection of vehicles driving on busy urban streets in videos acquired by airborne cameras is challenging due to the large distance between camera and vehicles, simultaneous vehicle and camera motion, shadows, or low contrast due to weak illumination. However, it is an important processing step for applications such as automatic traffic monitoring, detection of abnormal behaviour, border protection, or surveillance of restricted areas. In contrast to commonly applied object segmentation methods based on background subtraction or frame differencing, we detect moving vehicles using the combination of a track-before-detect (TBD) approach and machine learning: an AdaBoost classifier learns the appearance of vehicles in low resolution and is applied within a sliding window algorithm to detect vehicles inside a region of interest determined by the TBD approach. Our main contribution lies in the identification, optimization, and evaluation of the most important parameters to achieve both high detection rates and real-time processing.

Segmentation and classification of man-made maritime objects in TerraSAR-X images

Spaceborne monitoring of wide maritime areas can be suitable for many applications such as tracking of ship traffic, surveillance of fishery zones, or detecting criminal activities. We present novel approaches for segmentation and classification of man-made objects in TerraSAR-X images including estimation of orientation and size. This is a difficult task as detections are affected by clutter and noise effects, and each object can have different appearances. We chose a statistical approach to robustly segment given detections using Local Binary Pattern (LBP) and Histograms of Oriented Gradients (HOG). This is the fundament for subsequent feature analysis and 3-stage-classification based on Support Vector Machines (SVM) with separation of clutter and man-made objects in first, non-ships and ships in second, and different ship structure types in third stage. An experimental evaluation demonstrates the effective operation of our approaches.

Detection and classification of moving objects from UAVs with optical sensors

Small and medium sized UAVs like German LUNA have long endurance and define in combination with sophisticated image exploitation algorithms a very cost efficient platform for surveillance. At Fraunhofer IOSB, we have developed the video exploitation system ABUL with the target to meet the demands of small and medium sized UAVs. Several image exploitation algorithms such as multi-resolution, super-resolution, image stabilization, geocoded mosaiking and stereo-images/3D-models have been implemented and are used with several UAV-systems. Among these algorithms is the moving target detection with compensation of sensor motion. Moving objects are of major interest during surveillance missions, but due to movement of the sensor on the UAV and small object size in the images, it is a challenging task to develop reliable detection algorithms under the constraint of real-time demands on limited hardware resources. Based on compensation of sensor motion by fast and robust estimation of geometric transformations between images, independent motion is detected relatively to the static background. From independent motion cues, regions of interest (bounding-boxes) are generated and used as initial object hypotheses. A novel classification module is introduced to perform an appearance-based analysis of the hypotheses. Various texture features are extracted and evaluated automatically for achieving a good feature selection to successfully classify vehicles and people.

Fully Convolutional Region Proposal Networks for Multispectral Person Detection

Multispectral images that combine visual-optical (VIS) and infrared (IR) image information are a promising source of data for automatic person detection. Especially in automotive or surveillance applications, challenging conditions such as insufficient illumination or large distances between camera and object occur regularly and can affect image quality. This leads to weak image contrast or low object resolution. In order to detect persons under such conditions, we apply deep learning for effectively fusing the VIS and IR information in multispectral images. We present a novel multispectral Region Proposal Network (RPN) that is built up on the pre-trained very deep convolutional network VGG-16. The proposals of this network are further evaluated using a Boosted Decision Trees classifier in order to reduce potential false positive detections. With a log-average miss rate of 29:83% on the reasonable test set of the KAIST Multispectral Pedestrian Detection Benchmark, we improve the current state-of-the-art by about 18%.

A survey on moving object detection for wide area motion imagery

Wide Area Motion Imagery (WAMI) enables the surveillance of tens of square kilometers with one airborne sensor Each image can contain thousands of moving objects. Applications such as driver behavior analysis or traffic monitoring require precise multiple object tracking that is dependent on initial detections. However, low object resolution, dense traffic, and imprecise image alignment lead to split, merged, and missing detections. No systematic evaluation of moving object detection exists so far although many approaches have been presented in the literature. This paper provides a detailed overview of existing methods for moving object detection in WAMI data. Also we propose a novel combination of short-term background subtraction and suppression of image alignment errors by pixel neighborhood consideration. In total, eleven methods are systematically evaluated using more than 160,000 ground truth detections of the WPAFB 2009 dataset. Best performance with respect to precision and recall is achieved by the proposed one.

3D-segmentation of traffic environments with u/v-disparity supported by radar-given masterpoints

3D-segmentation of a traffic scene with two-dimensional row- and column-disparity-histograms, namely u/v-disparities, has become more and more popular for modern stereo-camera-based driver assistance systems due to its fast computation in real-time, few memory requirements and robustness against noisy or intermittent data. In this paper, we present a novel approach to support this pure vision-based method by projecting preprocessed radar-signals directly to u-disparity-space. We called the projection result “masterpoints”. This data fusion on low feature-level improved the segmentation process and increased the obstacle detection rate significantly. No assumptions about obstacle-type or -size are needed. Furthermore, the algorithms can be parallelized easily and run in real-time.