Lin Lei

Vehicle Detection in Aerial Images Based on Region Convolutional Neural Networks and Hard Negative Example Mining

Detecting vehicles in aerial imagery plays an important role in a wide range of applications. The current vehicle detection methods are mostly based on sliding-window search and handcrafted or shallow-learning-based features, having limited description capability and heavy computational costs. Recently, due to the powerful feature representations, region convolutional neural networks (CNN) based detection methods have achieved state-of-the-art performance in computer vision, especially Faster R-CNN. However, directly using it for vehicle detection in aerial images has many limitations: (1) region proposal network (RPN) in Faster R-CNN has poor performance for accurately locating small-sized vehicles, due to the relatively coarse feature maps; and (2) the classifier after RPN cannot distinguish vehicles and complex backgrounds well. In this study, an improved detection method based on Faster R-CNN is proposed in order to accomplish the two challenges mentioned above. Firstly, to improve the recall, we employ a hyper region proposal network (HRPN) to extract vehicle-like targets with a combination of hierarchical feature maps. Then, we replace the classifier after RPN by a cascade of boosted classifiers to verify the candidate regions, aiming at reducing false detection by negative example mining. We evaluate our method on the Munich vehicle dataset and the collected vehicle dataset, with improvements in accuracy and robustness compared to existing methods.

Point-pattern matching based on point pair local topology and probabilistic relaxation labeling

This paper presents a robust point-pattern matching (PPM) algorithm, in which the invariant feature and probabilistic relaxation labeling are combined to improve the assignment accuracy and efficiency. A local feature descriptor, namely, point pair local topology (PPLT), is proposed first. The feature descriptor is defined by histogram which is constructed using the weighting of distance measures and angle measures based on local point pair. We use the matching scores of point pair local topology descriptor’s statistic test to define new compatibility coefficients. Then, the robust support functions are constructed based on the obtained compatibility coefficients. Finally, according to the relaxed iterations of matching probability matrix and the mapping constraints required by the bijective correspondence, the correct matching results are obtained. A number of comparison and evaluation experiments on both synthetic point sets and real-world data demonstrate that the proposed algorithm performs better in the presence of outliers and positional jitter. In addition, it achieves the superior performance under similarity and even nonrigid transformation among point sets in the meantime compared with state-of-the-art approaches.

Arbitrary-Oriented Vehicle Detection in Aerial Imagery with Single Convolutional Neural Networks

Vehicle detection with orientation estimation in aerial images has received widespread interest as it is important for intelligent traffic management. This is a challenging task, not only because of the complex background and relatively small size of the target, but also the various orientations of vehicles in aerial images captured from the top view. The existing methods for oriented vehicle detection need several post-processing steps to generate final detection results with orientation, which are not efficient enough. Moreover, they can only get discrete orientation information for each target. In this paper, we present an end-to-end single convolutional neural network to generate arbitrarily-oriented detection results directly. Our approach, named Oriented_SSD (Single Shot MultiBox Detector, SSD), uses a set of default boxes with various scales on each feature map location to produce detection bounding boxes. Meanwhile, offsets are predicted for each default box to better match the object shape, which contain the angle parameter for oriented bounding boxes’ generation. Evaluation results on the public DLR Vehicle Aerial dataset and Vehicle Detection in Aerial Imagery (VEDAI) dataset demonstrate that our method can detect both the location and orientation of the vehicle with high accuracy and fast speed. For test images in the DLR Vehicle Aerial dataset with a size of 5616 × 3744 , our method achieves 76.1% average precision (AP) and 78.7% correct direction classification at 5.17 s on an NVIDIA GTX-1060.

Multi-temporal SAR images change detection based on nonsubsampled contourlet transform

A novel change detection method for multi-temporal SAR images using nonsubsampled contourlet transform (NSCT) has been proposed. Firstly, we obtain the log-ratio map by comparing two co-registered SAR images with a ratio operator in a logarithmic scale. Secondly, the log-ratio map is decomposed into multiscale and multidirectional sub-images by NSCT, and the speckle noise in each sub-image is suppressed by scale correlation method. Finally, the changes in every sub-image are detected through a constant false alarm rate (CFAR) detector and the result map is derived according to an adaptive fusion strategy. Experimental results on simulated data and real airborne SAR imagery data both confirm the effectiveness of the proposed method.

A robust non-rigid point set registration method based on inhomogeneous Gaussian mixture models

In this paper, we propose a novel robust non-rigid point set registration method adopting a new probability model called inhomogeneous Gaussian mixture models (IGMM), where we regard one point set as the centroids of a Gaussian mixture model and the other point set as the data. The IGMM is defined by applying local features and Gaussian mixture models. Considering the local relationship among neighboring points is stable, a neighborhood structural descriptor, named as local shape context, is first presented. On the basis of local descriptors, we can obtain a measure of compatibility between local features in the point sets. Then, the similarity of the local structure of point neighborhoods can be calculated on the basis of the matching scores. Each Gaussian mixture component is assigned a different weight depending on the feature similarity, which differs from the traditional Gaussian mixture model where each Gaussian mixture component has the same weight. The proposed IGMM makes point pairs with more similar features have bigger probability to formulate a match, while in algorithms based on GMMs, all point pairs have the same probability to construct correspondence points. Finally, we support our claims through regularization theory and formulate registration as a likelihood maximization problem, which is solved by updating transformation parameters and outlier ratios using the expectation maximization algorithm. Extensive comparison and evaluation experiments on synthetic point-sets datasets demonstrate that the proposed approach is robust and achieves superior performance in the presence of non-rigid deformation, noise, outliers and occlusion. In addition, a number of experiments on real images reveal that our proposed algorithm is more applicable than state-of-the-art algorithms.

Appearance and structural motion context for multi-target tracking in aerial video

For multi-object tracking in aerial videos which are acquired from moving cameras, the motion of cars are complicated by global camera movements and always unpredictable. To deal with such unexpected camera motion for online multi-vehicles tracking, structural motion context between objects has been used thanks to its robustness to camera motion. In this paper, we propose an effective data association method that exploits structural motion context in the presence of large camera motion. In addition, to further improve the robustness of algorithm against missing data due to the target being occluded behind other objects, an appearance context model is developed to represent appearance information of objects we need to track. The structural motion context and appearance context are then used to predict the location of the unobserved objects. Experimental results on VIVID datasets show the effectiveness of the proposed algorithm for multi-object tracking.

An enhanced deep convolutional neural network for densely packed objects detection in remote sensing images

Faster Region based convolutional neural networks (FRCN) has shown great success in object detection in recent years. However, its performance will degrade on densely packed objects in real remote sensing applications. To address this problem, an enhanced deep CNN based method is developed in this paper. Following the common pipeline of “CNN feature extraction + region proposal + Region classification”, our method is primarily based on the latest Residual Networks (ResNets) and consists of two sub-networks: an object proposal network and an object detection network. For detecting densely packed objects, the output of multi-scale layers are combined together to enhance the resolution of the feature maps. Our method is trained on the VHR-10 data set with limited samples and successfully tested on large-scale google earth images, such as aircraft boneyard or tank farm, containing a substantial number of densely packed objects.

Fast vehicle detection in UAV images

Fast and accurate vehicle detection in unmanned aerial vehicle (UAV) images remains a challenge, due to its very high spatial resolution and very few annotations. Although numerous vehicle detection methods exist, most of them cannot achieve real-time detection for different scenes. Recently, deep learning algorithms has achieved fantastic detection performance in computer vision, especially regression based convolutional neural networks YOLOv2. Its good both at accuracy and speed, outperforming other state-of-the-art detection methods. This paper for the first time aims to investigate the use of YOLOv2 for vehicle detection in UAV images, as well as to explore the new method for data annotation. Our method starts with image annotation and data augmentation. CSK tracking method is used to help annotate vehicles in images captured from simple scenes. Subsequently, a regression based single convolutional neural network YOLOv2 is used to detect vehicles in UAV images. To evaluate our method, UAV video images were taken over several urban areas, and experiments were conducted on this dataset and Stanford Drone dataset. The experimental results have proven that our data preparation strategy is useful, and YOLOv2 is effective for real-time vehicle detection of UAV video images.

A point matching algorithm based on reference point pair

Outliers and occlusions are important degradation in the real application of point matching. In this paper, a novel point matching algorithm based on the reference point pairs is proposed. In each iteration, it firstly eliminates the dubious matches to obtain the relatively accurate matching points (reference point pairs), and then calculates the shape contexts of the removed points with reference to them. After re-matching the removed points, the reference point pairs are combined to achieve better correspondences. Experiments on synthetic data validate the advantages of our method in comparison with some classical methods.

A novel scheme for automatic nonrigid image registration using deformation invariant feature and geometric constraint

Automatic image registration is a vital yet challenging task, particularly for non-rigid deformation images which are more complicated and common in remote sensing images, such as distorted UAV (unmanned aerial vehicle) images or scanning imaging images caused by flutter. Traditional non-rigid image registration methods are based on the correctly matched corresponding landmarks, which usually needs artificial markers. It is a rather challenging task to locate the accurate position of the points and get accurate homonymy point sets. In this paper, we proposed an automatic non-rigid image registration algorithm which mainly consists of three steps: To begin with, we introduce an automatic feature point extraction method based on non-linear scale space and uniform distribution strategy to extract the points which are uniform distributed along the edge of the image. Next, we propose a hybrid point matching algorithm using DaLI (Deformation and Light Invariant) descriptor and local affine invariant geometric constraint based on triangulation which is constructed by K-nearest neighbor algorithm. Based on the accurate homonymy point sets, the two images are registrated by the model of TPS (Thin Plate Spline). Our method is demonstrated by three deliberately designed experiments. The first two experiments are designed to evaluate the distribution of point set and the correctly matching rate on synthetic data and real data respectively. The last experiment is designed on the non-rigid deformation remote sensing images and the three experimental results demonstrate the accuracy, robustness, and efficiency of the proposed algorithm compared with other traditional methods.