Huiyan Wang

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning

Abstract Matching objects across multiple cameras with non-overlapping views is a necessary but difficult task in the wide area video surveillance. Owing to the lack of spatio-temporal information, only the visual information can be used in some scenarios, especially when the cameras are widely separated. This paper proposes a novel framework based on multi-feature fusion and incremental learning to match the objects across disjoint views in the absence of space–time cues. We first develop a competitive major feature histogram fusion representation (CMFH 1 ) to formulate the appearance model for characterizing the potentially matching objects. The appearances of the objects can change over time and hence the models should be continuously updated. We then adopt an improved incremental general multicategory support vector machine algorithm (IGMSVM 2 ) to update the appearance models online and match the objects based on a classification method. Only a small amount of samples are needed for building an accurate classification model in our method. Several tests are performed on CAVIAR, ISCAPS and VIPeR databases where the objects change significantly due to variations in the viewpoint, illumination and poses. Experimental results demonstrate the advantages of the proposed methodology in terms of computational efficiency, computation storage, and matching accuracy over that of other state-of-the-art classification-based matching approaches. The system developed in this research can be used in real-time video surveillance applications.

Recognition of Low-Resolution Logos in Vehicle Images Based on Statistical Random Sparse Distribution

Traditional image recognition approaches can achieve high performance only when the images have high resolution and superior quality. A new vehicle logo recognition (VLR) method is proposed to treat low-resolution and poor-quality images captured from urban crossings in intelligent transport system, and the proposed approach is based on statistical random sparse distribution (SRSD) feature and multiscale scanning. The SRSD feature is a novel feature representation strategy that uses the correlation between random sparsely sampled pixel pairs as an image feature and describes the distribution of a grayscale image statistically. Multiscale scanning is a creative classification algorithm that locates and classifies a logo integrally, which alleviates the effect of propagation errors in traditional methods by processing the location and classification separately. Experiments show an overall recognition rate of 97.21% for a set of 3370 vehicle images, which showed that the proposed algorithm outperforms classical VLR methods for low-resolution and inferior quality images and is very suitable for on-site supervision in ITSs.

Hybrid deep learning for automated lepidopteran insect image classification

Abstract Lepidopterans play an important role in human economy, since some of them are harmful to vegetation in agriculture and some others produce useful materials such as silks, etc. To recognize lepidopteran species correctly is very meaningful to farmers, forest workers, or even insect researchers. This study proposed a cascade architecture which combines the methods of deep convolutional neural network (DCNNs) and Supported Vector Machines (SVMs) to identify Lepidoptera species from their images. The data-set used in this study consists of 1301 Lepidoptera images from 22 species. Since the data-set is not large enough to fine-tune an end-to-end DCNN, we propose a customized solution using part of the DCNN as feature extractor, followed by SVMs as the insect classifiers. The proposed cascade architecture can achieve an accuracy of 100% with our testing data-set and it takes only about 200 ms to recognize insect species from an image, which suggests that the proposed method can be potentially used as a real time classifier for the identification of the Lepidopterans.

Pedestrian recognition in multi-camera networks using multilevel important salient feature and multicategory incremental learning

Multilevel important salient map and colour feature for robust appearance modeling.A novel multicategory incremental modeling for accurate realtime object recognition.Only few samples for building an accurate classification model.New target objects can be effectively recognized after incremental learning.Better performance in accuracy, robustness and computation efficiency. The ability to recognize pedestrians across multiple camera views is of great importance for many applications in the broad field of video surveillance. Due to the absence of the topology and calibration of distributed cameras, spatio-temporal reasoning becomes unavailable, and therefore only appearance information can be used in real-world scenarios, especially for disjoint camera views. This paper proposes a novel approach based on important salient feature and multi-category transfer incremental learning to recognize pedestrians for long-term tracking in multi-camera networks without space-time cues. An accurate and robust model can be built for pedestrian recognition using few samples. We first propose a novel multi-level important salient feature detection method (MImSF11MImSF is the abbreviation of Multi-level Important Salient Feature detection method) to formulate the appearance model. Due to environmental changes, the appearances of the pedestrians under the camera can change over time and across space, therefore the classification performance may be impaired. Hence, the appearance models should be continuously updated. We then adopt a novel object recognition multicategory incremental modeling algorithm (ORMIM22ORMIM is the abbreviation of Object Recognition Multicategory Incremental Modeling algorithm) to update the appearance model adaptively and recognize the pedestrians based on a classification approach. One of the major advantages of the proposed method is that it can identify new target objects that were never learned in the primary model while improving the matching accuracy of what has been learned. We conduct extensive experiments on CAVIAR, ISCAPS databases and our own databases where the camera views are disjoint and the appearance of objects changes significantly due to variations in the camera viewpoint, illumination, weather and poses. The experiments demonstrate that our proposed model is superior to that of existing classification-based recognition methods in terms of accuracy, robustness and computation efficiency. The developed methodology can be used in retrieval, matching and other real-time video surveillance applications.

Semantic annotation for complex video street views based on 2D3D multi-feature fusion and aggregated boosting decision forests

Accurate and efficient semantic annotation is an important but difficult step in large-scale video interpretation. This paper presents a novel framework based on 2D3D multi-feature fusion and aggregated boosting decision forest (ABDF) for semantic annotation of video street views. We first integrate the 3D and 2D features to define the appearance model for characterizing the different types of superpixels and the similarities between two adjacent superpixel blocks. We then propose the ABDF algorithm to build the weak classifier by using a modified integrated splitting strategy for decision trees. And a Markov random field is then adopted to perform global superpixel block optimization to correct the minor errors and make the boundary for semantic annotation smoother. Finally, a boosting strategy is used to aggregate the different weak decision trees into one final strong classification decision tree. The superpixel block instead of the pixel is used as the basic processing unit, thus only a small amount of features are required to build an accurate and efficient model. The experimental results demonstrate the advantages of the proposed method in terms of classification accuracy and computation efficiency over those of existing semantic segmentation methods. The proposed framework can be used in real-time video processing applications. Highlights2D and 3D superpixel features for object representation.A modified aggregated boosting decision forest for fast and accurate classification.Only a small amount of samples for building classification model.An effective tool for accurate and efficient semantic annotation of complex street views.Better performance in accuracy, robustness and computation efficiency.

Hybrid approach using map-based estimation and class-specific Hough forest for pedestrian counting and detection

The system proposed in this study deals with pedestrian counting and detection in intelligent video surveillance systems. It is a hybrid of map-based and detection-based approaches, and combines the advantages of both. After the foreground objects being segmented, the map-based module, which implicitly compensates the perspective distortion by integrally projecting the features onto a given direction, is triggered to estimate the number of pedestrians in each foreground region. Then, a class-specific Hough forest is employed to locate individuals. Experimental results have validated our strategy. The proposed map-based module has the ability of accurately estimating the count for each region. Also, the estimation can speed up the process of locating individuals by providing cues like the number of targets and the approximate size of each target. The proposed detection-based module not only locates pedestrians, but deals with enhancing the accuracy of the counting as well.

Pedestrian recognition in multi-camera networks based on deep transfer learning and feature visualization

Abstract The extensive deployment of surveillance cameras in public places, such as subway stations and shopping malls, necessitates automated visual-data processing approaches to match pedestrians across non-overlapping multiple cameras. However, due to the insufficient number of labeled training samples in real surveillance scene, it is difficult to train an effective deep neural network for cross-camera pedestrian recognition. Moreover, the cross-camera variation in viewpoint, illumination, and background makes the task even more challenging. To address these issues, in this paper we propose to transfer the parameters of a pre-trained network to our target network and then update the parameters adaptively using training samples from the target domain. More importantly, we develop new network structures that are specially tailored for cross-camera pedestrian recognition task, and implement a simple yet effective multi-level feature fusion method that yield more discriminative and robust features for pedestrian recognition. Specifically, rather than conventionally perform classification on the single-level feature of the last feature layer, we instead utilize multi-level feature by associating feature visualization with multi-level feature fusion. As another contribution, we have published our codes and extracted features to facilitate further research. Extensive experiments are conducted on WARD, PRID and MARS datasets, we show that the proposed method consistently outperforms state-of-the-arts.

Depth Map Enhancement with Interaction in 2D-to-3D Video Conversion

The demand for 3D video content is growing. Conventional 3D video creation approaches need certain devices to take the 3D videos or lots of people to do the labor-intensive depth labeling work. To reduce the manpower and time consumption, many automatic approaches has been developed to convert legacy 2D videos into 3D. However, due to the strict quality requirements in video production industry, most of the automatic conversion methods are suffered from many quality issues and could not be used in the actual production. As a result manual or semi-automatic 3D video generation approaches are still mainstream 3D video generation technologies. In our project, we took advantage of an automatic video generation method and tried to apply human-computer interactions in its process procedure [1] in the aim to find a balance between time efficiency and depth map generation quality. The novelty of the paper relies on the successful attempt on improving an automatic 3D video generation method in the angle of video and film industry.

Single image depth estimation based on convolutional neural network and sparse connected conditional random field

Abstract. Deep convolutional neural networks (DCNNs) have attracted significant interest in the computer vision community in the recent years and have exhibited high performance in resolving many computer vision problems, such as image classification. We address the pixel-level depth prediction from a single image by combining DCNN and sparse connected conditional random field (CRF). Owing to the invariance properties of DCNNs that make them suitable for high-level tasks, their outputs are generally not localized enough for detailed pixel-level regression. A multiscale DCNN and sparse connected CRF are combined to overcome this localization weakness. We have evaluated our framework using the well-known NYU V2 depth dataset, and the results show that the proposed method can improve the depth prediction accuracy both qualitatively and quantitatively, as compared to previous works. This finding shows the potential use of the proposed method in three-dimensional (3-D) modeling or 3-D video production from the given two-dimensional (2-D) images or 2-D videos.

Fine-grained bird recognition by using contour-based pose transfer

Abstract. We propose a pose transfer method for fine-grained classifications of birds that have wide variations in appearance due to different poses and subcategories. Specifically, bird pose is transferred by using Radon-transform-based contour descriptor, k-means clustering, and K nearest neighbors (KNN) classifier. During training, we clustered annotated image samples into certain poses based on their normalized part locations and used the cluster centers as their consistent part constellations for a particular pose. At the testing stage, Radon-transform-based contour descriptor is used to find the pose a sample belongs to with a KNN classifier by using cosine similarity, and normalized part constellations are transferred to the unannotated image according to the pose type. Bag-of-visual words with OpponentSIFT and color names extracted from each part and from the global image are concatenated as feature vector, which is input to support vector machine for classification. Experimental results demonstrate significant performance gains from our method on the Caltech-UCSD Birds-2011 dataset for the fine-grained bird classification task.