Chunping Liu

An Improved Adaptive Document Image Binarization Method

Document image binarization is the basis of Optical Character Recognition (OCR). For B.Gatos’ adaptive binarization method exists the shortcomings, we propose an improved adaptive document image binarization method which consists of four steps. The first step is dedicated to a denoising procedure using a low-pass Wiener filter based on local statistics. In the second step, we use a first rough estimation of foreground regions using binarization method based on the Laplacian-Gauss algorithm. As a third step, we compute the background surface of the image by interpolating neighboring background intensities into the foreground areas that result from the previous step. In the fourth step, we proceed to the final binarization by combining information from the calculated background surface and the original image, and accomplish final binary. The method has good robustness for uneven illumination, using the algorithm extract foreground regions, it will get fewer lost strokes and can be effective to retain the edge information. The experimental results show that the improved method has better characteristics than other four kinds of typical document image binarization methods. Keywords-Local adaptive binarization; LaplacianGauss; Uneven illumination document images

Person re-identification by improved Local Maximal Occurrence with color names

Person re-identification is the task of associating people across cameras with non-overlapping view field. Two key aspects of Person re-identification are the feature representation and metric learning. The feature representation employed should be both discriminative and invariant, which is also our considering in this paper. To enhance person re-identification performance, we propose to combine improved Local Maximal Occurrence (LOMO) descriptor with semantic color names (SCN). Especially, we introduce symmetry information of human body to suppress the impact of background in LOMO. When fused with mid-level attribute-based description - sematic color names, our more discriminative signature is obtained. Based on the KISS metric, evaluation on the challenging VIPeR dataset shows that the proposed method improves the re-identification significantly.

Image and Video Segmentation

This chapter is devoted to some segmentation method of image and video. For image segmentation, five types of methods are detailed, including threshold segmentation, region-based segmentation, partial differential equation based segmentation, clustering based segmentation, and the graph theory based segmentation. For video segmentation, we shall introduce the motion region extraction method based on cumulative difference.

Dynamic Scene Classification Based on Topic Models

This chapter briefly introduces the background reading of scene classification and two topic models: LDA model and Topic Model using Belief Propagation (TMBP).

Visual Object Tracking

Moving object tracking is to find out the candidate object region which is the most similar area in the image sequence through the effective expression of the object, that is to locate the target in the sequence image so as to obtain the complete motion trajectory of the moving target. In this chapter, we first introduce the moving object detection method in static background. We also present the Adaptive background modeling method by using a mixture Gaussians. In the next three sections, there are three methods for object tracking: Ransac, Meanshift and Particle Filter. In the last section, we introduce the multi-object tracking method.

Image Understanding-Person Re-identification

In this chapter, we talk about one of the typical image understanding problems—cross-camera person re-identification. Some classical visual descriptors and metric learning algorithms for person re-identification are detailed.

Image and Video Understanding Based on Deep Learning

In this chapter we firstly introduce the development and the main reasons of the success of deep learning, then the structure and principle of the deep CNN are explored, and several classical convolution network models are analyzed, finally two instances based on CNN architecture are given.

Multi-decoder Based Co-attention for Image Captioning

Recently image caption has gained increasing attention in artificial intelligence. Existing image captioning models typically adopt visual mechanism only once to capture the related region maps, which is difficult to attend the regions relevant to each generated word effectively. In this paper, we propose a novel multi-decoder based co-attention framework for image captioning, which is composed of multiple decoders that integrate the detection-based mechanism and free-form region based attention mechanism. Our proposed approach effectively produce more precise caption by co-attending the free-form regions and detections. Particularly, given the “Teacher-Forcing”, which leads to a mismatch between training and testing, and exposure bias, we use a reinforcement learning approach to optimize. The proposed method is evaluated on the benchmark MSCOCO dataset, and achieves state-of-the-art performance.

Text to Region: Visual-Word Guided Saliency Detection.

Image/video captioning based on neural network can generate accurate description. But how to convert visual information into natural language representation is a true enigma. Existing caption-guided saliency methods take the entire sentence as input to generate a saliency map, which exposes the region-to-word mapping. However, visual information is not related to every word in caption. We eliminate these meaningless stop words such as ‘the’, ‘of’ to avoid misleading. We also utilize MFB (Multi-modal Factorized Bilinear Pooling) to fuse C3D features, which could provide richer spatiotemporal information to exposure visual-word guided saliency. Such the system produces better spatiotemporal heatmaps for both predicted captions and arbitrary query sentences without introducing attentional layers. The experimental results on MSR-VTT and Flickr30K dataset surpasses the state-of-the-art by a significant margin.

Image categorization using a semantic hierarchy model with sparse set of salient regions

Image categorization in massive image database is an important problem. This paper proposes an approach for image categorization, using sparse set of salient semantic information and hierarchy semantic label tree (HSLT) model. First, to provide more critical image semantics, the proposed sparse set of salient regions only at the focuses of visual attention instead of the entire scene was formed by our proposed saliency detection model with incorporating low and high level feature and Shotton’s semantic texton forests (STFs) method. Second, we also propose a new HSLT model in terms of the sparse regional semantic information to automatically build a semantic image hierarchy, which explicitly encodes a general to specific image relationship. And last, we archived image dataset using image hierarchical semantic, which is help to improve the performance of image organizing and browsing. Extension experimental results showed that the use of semantic hierarchies as a hierarchical organizing framework provides a better image annotation and organization, improves the accuracy and reduces human’s effort.