Zhipeng Ye

Practice makes perfect

Active learning is an effective method for iteratively selecting a subset of images from an unlabeled dataset. One of the most widely used active learning strategies is uncertainty sampling. However, traditional sampling strategies do not take the category of samples into consideration, and the selected images do not reflect the desired training distribution, leading to the result that additional labeling work needs to be done. To deal with these problems, from the aspect of visual perception, we improve the traditional entropy-based uncertainty sampling strategy by introducing a certainty measurement estimated by a bag-of-visual-words (BoVW). The Rescorla-Wagner perceptive model is utilized to quantify the stop criterion. This method differs from previous approaches that treated sampling and classifying process separately: we treat the learning process as a uniform model by proposing a new evolving sample selection method that uses the unified negative-accelerated learning principle and takes category distribution into consideration. A classifier is trained to provide category distributions for the sampling process to improve its sampling performance and reduce additional annotation costs for the human annotator. During the training process, weights for both modules are adaptively initialized by the inner similarity of sample set measured by structural similarity (SSIM), and dynamically adjusted according to the learning process of the human. In addition to the regular tests that are commonly utilized by traditional sampling methods, the transfer test, based on transfer learning theory, is utilized to further evaluate the performance of different sampling strategies. Experimental results on real world datasets show that our active sampling framework outperforms both baseline and state-of-the-art adaptive active learning strategies.

May the torcher light our way: A negative-accelerated active learning framework for image classification

Uncertainty sampling is one of the most widely used strategy for pool-based active learning, however, there exists the problem that selected images do not reflect the desired training distribution and need additional labeling cost. To deal with this problem, from aspects of image classification and visual perception, we improve the traditional entropy-based sampling strategy by introducing bag-of-visual-words classification method and negative-accelerated learning principle from Rescorla-Wagner perceptive model. Differs from previous researches that treated sampling and classifying process separately, under the unified negative-accelerated learning model, we combine the two processes as a uniform model, named as negative-accelerated uncertainty sampling strategy with BoVW (NUSB) by proposing a new evolving sample selection measure, which takes category distribution into consideration. Classifier is trained to provide category distribution for the sampling process, reducing additional cost of annotation. Also, transfer test is utilized to prevent over-fitting and further evaluate the performance of different sampling strategies. Experimental results on real world datasets show that our active sampling framework outperforms both baseline active sampling strategies and state-of-the-art active learning based image classification method.

Cognition inspired framework for indoor scene annotation

Abstract. We present a simple yet effective scene annotation framework based on a combination of bag-of-visual words (BoVW), three-dimensional scene structure estimation, scene context, and cognitive theory. From a macroperspective, the proposed cognition-based hybrid motivation framework divides the annotation problem into empirical inference and real-time classification. Inspired by the inference ability of human beings, common objects of indoor scenes are defined for experience-based inference, while in the real-time classification stage, an improved BoVW-based multilayer abstract semantics labeling method is proposed by introducing abstract semantic hierarchies to narrow the semantic gap and improve the performance of object categorization. The proposed framework was evaluated on a variety of common data sets and experimental results proved its effectiveness.

Hierarchical abstract semantic model for image classification

Abstract. Semantic gap limits the performance of bag-of-visual-words. To deal with this problem, a hierarchical abstract semantics method that builds abstract semantic layers, generates semantic visual vocabularies, measures semantic gap, and constructs classifiers using the Adaboost strategy is proposed. First, abstract semantic layers are proposed to narrow the semantic gap between visual features and their interpretation. Then semantic visual words are extracted as features to train semantic classifiers. One popular form of measurement is used to quantify the semantic gap. The Adaboost training strategy is used to combine weak classifiers into strong ones to further improve performance. For a testing image, the category is estimated layer-by-layer. Corresponding abstract hierarchical structures for popular datasets, including Caltech-101 and MSRC, are proposed for evaluation. The experimental results show that the proposed method is capable of narrowing semantic gaps effectively and performs better than other categorization methods.

A hierarchical inferential method for indoor scene classification

Abstract Indoor scene classification forms a basis for scene interaction for service robots. The task is challenging because the layout and decoration of a scene vary considerably. Previous studies on knowledge-based methods commonly ignore the importance of visual attributes when constructing the knowledge base. These shortcomings restrict the performance of classification. The structure of a semantic hierarchy was proposed to describe similarities of different parts of scenes in a fine-grained way. Besides the commonly used semantic features, visual attributes were also introduced to construct the knowledge base. Inspired by the processes of human cognition and the characteristics of indoor scenes, we proposed an inferential framework based on the Markov logic network. The framework is evaluated on a popular indoor scene dataset, and the experimental results demonstrate its effectiveness.

Hierarchy-associated semantic-rule inference framework for classifying indoor scenes

Abstract. Typically, the initial task of classifying indoor scenes is challenging, because the spatial layout and decoration of a scene can vary considerably. Recent efforts at classifying object relationships commonly depend on the results of scene annotation and predefined rules, making classification inflexible. Furthermore, annotation results are easily affected by external factors. Inspired by human cognition, a scene-classification framework was proposed using the empirically based annotation (EBA) and a match-over rule-based (MRB) inference system. The semantic hierarchy of images is exploited by EBA to construct rules empirically for MRB classification. The problem of scene classification is divided into low-level annotation and high-level inference from a macro perspective. Low-level annotation involves detecting the semantic hierarchy and annotating the scene with a deformable-parts model and a bag-of-visual-words model. In high-level inference, hierarchical rules are extracted to train the decision tree for classification. The categories of testing samples are generated from the parts to the whole. Compared with traditional classification strategies, the proposed semantic hierarchy and corresponding rules reduce the effect of a variable background and improve the classification performance. The proposed framework was evaluated on a popular indoor scene dataset, and the experimental results demonstrate its effectiveness.

Precise Image Matching: A Similarity Measure Approach

An algorithm that utilizes the similarity comparison is proposed to get more proper match result, which is easy to implement. SIFT depends on principal direction which will lead to low precision rate when the direction is incorrectly computed. In this paper, similarities are tested by cosine theorem of matched points in some area to find stable matches and exclude mismatches (push) at first. Part of correct matches in excluded points are revived (pull) through stable matches, which are located in cluster sets centered by stable matched points, thus shrink search field and boosting the algorithm. Sum of Square Distance (SSD) measurement function is tested and chosen as similarity function to accomplish the reviving step. Experimental results show that the proposed method exhibits improved performance compared with SIFT and other methods.

Knowledge as action: A cognitive framework for indoor scene classification

Indoor scene classification is an important topic in computer vision, which is challenging due to the variability of decoration. Human vision system, on the other hand, is marvelous in adaptively recognizing scene categories with excellent performance and can be used for reference. Although bio-inspired computer vision algorithms have proven their effectiveness in classification applications, nowadays few researches on indoor scene classification algorithms attempt to model human vision system, restricting further improvement of performance and making it difficult to achieve adaptive scene understanding. To deal with this problem, in this paper we attempt to model the human vision system and achieve scene classification according to the cognitive theory, by dividing the problem into low-level objection annotation and high-level knowledge inference respectively on a macro perspective. Inspired by the biotical perception principle, a novel cognitive hybrid motivation framework is proposed, including empirical based annotation and inference over knowledge base, which is a simple yet effective framework based on techniques of object detection and classification. For a given indoor scene, objects of indoor scene are first annotated, then knowledge base is utilized to infer the category, reducing the effect of variable background. Environmental context is also utilized to assist classification. The proposed framework are evaluated on popular indoor scene dataset, and its effectiveness is proved by experimental results.