Maheen Bakhtyar

Shot boundary detection from videos using entropy and local descriptor

Video shot segmentation is an important step in key frame selection, video copy detection, video summarization, and video indexing for retrieval. Although some types of video data, e.g., live sports coverage, have abrupt shot boundaries that are easy to identify using simple heuristics, it is much more difficult to identify shot boundaries in other types such as cinematic movies. We propose an algorithm for shot boundary detection able to accurately identify not only abrupt shot boundaries, but also the fade-in and fade-out boundaries typical of cinematic movies. The algorithm is based on analysis of changes in the entropy of the gray scale intensity over consecutive frames and analysis of correspondences between SURF features over consecutive frames. In an experimental evaluation on the TRECVID-2007 shot boundary test set, the algorithm achieves substantial improvements over state of the art methods, with a precision of 97.8% and a recall of 99.3%.

Video segmentation into scenes using entropy and SURF

In this paper we present a framework for video segmentation into scenes. Segmenting videos into scenes is the basic step for video analysis, efficient video indexing and content-based video retrieval. In our framework we used frame entropy and SURF descriptor to find shot boundaries from the videos. We extracted key frames from each shot and segmented the video into semantic scenes by key frame matching. The proposed algorithm gave promising results when applied to different genres of videos and dramas.

Q-CSLBP: compression of CSLBP descriptor

Center Symmetric-Local Binary Pattern (CSLBP) is textured based operator which is mostly used as keypoint descriptor, it is 256-length descriptor to represent single keypoint or affine patch. This operator is an extension of Local Binary Pattern (LBP) operator. The CSLBP descriptor is computationally simple, effective, and robust for various image transformations such as illumination change and image blurring. However, the space and time utilization of CSLBP can be improved by simple compression which can make CSLBP a smart selection for large databases and smart phones. In this paper, we propose simple compression of CSLBP without loss of its discriminative power. We reduce the descriptor length (dimensions) upto 50% without applying any dimensionality reduction techniques such as PCA or LDA. We evaluate our framework on state-of-the-art matching protocols and compare the effectiveness of proposed compressed descriptor (Q-CSLBP) with CSLBP, SIFT and PCA-SIFT.

Implementing Inductive Concept Learning For Cooperative Query Answering

Generalization operators have long been studied in the area of Conceptual Inductive Learning (Michalski, A theory and methodolgy of inductive learning. In: Machine learning: An artificial intelligence approach (pp. 111–161). TIOGA Publishing, 1983; De Raedt, About knowledge and inference in logical and relational learning. In: Advances in machine learning II (pp. 143–153). Springer, Berlin, 2010). We present an implementation of these learning operators in a prototype system for cooperative query answering. The implementation can however also be used as a usual concept learning mechanism for concepts described in first-order predicate logic. We sketch an extension of the generalization process by a ranking mechanism on answers for the case that some answers are not related to what user asked.

Performance Enhancement of Patch-based Descriptors for Image Copy Detection

Images have become main sources for the informa-tion, learning, and entertainment, but due to the advancement and progress in multimedia technologies, millions of images are shared on Internet daily which can be easily duplicated and redistributed. Distribution of these duplicated and transformed images cause a lot of problems and challenges such as piracy, redundancy, and content-based image indexing and retrieval. To address these problems, copy detection system based on local features are widely used. Initially, keypoints are detected and represented by some robust descriptors. The descriptors are computed over the affine patches around the keypoints, these patches should be repeatable under photometric and geometric transformations. However, there exist two main challenges with patch based descriptors, (1) the affine patch over the keypoint can produce similar descriptors under entirely different scene or the context which causes “ambiguity”, and (2) the descriptors are not enough “distinctive” under image noise. Due to these limitations, the copy detection systems suffer in performance. We present a framework that makes descriptor more distinguishable and robust by influencing them with the texture and gradients in vicinity. The experimental evaluation on keypoints matching and image copy detection under severe transformations shows the effectiveness of the proposed framework.

Automatic Image Segmentation Based on Maximal Similarity Based Region Merging

Image segmentation is one of the most significant tasks in computer vision. Since automatic techniques are hard for this purpose, a number of interactive techniques are used for image segmentation. The result of these techniques largely depends on user feedback. It is difficult to get good interactions for large databases. On the other hand, automatic image segmentation is becoming a significant objective in computer vision and image analysis. We propose an automatic framework to detect foreground. We are applying Maximal Similarity Based Region Merging (MSRM) technique for region merging and using image boundary to identify foreground regions. The results confirm the effectiveness of the proposed framework. The proposed framework reveals its effectiveness especially to extract multiple objects from background.

Creating missing classes automatically to improve question classification in question answering systems

Internet is one of the main sources for information which attracts million of users to find the answers for their questions. Finding the accurate answer of the question from giant databases or web pages is a very challenging task. Question answering systems answer the question from structured or unstructured databases. Question answering is different from the traditional ad-hoc document retrieval tasks in a way that in simple document search engine, the set of relevant documents are returned in response of the query, whereas, in the question answering systems the response of the query is the correct answer to what is asked. Finding the exact answer is more interesting and useful than getting a list of documents to look through and find the answer manually. Generally, the question classification is first phase in question answering systems. This phase reduces the answer space by pruning out the extra information that is irrelevant by finding out the expected answer type. This paper mainly focuses on Numeric type questions and also discusses briefly about the questions of type Entity and Location. Almost all the previous question classification algorithms evaluated their work by using the classes defined by Li and Roth [1]. The coarse grained class Numeric has fine grained class Other. In this paper, we target and present the mechanism to create new classes to replace the Other class in Numeric class. We present an automatic hierarchy creation method to add new class nodes using the knowledge resources and shallow language processing.

Performance Evaluation of SIFT and Convolutional Neural Network for Image Retrieval

Convolutional Neural Network (NN) has gained a lot of attention of the researchers due to its high accuracy in classification and feature learning. In this paper, we evaluated the performance of CNN used as feature for image retrieval with the gold standard feature, aka SIFT. Experiments are conducted on famous Oxford 5k data-set. The mAP of SIFT and CNN is 0.6279 and 0.5284, respectively. The performance of CNN is also compared with bag of visual word (BoVW) model. CNN achieves better accuracy than BoVW.

Filtering of Unrelated Answers in a CooperativeQuery Answering System

A database system may not always return answers for a query. Such a query is called a failing query. Under normal circumstances, an empty answer would be returned in response to such queries. Cooperative query answering systems produce generalized and relevant answers when an exact answer does not exist, by enhancing the query scope and including a broader range of information. Such systems may apply various generalization techniques, also referred to as generalization operators, to relax certain conditions and obtain related answers. These answers are not exact but informative answers that potentially contain some of the information that the user needs. Therefore, we propose a method to filter out unrelated answers and return only related answers to the user. We also propose a mechanism to have a restricted and optimized generalized query space by limiting the number of queries produced. We determine the similarity between user query and the answer produced. Unrelated answers are pruned out, and only the related and informative answers are returned to the user.