Veena Thenkanidiyoor

The semantic multinomial representation of images obtained using dynamic kernel based pseudo-concept SVMs

Hand-engineered local image features have been proven to be intended representation for a variety of high-level visual recognition tasks. But as the visual recognition tasks such as scene classification and object detection become more challenging, the semantic gap between low-level feature and the concept descriptor of the scene images increases. In this paper, we present novel semantic multinomial (SMN) image representation that renders it possible to represent natural scenes by complex semantic description. SMN is a semantic representation of an image that corresponds to a vector of posterior probabilities of concepts. Proposed SMN representation uses dynamic kernel based support vector machines (SVMs) to model the semantic content of images. It is necessary to have ground truth (true) concept labels to obtain SMN representation. In this work, we also propose to use pseudo-concepts in the absence of true concept labels. The proposed SMN representation is also complementary to the low-level visual representation. Combining the scores of classifiers using SMN representation and low-level visual representation is shown to achieve state-of-the-art results for high-level visual tasks such as scene classification on standard datasets.

Text classification using combined sparse representation classifiers and support vector machines

Text classification is an important task in managing huge repository of textual content prevailing in various domains. In this paper, we propose to use sparse representation classifier (SRC) and support vector machines (SVMs) based classifiers using frequency-based kernels for text classification. We consider term-frequency (TF) representation for a text document. The sparse representation of an example is obtained by using an overcomplete dictionary made up of TF vectors corresponding to all the training documents [1]. We propose to seed the dictionary using principal components of TF vector representation corresponding to training text documents. SVM-based text classifiers use linear kernel or Gaussian kernel on the TF vector representation of documents. TF representation being a non-negative, histogram representation, we propose to build SVM-based text classifiers using frequency-based kernels such as histogram intersection kernel, Chi-square (X2) kernel and Hellingers kernel. It is observed that the examples misclassified by one classifier is correctly classified in another classifier. To take advantage of the various classifiers, we introduce an approach to combine classifiers to improve the performance of text classification. The effectiveness of all the proposed techniques for text classification is demonstrated on 20 Newsgroup Corpus.

Segment-level pyramid match kernels for the classification of varying length patterns of speech using SVMs

Classification of long duration speech, represented as varying length sets of feature vectors using support vector machine (SVM) requires a suitable kernel. In this paper we propose a novel segment-level pyramid match kernel (SLPMK) for the classification of varying length patterns of long duration speech represented as sets of feature vectors. This kernel is designed by partitioning the speech signal into increasingly finer segments and matching the corresponding segments. We study the performance of the SVM-based classifiers using the proposed SLPMKs for speech emotion recognition and speaker identification and compare with that of the SVM-based classifiers using other dynamic kernels.

Example-Specific Density Based Matching Kernel for Classification of Varying Length Patterns of Speech Using Support Vector Machines

In this paper, we propose example-specific density based matching kernel (ESDMK) for the classification of varying length patterns of long duration speech represented as sets of feature vectors. The proposed kernel is computed between the pair of examples, represented as sets of feature vectors, by matching the estimates of the example-specific densities computed at every feature vector in those two examples. In this work, the number of feature vectors of an example among the K nearest neighbors of a feature vector is considered as an estimate of the example-specific density. The minimum of the estimates of two example-specific densities, one for each example, at a feature vector is considered as the matching score. The ESDMK is then computed as the sum of the matching score computed at every feature vector in a pair of examples. We study the performance of the support vector machine (SVM) based classifiers using the proposed ESDMK for speech emotion recognition and speaker identification tasks and compare the same with that of the SVM-based classifiers using the state-of-the-art kernels for varying length patterns.

Modified Time Flexible Kernel for Video Activity Recognition using Support Vector Machines.

Video activity recognition involves automatically assigning a activity label to a video. This is a challenging task due to the complex nature of video data. There exists many sub activities whose temporal order is important. For building an SVM-based activity recognizer it is necessary to use a suitable kernel that considers varying length temporal data corresponding to videos. In (Mario Rodriguez and Makris, 2016), a time flexible kernel (TFK) is proposed for matching a pair of videos by encoding a video into a sequence of bag of visual words (BOVW) vectors. The TFK involves matching every pair of BOVW vectors from a pair of videos using linear kernel. In this paper we propose modified TFK (MTFK) where better approaches to match a pair of BOVW vectors are explored. We propose to explore the use of frequency based kernels for matching a pair of BOVW vectors. We also propose an approach for encoding the videos using Gaussian mixture models based soft clustering technique. The effectiveness of the proposed approaches are studied using benchmark datasets.

Deep Spatial Pyramid Match Kernel for Scene Classification.

Several works have shown that Convolutional Neural Networks (CNNs) can be easily adapted to different datasets and tasks. However, for extracting the deep features from these pre-trained deep CNNs a fixedsize (e.g., 227×227) input image is mandatory. Now the state-of-the-art datasets like MIT-67 and SUN-397 come with images of different sizes. Usage of CNNs for these datasets enforces the user to bring different sized images to a fixed size either by reducing or enlarging the images. The curiosity is obvious that “Isn’t the conversion to fixed size image is lossy ?”. In this work, we provide a mechanism to keep these lossy fixed size images aloof and process the images in its original form to get set of varying size deep feature maps, hence being lossless. We also propose deep spatial pyramid match kernel (DSPMK) which amalgamates set of varying size deep feature maps and computes a matching score between the samples. Proposed DSPMK act as a dynamic kernel in the classification framework of scene dataset using support vector machine. We demonstrated the effectiveness of combining the power of varying size CNN-based set of deep feature maps with dynamic kernel by achieving state-of-the-art results for high-level visual recognition tasks such as scene classification on standard datasets like MIT67 and SUN397.

Deep CNN based pseudo-concept selection and modeling for generation of semantic multinomial representation of scene images

Though recent convolutional neural network (CNN) based method for scene classification task show impressive results but lacks in capturing the complex semantic content of the scene images. To reduce the semantic gap a semantic multinomial (SMN) representation is introduced. SMN representation corresponds to a vector of posterior probabilities of concepts. The core part of SMN generation is building the concept model. For building the concept model, it is necessary to have ground truth (true) concept labels for every image in the database. In this research work, we propose novel deep CNN based SMN representation which exploits convolutional layer filters response as pseudo concepts to build the concept model in the absence of true concept labels. The effectiveness of the proposed approach is studied for scene classification tasks on standard datasets like MIT67 and SUN397.

Semantic Multinomial Representation for Scene Images Using CNN-Based Pseudo-concepts and Concept Neural Network

For challenging visual recognition tasks such as scene classification and object detection there is a need to bridge the semantic gap between low-level features and the semantic concept descriptors. This requires mapping a scene image onto a semantic representation. Semantic multinomial (SMN) representation is a semantic representation of an image that corresponds to a vector of posterior probabilities of concepts. In this work we propose to build a concept neural network (CoNN) to obtain the SMN representation for a scene image. An important issue in building a CoNN is that it requires the availability of ground truth concept labels. In this work we propose to use pseudo-concepts obtained from feature maps of higher level layers of convolutional neural network. The effectiveness of the proposed approaches are studied using standard datasets.

Segment-Level Probabilistic Sequence Kernel Based Support Vector Machines for Classification of Varying Length Patterns of Speech

In this work we propose the segment-level probabilistic sequence kernel (SLPSK) as dynamic kernel to be used in support vector machine (SVM) for classification of varying length patterns of long duration speech represented as sets of feature vectors. SLPSK is built upon a set of Gaussian basis functions, where half of the basis functions contain class specific information while the other half implicates the common characteristics of all the speech utterances of all classes. The proposed kernel is computed between the pair of examples, by partitioning the speech signal into fixed number of segments and then matching the corresponding segments. We study the performance of the SVM-based classifiers using the proposed SLPSK using different pooling technique for speech emotion recognition and speaker identification and compare with that of the SVM-based classifiers using other kernels for varying length patterns.

A Family of Example-Specific Density Based Matching Kernels for Classification of Varying Length Patterns of Speech Using Support Vector Machines

In this paper, we propose example-specific density based matching kernel (ESDMK) for the classification of varying length patterns of long duration speech represented as sets of feature vectors. The proposed kernel is computed between the pair of examples, represented as sets of feature vectors, by matching the estimates of the example-specific densities computed at every feature vector in those two examples. In this work, the number of feature vectors of an example among the K nearest neighbors of a feature vector is considered as an estimate of the example-specific density. The minimum of the estimates of two example-specific densities, one for each example, at a feature vector is considered as the matching score. The ESDMK is then computed as the sum of the matching score computed at every feature vector in a pair of examples. We also propose to compute pyramid match ESDMK by considering the increasingly larger number of neighbors to form pyramid of neighbors. We study the performance of the support vector machine (SVM) based classifiers using the proposed ESDMK and pyramid match ESDMK for speech emotion recognition and speaker identification tasks and compare the same with that of the SVM-based classifiers using the state-of-the-art kernels for varying length patterns.