Sriharsha Veeramachaneni

Style context with second-order statistics

Patterns often occur as homogeneous groups or fields generated by the same source. In multisource recognition problems, such isogeny induces statistical dependencies between patterns (termed style context). We model these dependencies by second-order statistics and formulate the optimal classifier for normally distributed styles. We show that model parameters estimated only from pairs of classes suffice to train classifiers for any test field length. Although computationally expensive, the style-conscious classifier reduces the field error rate by up to 20 percent on quadruples of handwritten digits from standard NIST data sets.

Adaptive classifiers for multisource OCR

Abstract.When patterns occur in large groups generated by a single source (style consistent test data), the statistics of the test data differ from those of the training data, which consist of patterns from all sources. We present a Gaussian model for continuously distributed sources under which we develop adaptive classifiers that specialize in the statistics of style-consistent test data. On NIST handwritten digit data, the adaptive classifiers reduce the error rate by more than 50% operating on one writer (

Analytical Results on Style-Constrained Bayesian Classification of Pattern Fields

\thickapprox 10

Bayesian hypothesis testing for pattern discrimination in brain decoding

samples/class) at a time.

Inferring cognition from fMRI brain images

We formalize the notion of style context, which accounts for the increased accuracy of the field classifiers reported in this journal recently. We argue that style context forms the basis of all order-independent field classification schemes. We distinguish between intraclass style, which underlies most adaptive classifiers, and interclass style, which is a manifestation of interpattern dependence between the features of the patterns of a field. We show how style-constrained classifiers can be optimized either for field error (useful for short fields like zip codes) or for singlet error (for long fields, like business letters). We derive bounds on the reduction of error rate with field length and show that the error rate of the optimal style-constrained field classifier converges asymptotically to the error rate of a style-aware Bayesian singlet classifier.

Active sampling for detecting irrelevant features

Research in cognitive neuroscience and in brain-computer interfaces (BCI) is frequently concerned with finding evidence that a given brain area processes, or encodes, given stimuli. Experiments based on neuroimaging techniques consist of a stimulation protocol presented to a subject while his or her brain activity is being recorded. The question is then whether there is enough evidence of brain activity related to the stimuli within the recorded data. Finding a link between brain activity and stimuli has recently been proposed as a classification task, called brain decoding. A classifier that can accurately predict which stimuli were presented to the subject provides support for a positive answer to the question. However, it is only the answer for a given data set and the question still remains whether it is a general rule that will apply also to new data. In this paper we try to reliably answer the neuroscientific question about the presence of a significant link between brain activity and stimuli once we have the classification results. The proposed method is based on a Beta-Binomial model for the population of generalization errors of classifiers from multi-subject studies within the Bayesian hypothesis testing framework. We present an application on nine brain decoding investigations from a real functional magnetic resonance imaging (fMRI) experiment about the relation between mental calculation and eye movements.

Adaptive and Interactive Approaches to Document Analysis

Over the last few years, functional Magnetic Resonance Imaging (fMRI) has emerged as a new and powerful method to map the cognitive states of a human subject to specific functional areas of the subject brain. Although fMRI has been widely used to determine average activation in different brain regions, the problem of automatically decoding the cognitive state from instantaneous brain activations has received little attention. In this paper, we study this prediction problem on a complex time-series dataset that relates fMRI data (brain images) with the corresponding cognitive states of the subjects while watching three 20 minute movies. This work describes the process we used to reduce the extremely high-dimensional feature space and a comparison of the models used for prediction. To solve the prediction task we explored a standard linear model frequently used by neuroscientists, as well as a k-nearest neighbor model, that now are the state-of-art in this area. Finally, we provide experimental evidence that non-linear models such as multi-layer perceptron and especially recurrent neural networks are significantly better.

Word discrimination based on bigram co-occurrences

The general approach for automatically driving data collection using information from previously acquired data is called active learning. Traditional active learning addresses the problem of choosing the unlabeled examples for which the class labels are queried with the goal of learning a classifier. In contrast we address the problem of active feature sampling for detecting useless features. We propose a strategy to actively sample the values of new features on class-labeled examples, with the objective of feature relevance assessment. We derive an active feature sampling algorithm from an information theoretic and statistical formulation of the problem. We present experimental results on synthetic, UCI and real world datasets to demonstrate that our active sampling algorithm can provide accurate estimates of feature relevance with lower data acquisition costs than random sampling and other previously proposed sampling algorithms.

Modeling context as statistical dependence

This chapter explores three aspects of learning in document analysis: (1) field classification, (2) interactive recognition, and (3) portable and networked applications. Context in document classification conventionally refers to language context, i.e., deterministic or statistical constraints on the sequence of letters in syllables or words, and on the sequence of words in phrases or sentences. We show how to exploit other types of statistical dependence, specifically the dependence between the shape features of several patterns due to the common source of the patterns within a field or a document. This type of dependence leads to field classification, where the features of some patterns may reveal useful information about the features of other patterns from the same source but not necessarily from the same class. We explore the relationship between field classification and the older concepts of unsupervised learning and adaptation. Human interaction is often more effective interspersed with algorithmic processes than only before or after the automated parts of the process. We develop a taxonomy for interaction during training and testing, and show how either human-initiated and machine-initiated interaction can lead to human and machine learning. In a section on new technologies, we discuss how new cameras and displays, web-wide access, interoperability, and essentially unlimited storage provide fertile new approaches to document analysis.

Active sampling for knowledge discovery from biomedical data

Very few pairs of English words share exactly the same letter bigrams. This linguistic property can be exploited to bring lexical context into the classification stage of a word recognition system. The lexical n-gram matches between every word in a lexicon and a subset of reference words can be precomputed. If a match function can detect matching segments of at least n-gram length from the feature representation of words, then an unknown word can be recognized by determining the subset of reference words having an n-gram match at the feature level with the unknown word. We show that with a reasonable number of reference words, bigrams represent the best compromise between the recall ability of single letters and the precision of trigrams. Our simulations indicate that using a longer reference list can compensate errors in feature extraction. The algorithm is fast enough, even with a slow processor, for human-computer interaction.