Chayan Chakrabarti

Automated annotation of functional imaging experiments via multi-label classification.

Identifying the experimental methods in human neuroimaging papers is important for grouping meaningfully similar experiments for meta-analyses. Currently, this can only be done by human readers. We present the performance of common machine learning (text mining) methods applied to the problem of automatically classifying or labeling this literature. Labeling terms are from the Cognitive Paradigm Ontology (CogPO), the text corpora are abstracts of published functional neuroimaging papers, and the methods use the performance of a human expert as training data. We aim to replicate the experts annotation of multiple labels per abstract identifying the experimental stimuli, cognitive paradigms, response types, and other relevant dimensions of the experiments. We use several standard machine learning methods: naive Bayes (NB), k-nearest neighbor, and support vector machines (specifically SMO or sequential minimal optimization). Exact match performance ranged from only 15% in the worst cases to 78% in the best cases. NB methods combined with binary relevance transformations performed strongly and were robust to overfitting. This collection of results demonstrates what can be achieved with off-the-shelf software components and little to no pre-processing of raw text.

A semantic architecture for artificial conversations

Artificial conversations have many applications in chatter bot-based customer service including website navigation tools and guided online shopping. Existing approaches to generating conversations leverage linguistic and stochastic principles, where lower level grammatical and structural artifacts are modeled. These approaches perform well in pairwise utterance exchanges, but not so well in longer conversational contexts. We simulate more meaningful chatter bot conversations using an architecture that can leverage content and context. Grices cooperative maxims, which form the central idea in the theory of pragmatics, is our framework for evaluation. The domain of our research is customer service situations, and we compare our artificial conversations with actual conversations of existing chatter bots deployed in the same domain.

Artificial conversations for customer service chatter bots

We describe limitations of the current generation of computer-based chatter-bots.We demonstrate structures and algorithms producing computer-based conversations.Our conversations combine semantic and pragmatic knowledge for customer service.Our computer-based dialogs are based on conversation theory and Grices maxims.We demonstrate methods to evaluate conversations in the customer service domain. Chatter bots are software programs that engage in artificial conversations through a text-based input medium. They are extensively deployed in customer service applications. Existing approaches to artificial conversation generation emphasize grammatical and linguistic modeling techniques. They focus on generation of discrete sentence-level utterances. These approaches perform poorly in conversational situations requiring contextual continuity over a series of utterances. We present an approach that combines pragmatics with content semantics to generate artificial conversations in the customer service domain. A conversation is a process that adheres to well-defined semantic conventions and is contextually grounded in domain-specific knowledge. We model this using stochastic finite state machines, where the parameters of the model are learned from a corpus of human conversations. We present a specific set of criteria which we then use to evaluate the quality of artificial conversations in the customer service domain. We also compare chatter bot generated artificial conversations with human generated natural conversations in this domain.

Statistical algorithms for ontology-based annotation of scientific literature

BackgroundOntologies encode relationships within a domain in robust data structures that can be used to annotate data objects, including scientific papers, in ways that ease tasks such as search and meta-analysis. However, the annotation process requires significant time and effort when performed by humans. Text mining algorithms can facilitate this process, but they render an analysis mainly based upon keyword, synonym and semantic matching. They do not leverage information embedded in an ontologys structure.MethodsWe present a probabilistic framework that facilitates the automatic annotation of literature by indirectly modeling the restrictions among the different classes in the ontology. Our research focuses on annotating human functional neuroimaging literature within the Cognitive Paradigm Ontology (CogPO). We use an approach that combines the stochastic simplicity of naïve Bayes with the formal transparency of decision trees. Our data structure is easily modifiable to reflect changing domain knowledge.ResultsWe compare our results across naïve Bayes, Bayesian Decision Trees, and Constrained Decision Tree classifiers that keep a human expert in the loop, in terms of the quality measure of the F1-mirco score.ConclusionsUnlike traditional text mining algorithms, our framework can model the knowledge encoded by the dependencies in an ontology, albeit indirectly. We successfully exploit the fact that CogPO has explicitly stated restrictions, and implicit dependencies in the form of patterns in the expert curated annotations.

THE DESIGN AND TESTING OF A FIRST-ORDER LOGIC-BASED STOCHASTIC MODELING LANGUAGE

We have created a logic-based, Turing-complete language for stochastic modeling. Since the inference scheme for this language is based on a variant of Pearls loopy belief propagation algorithm, we...