Tetsuya Nasukawa

Sentiment analysis: capturing favorability using natural language processing

This paper illustrates a sentiment analysis approach to extract sentiments associated with polarities of positive or negative for specific subjects from a document, instead of classifying the whole document into positive or negative.The essential issues in sentiment analysis are to identify how sentiments are expressed in texts and whether the expressions indicate positive (favorable) or negative (unfavorable) opinions toward the subject. In order to improve the accuracy of the sentiment analysis, it is important to properly identify the semantic relationships between the sentiment expressions and the subject. By applying semantic analysis with a syntactic parser and sentiment lexicon, our prototype system achieved high precision (75-95%, depending on the data) in finding sentiments within Web pages and news articles.

Fully Automatic Lexicon Expansion for Domain-oriented Sentiment Analysis

This paper proposes an unsupervised lexicon building method for the detection of polar clauses, which convey positive or negative aspects in a specific domain. The lexical entries to be acquired are called polar atoms, the minimum human-understandable syntactic structures that specify the polarity of clauses. As a clue to obtain candidate polar atoms, we use context coherency, the tendency for same polarities to appear successively in contexts. Using the overall density and precision of coherency in the corpus, the statistical estimation picks up appropriate polar atoms among candidates, without any manual tuning of the threshold values. The experimental results show that the precision of polarity assignment with the automatically acquired lexicon was 94% on average, and our method is robust for corpora in diverse domains and for the size of the initial lexicon.

Layout and language: integrating spatial and linguistic knowledge for layout understanding tasks

Complex documents stored in a flat or partially marked up file format require layout sensitive preprocessing before any natural language processing can be carried out on their textual content. Contemporary technology for the discovery of basic textual units is based on either spatial or other content insensitive methods. However, there are many cases where knowledge of both the language and layout is required in order to establish the boundaries of the basic textual blocks. This paper describes a number of these cases and proposes the application of a general method combining knowledge about language with knowledge about the spatial arrangement of text. We claim that the comprehensive understanding of layout can only be achieved through the exploitation of layout knowledge and language knowledge in an inter-dependent manner.

Getting insights from the voices of customers: Conversation mining at a contact center

Business-oriented conversations between customers and agents need to be analyzed to obtain valuable insights that can be used to improve product and service quality, operational efficiency, and revenue. For such an analysis, it is critical to identify appropriate textual segments and expressions to focus on, especially when the textual data consists of complete transcripts, which are often lengthy and redundant. In this paper, we propose a method to identify important segments from the conversations by looking for changes in the accuracy of a categorizer designed to separate different business outcomes. We then use text mining to extract important associations between key entities (insights). We show the effectiveness of the method for making chance discoveries by using real life data from a car rental service center.

A Conversation-Mining System for Gathering Insights to Improve Agent Productivity

We describe a method to analyze transcripts of conversations between customers and agents in a contact center. The aim is to obtain actionable insights from the conversations to improve agent performance. Our approach has three steps. First we segment the call into logical parts. Next we extract relevant phrases within different segments. Finally we do two dimensional association analysis to identify actionable trends. We use real data from a contact center to identify specific actions by agents that result in positive outcomes. We also show that implementing the actionable results in improved agent productivity.

Textual Demand Analysis: Detection of Users' Wants and Needs from Opinions

This paper tackles textual demand analysis, the task of capturing what people want or need, rather than identifying what they like or dislike, on which much conventional work has focused. It exploits syntactic patterns as clues to detect previously unknown demands, and requires domaindependent knowledge to get high recall. To build such patterns we created an unsupervised pattern induction method relying on the hypothesis that there are commonly desired aspects throughout a domain corpus. Experimental results show that the proposed method detects twice to four times as many demand expressions in Japanese discussion forums compared to a baseline method.

Robust Parsing Based on Discourse Information: Completing partial parses of ill-formed sentences on the basis of discourse information

In a consistent text, many words and phrases are repeatedly used in more than one sentence. When an identical phrase (a set of consecutive words) is repeated in different sentences, the constituent words of those sentences tend to be associated in identical modification patterns with identical parts of speech and identical modifiee-modifier relationships. Thus, when a syntactic parser cannot parse a sentence as a unified structure, parts of speech and modifiee-modifier relationships among morphologically identical words in complete parses of other sentences within the same text provide useful information for obtaining partial parses of the sentence.In this paper, we describe a method for completing partial parses by maintaining consistency among morphologically identical words within the same text as regards their part of speech and their modifiee-modifier relationship. The experimental results obtained by using this method with technical documents offer good prospects for improving the accuracy of sentence analysis in a broad-coverage natural language processing system such as a machine translation system.

Unsupervised lexicon induction for clause-level detection of evaluations

This article proposes clause-level evaluation detection, which is a fine-grained type of opinion mining, and describes an unsupervised lexicon building method for capturing domain-specific knowledge by leveraging the similar polarities of sentiments between adjacent clauses. The lexical entries to be acquired are called polar atoms, the minimum human-understandable syntactic structures that specify the polarity of clauses. As a hint to obtain candidate polar atoms, we use context coherency, the tendency for the same polarity to appear successively in a context. Using the overall density and precision of coherency in the corpus, the statistical estimation picks up appropriate polar atoms from among the candidates, without any manual tuning of the threshold values. The experimental results show that the precision of polarity assignment with the automatically acquired lexicon was 83 per cent on average, and our method is robust for corpora in diverse domains and for the size of the initial lexicon.

Shalt2: a symmetric machine translation system with conceptual transfer

Shalt2 is a knowledge-based machine translation system with a symmetric architecture. The grammar rules, mapping rules between syntactic and conceptual (semantic) representations, and transfer rules for conceptual paraphrasing are all bi-directional knowledge sources used by both a parser and a generator.

Term aggregation: mining synonymous expressions using personal stylistic variations

We present a text mining method for finding synonymous expressions based on the distributional hypothesis in a set of coherent corpora. This paper proposes a new methodology to improve the accuracy of a term aggregation system using each authors text as a coherent corpus. Our approach is based on the idea that one person tends to use one expression for one meaning. According to our assumption, most of the words with similar context features in each authors corpus tend not to be synonymous expressions. Our proposed method improves the accuracy of our term aggregation system, showing that our approach is successful.