Giuseppe Carenini

Extracting knowledge from evaluative text

Capturing knowledge from free-form evaluative texts about an entity is a challenging task. New techniques of feature extraction, polarity determination and strength evaluation have been proposed. Feature extraction is particularly important to the task as it provides the underpinnings of the extracted knowledge. The work in this paper introduces an improved method for feature extraction that draws on an existing unsupervised method. By including user-specific prior knowledge of the evaluated entity, we turn the task of feature extraction into one of term similarity by mapping crude (learned) features into a user-defined taxonomy of the entitys features. Results show promise both in terms of the accuracy of the mapping as well as the reduction in the semantic redundancy of crude features.

MULTI-DOCUMENT SUMMARIZATION OF EVALUATIVE TEXT

In many decision‐making scenarios, people can benefit from knowing what other peoples opinions are. As more and more evaluative documents are posted on the Web, summarizing these useful resources becomes a critical task for many organizations and individuals. This paper presents a framework for summarizing a corpus of evaluative documents about a single entity by a natural language summary. We propose two summarizers: an extractive summarizer and an abstractive one. As an additional contribution, we show how our abstractive summarizer can be modified to generate summaries tailored to a model of the user preferences that is solidly grounded in decision theory and can be effectively elicited from users. We have tested our framework in three user studies. In the first one, we compared the two summarizers. They performed equally well relative to each other quantitatively, while significantly outperforming a baseline standard approach to multidocument summarization. Trends in the results as well as qualitative comments from participants suggest that the summarizers have different strengths and weaknesses. After this initial user study, we realized that the diversity of opinions expressed in the corpus (i.e., its controversiality) might play a critical role in comparing abstraction versus extraction. To clearly pinpoint the role of controversiality, we ran a second user study in which we controlled for the degree of controversiality of the corpora that were summarized for the participants. The outcome of this study indicates that for evaluative text abstraction tends to be more effective than extraction, particularly when the corpus is controversial. In the third user study we assessed the effectiveness of our user tailoring strategy. The results of this experiment confirm that user tailored summaries are more informative than untailored ones.

Generating and evaluating evaluative arguments

Evaluative arguments are pervasive in natural human communication. In countless situations people attempt to advise or persuade their interlocutors that something is desirable (vs. undesirable) or right (vs. wrong). With the proliferation of on-line systems serving as personal advisors and assistants, there is a pressing need to develop general and testable computational models for generating and presenting evaluative arguments. Previous research on generating evaluative arguments has been characterized by two major limitations. First, researchers have tended to focus only on specific aspects of the generation process. Second, the proposed approaches were not empirically tested. The research presented in this paper addresses both limitations. We have designed and implemented a complete computational model for generating evaluative arguments. For content selection and organization, we devised an argumentation strategy based on guidelines from argumentation theory. For expressing the content in natural language, we extended and integrated previous work in computational linguistics on generating evaluative arguments. The key knowledge source for both tasks is a quantitative model of user preferences. To empirically test critical aspects of our generation model, we have devised and implemented an evaluation framework in which the effectiveness of evaluative arguments can be measured with real users. Within the framework, we have performed an experiment to test two basic hypotheses on which the design of the computational model is based; namely, that our proposal for tailoring an evaluative argument to the addressees preferences increases its effectiveness, and that differences in conciseness significantly influence argument effectiveness. The second hypothesis was confirmed in the experiment. In contrast, the first hypothesis was only marginally confirmed. However, independent testing by other researchers has recently provided further support for this hypothesis.

Summarizing email conversations with clue words

Accessing an ever increasing number of emails, possibly on small mobile devices, has become a major problem for many users. Email summarization is a promising way to solve this problem. In this paper, we propose a new framework for email summarization. One novelty is to use a fragment quotation graph to try to capture an email conversation. The second novelty is to use clue words to measure the importance of sentences in conversation summarization. Based on clue words and their scores, we propose a method called CWS, which is capable of producing a summary of any length as requested by the user. We provide a comprehensive comparison of CWS with various existing methods on the Enron data set. Preliminary results suggest that CWS provides better summaries than existing methods.

User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities

Information Visualization systems have traditionally followed a one-size-fits-all model, typically ignoring an individual users needs, abilities and preferences. However, recent research has indicated that visualization performance could be improved by adapting aspects of the visualization to each individual user. To this end, this paper presents research aimed at supporting the design of novel user-adaptive visualization systems. In particular, we discuss results on using information on user eye gaze patterns while interacting with a given visualization to predict the users visualization tasks, as well as user cognitive abilities including perceptual speed, visual working memory, and verbal working memory. We show that such predictions are significantly better than a baseline classifier even during the early stages of visualization usage. These findings are discussed in view of designing visualization systems that can adapt to each individual user in real-time.

Towards more conversational and collaborative recommender systems

Current recommender systems, based on collaborative filtering, implement a rather limited model of interaction. These systems intelligently elicit information from a user only during the initial registration phase. Furthermore, users tend to collaborate only indirectly. We believe there are several unexplored opportunities in which information can be effectively elicited from users by making the underlying interaction model more conversational and collaborative. In this paper, we propose a set of techniques to intelligently select what information to elicit from the user in situations in which the user may be particularly motivated to provide such information. We argue that the resulting interaction improves the user experience. We conclude by reporting results of an offline experiment in which we compare the influence of different elicitation techniques on both the accuracy of the systems predictions and the users effort

Dealing with the expert inconsistency in probability elicitation

In this paper, we present and discuss our experience in the task of probability elicitation from experts for the purpose of belief network construction. In our study, we applied four techniques. Three of these techniques are available from the literature, whereas the fourth one is a technique that we developed by adapting a method for the assessment of preferences to the task of probability elicitation. The new technique is based on the analytic hierarchy process (AHP) proposed by Saaty (1980, 1994), and it allows for the quantitative assessment of the expert inconsistency. The method is, in our opinion, very promising and lends itself to be applied more extensively to the task of probability elicitation.

Summarizing Spoken and Written Conversations

In this paper we describe research on summarizing conversations in the meetings and emails domains. We introduce a conversation summarization system that works in multiple domains utilizing general conversational features, and compare our results with domain-dependent systems for meeting and email data. We find that by treating meetings and emails as conversations with general conversational features in common, we can achieve competitive results with state-of-the-art systems that rely on more domain-specific features.

Interactive multimedia summaries of evaluative text

We present an interactive multimedia interface for automatically summarizing large corpora of evaluative text (e.g. online product reviews). We rely on existing techniques for extracting knowledge from the corpora but present a novel approach for conveying that knowledge to the user. Our system presents the extracted knowledge in a hierarchical visualization mode as well as in a natural language summary. We propose a method for reasoning about the extracted knowledge so that the natural language summary can include only the most important information from the corpus. Our approach is interactive in that it allows the user to explore in the original dataset through intuitive visual and textual methods. Results of a formative evaluation of our interface show general satisfaction among users with our approach.

Generating explanations in context

If user interfaces are to reap the bene ts of natural language interaction they must be endowed with the properties that make human natural language interaction so e ective Human human explanation is an inherently incremental and interactive process New information must be highlighted and related to what has already been presented In this paper we describe the explanation component of a medical information giving system We describe the architectural features that enable this component to generate subsequent explanations that take into account the context created by its prior utterances GENERATING EXPLANATIONS IN CONTEXT Giuseppe Carenini and Johanna D Moore University of Pittsburgh Department of Computer Science Pittsburgh PA Phone fcarenini jmooreg cs pitt edu