Anne Kao

Natural Language Processing and Text Mining

Natural Language Processing and Text Mining not only discusses applications of Natural Language Processing techniques to certain Text Mining tasks, but also the converse, the use of Text Mining to assist NLP. It assembles a diverse views from internationally recognized researchers and emphasizes caveats in the attempt to apply Natural Language Processing to text mining. This state-of-the-art survey is a must-have for advanced students, professionals, and researchers.

Text mining and natural language processing: introduction for the special issue

This paper provides an introduction to this special issue of SIGKDD Explorations devoted to Natural Language Processing and Text Mining.

Visualizing text data sets

The authors present a visualization methodology which provides users with a way to alter perspectives and interpret visualization so that they can quickly identify trends, outliers, and possible clusters while tuning for a particular context. The technology developed for text mining is called Trust, or Text Representation Using Subspace Transformation. Trust provides an analysis environment that can supply meaningful representations of text documents; it also supports the functional ability to visually present a collection of documents in a meaningful context that allows for user insight and textual content. Contrary to other similar technologies, Trust applies a novel analysis ability that allows different subspaces to generate views, providing content information for the basis of the visualization and allowing an analyst to specify subspaces for it based on content.

User assisted text classification and knowledge management

While there are many aspects to managing corporate knowledge, one key issue is how to organize corporate documents into categories of interest. In this paper, we focus on using user assisted text classification in conjunction with a web portal, multiple document management systems and an ontology, to provide a powerful solution for organizing information about a companys technology. We propose a system that interacts with an author using an automatic text classifier to suggest controlled keywords to be used as metadata. The proposed approach does not require professional librarians or that the end users have extensive training. The use of a controlled vocabulary allows for a more consistent description of corporate documents, and promotes easier access by people across the company. It is easier to find similar documents which use different nomenclature. Finally, the interactive nature of the system results in a more correct and precise description of each document than a fully automatic system would.

Report on KDD conference 2004 panel discussion can natural language processing help text mining

With large amounts of text data now available on-line, both on the Internet and in corporate repositories, text mining is an area of growing interest. Historically, Data Mining researchers have come out of the statistics, database and machine learning communities. Despite a few exceptions, it has by and large had little interaction with the computational linguistics and natural language processing (NLP) community. Given that the subject matter of text mining is free text, one might naturally assume techniques developed over the last few decades in computational linguistics and NLP, should make big contributions towards the younger field of text mining. However, other than in the area of information extraction, empirical evidence has not borne this out.

DataJewel: Integrating Visualization with Temporal Data Mining

In this chapter we describe DataJewel, a new temporal data mining architecture. DataJewel tightly integrates a visualization component, an algorithmic component and a database component. We introduce a new visualization technique called CalendarView as an implementation of the visualization component, and we introduce a data structure that supports temporal mining of large databases. In our architecture, algorithms can be tightly integrated with the visualization component and most existing temporal data mining algorithms can be leveraged by embedding them into DataJewel. This integration is achieved by an interface that is used by both the user and the algorithms to assign colors to events. The user interactively assigns colors to incorporate domain knowledge or to formulate hypotheses. The algorithm assigns colors based on discovered patterns. The same visualization technique is used for displaying both data and patterns to make it more intuitive for the user to identify useful patterns while exploring data interactively or while using algorithms to search for patterns. Our experiments in analyzing several large datasets from the airplane maintenance domain demonstrate the usefulness of our approach and we discuss its applicability to domains like homeland security, market basket analysis and web mining.

TALISON - Tensor analysis of social media data

With the growth in social media, online forums have become major sources of data for social network analysis and a major research area in data analytics. However, many aspects of social networks have not been addressed fully, if at all. First, there is the need to capture various parts of the content of the exchanges, for example, bare text and hashtags, which constitute some of the major features of social media networks. Second, as in all social networks, there are many different types of relationships; for example, “following” and “friend” relationships, as well as relationships defined in terms of how users interact with one another such as replying, quoting, retweeting, and mentioning. Finally, an important aspect of analysis of social networks is temporal dynamics and topic evolution. We offer an analysis of Twitter data using tensors which can incorporate all of these different aspects in a single representation and both identify salient events and distinguish important views of these events.

Knowledge Management for Coalition Information Sharing at the Network Edge

This article describes ongoing research on data integration and query services for knowledge management. For such services, Controlled English (CE), a human-friendly, machine-readable language, can represent information content and context.

Representing Uncertainty in CE

Providing analysts and decision makers with a means of assessing how certain extracted information is, and what the sources of uncertainty are, is an important part of the provenance of a piece of intelligence for decision-making. A major source of (un)certainty derives from the text in reports that analysts and decision makers rely on. We outline an analysis of uncertainty expressions used in English. We show how a controlled English can be used to represent the uncertainty expressions and infer important information, and how this information might be represented to analysts in support of decision-making for military operations.