Ingrid Renz

Adaptive information filtering: detecting changes in text streams

The task of information filtering is to classify documents from a stream as either relevant or non-relevant according to a particular user interest with the objective to reduce information load. When using an information filter in an environment that is changing with time, methods for adapting the filter should be considered in order to retain classification accuracy. We favor a methodology that attempts to detect changes and adapts the information filter only if inevitable in order to minimize the amount of user feedback for providing new training data. Yet, detecting changes may require costly user feedback as well. This paper describes two methods for detecting changes without user feedback. The first method is based on evaluating an expected error rate, while the second one observes the fraction of classification decisions made with a confidence below a given threshold. Further, a heuristics for automatically determining this threshold is suggested and the performance of this approach is experimentally explored as a function of the threshold parameter. Some empirical results show that both methods work well in a simulated change scenario with real world data.

Categorizing Paper Documents

Text categorization assigns predefined categories to either electronically available texts or those resulting from document image analysis. A generic system for text categorization is presented which is based on statistical analysis of representative text corpora. Significant features are automatically derived from training texts by selecting substrings from actual word forms and applying statistical information and general linguistic knowledge. The dimension of the feature vectors is then reduced by linear transformation, keeping the essential information. The classification is a minimum least-squares approach based on polynomials. The described system can be efficiently adapted to new domains or different languages. In application, the adapted text categorizers are reliable, fast, and completely automatic. Two example categorization tasks achieve recognition scores of approximately 80% and are very robust against recognition or typing errors.

Textual Analysis of Customer Statements for Quality Control and Help Desk Support

Several business to customer applications, i.e. analysis of customer feedback and inquiries, can be improved by text mining approaches. They give new insights in the customer’s needs and desires by automatically processing their messages. Previously unknown facts and relations can be detected and organizations as well as employees profit by these document and knowledge management tools. The techniques used are rather simple but robust: they are derived from basic distance calculation between feature vectors in the vector space model.

Regular ArticleCategorizing Paper Documents: A Generic System for Domain and Language Independent Text Categorization☆

Text categorization assigns predefined categories to either electronically available texts or those resulting from document image analysis. A generic system for text categorization is presented which is based on statistical analysis of representative text corpora. Significant features are automatically derived from training texts by selecting substrings from actual word forms and applying statistical information and general linguistic knowledge. The dimension of the feature vectors is then reduced by linear transformation, keeping the essential information. The classification is a minimum least-squares approach based on polynomials. The described system can be efficiently adapted to new domains or different languages. In application, the adapted text categorizers are reliable, fast, and completely automatic. Two example categorization tasks achieve recognition scores of approximately 80% and are very robust against recognition or typing errors.

Document retrieval from user-selected web sites

We present a new tool for gathering textual information according to a query (texts) on arbitrary web sites specified by an information-seeking user. This tool is helpful in any knowledge-intensive area. Its technology is based on the vector space model with optimized feature definition. .

Inspecting Document Collections

The paper introduces two procedures which allow information seekers to inspect large document collections. The first method structures document collections into sensible groups. Here, three different approaches are presented: grouping based on the topology of the collection (i.e. linking and directory structure of intranet documents), grouping based on the content of the documents (i.e. similarity relation), and grouping based on the reader’s behavior when using the document collection. After the formation of groups, the second method supports readers by characterizing text through extracting short and relevant information from single documents and groups. Using statistical approaches, representative keywords of each document and also of the document groups are calculated. Later, the most important sentences from single documents and document groups are extracted as summaries. Geared to the different information needs, algorithms for indicative, informative, and thematic summaries are developed. In this process, special care is taken to generate readable and sensible summaries. Finally, we present three applications which utilize these procedures to fulfill various information-seeking needs.

Natürliche Sprache - ein multimedialer Träger von Information InfoPort - ein Projekt zur Überbrückung von Medienbrüchen bei der Verarbeitung sprachlicher Information

Es wird ein Projekt vorgestellt, das zum Ziel eine medienunabhangige Verarbeitung sprachlicher Information hat. Sprachliche Information erscheint in geschriebener oder gesprochener Form (Medien: Papier, Fax, elektonischer Text, e-mail, voice-mail, Telefon,…). Die Einsatzgebiete sind Retrieval, aktive Informationsvermittlung und Assistenz. Bereits realisierte Anwendungen liegen in den Bereichen Analyse von schriftlichen Anfragen (Geschaftsberichte), telefonische Auskunftssysteme und Datenbankzugriff (STORM). Die eingesetzten Techniken sind einerseits signalnahe Mustererkennungsalgorithmen zum Hypothetisieren von Wortern aus Bildern oder dem Sprachsignal (Dokumentbild- analyse, OCR, HMM), anderseits wissensbasierte Techniken zur Interpretation der sprachlichen Information. Eine robuste Verarbeitung verlangt eine enge Verzahnimg von Erkennung und Interpretation. Auch eine bruchteilhafte Erkennung mus interpretiert werden.