Timo Honkela

WEBSOM – Self-organizing maps of document collections

Abstract With the WEBSOM method a textual document collection may be organized onto a graphical map display that provides an overview of the collection and facilitates interactive browsing. Interesting documents can be located on the map using a content-directed search. Each document is encoded as a histogram of word categories which are formed by the self-organizing map (SOM) algorithm based on the similarities in the contexts of the words. The encoded documents are organized on another self-organizing map, a document map, on which nearby locations contain similar documents. Special consideration is given to the computation of very large document maps which is possible with general-purpose computers if the dimensionality of the word category histograms is first reduced with a random mapping method and if computationally efficient algorithms are used in computing the SOMs.

Websom for Textual Data Mining

New methods that are user-friendly and efficient are needed for guidanceamong the masses of textual information available in the Internet and theWorld Wide Web. We have developed a method and a tool called the WEBSOMwhich utilizes the self-organizing map algorithm (SOM) for organizing largecollections of text documents onto visual document maps. The approach toprocessing text is statistically oriented, computationally feasible, andscalable – over a million text documents have been ordered on a single map.In the article we consider different kinds of information needs and tasksregarding organizing, visualizing, searching, categorizing and filteringtextual data. Furthermore, we discuss and illustrate with examples howdocument maps can aid in these situations. An example is presented wherea document map is utilized as a tool for visualizing and filtering a stream ofincoming electronic mail messages.

Very Large Two-Level SOM for the Browsing of Newsgroups

On January 19, 1996 we published in the Internet a demo of how to use Self-Organizing Maps (SOMs) for the organization of large collections of full-text files. Later we added other newsgroups to the demo. It can be found at the address http://websom.hut.fi/websom/. In the present paper we describe the main features of this system, called the WEBSOM, as well as some newer developments of it.

Adaptive Ensemble Models of Extreme Learning Machines for Time Series Prediction

In this paper, we investigate the application of adaptive ensemble models of Extreme Learning Machines (ELMs) to the problem of one-step ahead prediction in (non)stationary time series. We verify that the method works on stationary time series and test the adaptivity of the ensemble model on a nonstationary time series. In the experiments, we show that the adaptive ensemble model achieves a test error comparable to the best methods, while keeping adaptivity. Moreover, it has low computational cost.

Self-Organizing Maps of Very Large Document Collections: Justification for the WEBSOM Method

Powerful methods are needed for interactive exploration and search from collections of miscellaneous textual documents that are available in the electronic media. Searching from text documents has traditionally been based on keywords and Boolean expressions. With the WEBSOM method a document collection may be organized into a map display that provides an overview of the collection and facilitates interactive browsing. Interesting documents can be retrieved by a content addressable search. The WEBSOM method is based on using the Self-Organizing Map algorithm for automatically learning relevant structures in the text and for organizing the document collection.

Exploration of full-text databases with self-organizing maps

Availability of large full-text document collections in electronic form has created a need for intelligent information retrieval techniques, especially the expanding World Wide Web which presupposes methods for systematic exploration of miscellaneous document collections. In this paper we introduce a new method, the WEBSOM, for this task. Self-organizing maps (SOMs) are used to represent documents on a map that provides an insightful view of the text collection. This view visualizes similarity relations between the documents, and the display can be utilized for orderly exploration of the material rather than having to rely on traditional search expressions. The complete WEBSOM method involves a two-level SOM architecture comprising of a word category map and a document map, and means for interactive exploration of the database.

Counterintuitiveness as the hallmark of religiosity

Abstract This article presents empirical evidence for the hypothesis that persons consider counterintuitive representations more likely to be religious than other kinds of beliefs. In three studies the subjects were asked to rate the probable religiousness of various kinds of imaginary beliefs. The results show that counterintuitive representations in general, and counterintuitive representations involving a conscious agent in particular, are considered much more likely to be religious. Counterintuitiveness thus seems to be an important element in a folk-understanding of religion.

Learning a taxonomy from a set of text documents

We present a methodology for learning a taxonomy from a set of text documents that each describes one concept. The taxonomy is obtained by clustering the concept definition documents with a hierarchical approach to the Self-Organizing Map. In this study, we compare three different feature extraction approaches with varying degree of language independence. The feature extraction schemes include fuzzy logic-based feature weighting and selection, statistical keyphrase extraction, and the traditional tf-idf weighting scheme. The experiments are conducted for English, Finnish, and Spanish. The results show that while the rule-based fuzzy logic systems have an advantage in automatic taxonomy learning, taxonomies can also be constructed with tolerable results using statistical methods without domain- or style-specific knowledge.

Linguistic feature extraction using independent component analysis

Our aim is to find syntactic and semantic relationships of words based on the analysis of corpora. We propose the application of independent component analysis, which seems to have clear advantages over two classic methods: latent semantic analysis and self-organizing maps. Latent semantic analysis is a simple method for automatic generation of concepts that are useful, e.g., in encoding documents for information retrieval purposes. However, these concepts cannot easily be interpreted by humans. Self-organizing maps can be used to generate an explicit diagram which characterizes the relationships between words. The resulting map reflects syntactic categories in the overall organization and semantic categories in the local level. The self-organizing map does not, however, provide any explicit distinct categories for the words. Independent component analysis applied on word context data gives distinct features which reflect syntactic and semantic categories. Thus, independent component analysis gives features or categories that are both explicit and can easily be interpreted by humans. This result can be obtained without any human supervision or tagged corpora that would have some predetermined morphological, syntactic or semantic information.

Self-Organizing Maps in Symbol Processing

A symbol as such is disassociated from the world. In addition, as a discrete entity a symbol does not mirror all the details of the portion of the world that it is meant to refer to. Humans establish the association between the symbols and the referenced domain – the words and the world – through a long learning process in a community. This paper studies how Kohonen self-organizing maps can be used for modeling the learning process needed in order to create a conceptual space based on a relevant context with which the symbols are associated. The categories that emerge in the self-organizing process and their implicitness are considered as well as the possibilities to model contextuality, subjectivity and intersubjectivity of interpretation.