Keinosuke Matsumoto

Passage Retrieval Based on Density Distributions of Terms and Its Applications to Document Retrieval and Question Answering

A huge amount of electronic documents has created the demand of intelligent access to their information. Document retrieval has been investigated for providing a fundamental tool for the demand. However, it is not satisfactory due to (1) inaccuracies of retrieving long documents with short queries (a few terms), (2) a user’s burden on finding relevant parts from retrieved long documents. In this paper, we apply a passage retrieval method called “density distributions” (DD) to tackle these problems. For the first problem, it is experimentally shown that a passage-based method outperforms conventional document retrieval methods if long documents are retrieved with short queries. For the second problem, we apply DD to the question answering task: locating short passages in response to natural language queries of seeking facts. Preliminary experiments show that correct answers can be located within a window of 50 terms for about a half of such queries.

Experimental evaluation of passage-based document retrieval

Retrieval of electronic documents is a fundamental component for intelligent access to the contents of documents. For the retrieval of long documents, a method called passage-based document retrieval has proven to be effective. In this paper we experimentally show that the passage-based retrieval is also advantageous for dealing with short queries on condition that documents are long. We employ a passage-based method based on density distributions of query terms in documents, and compare it with three conventional methods: the vector space model, pseudo-feedback and latent semantic indexing.

An interactive approach to demand side management based on utility functions

Computational markets have been suggested as a solution to resource allocation, scheduling, and optimization problems. This paper describes how computational markets can be used to handle the application of demand side management. Demand side management enables electric power companies to reduce peak loads and thereby save money. A load is any device that consumes electric energy in this situation. Due to the large number of different loads, demand side management is a complicated optimization problem. The authors present a new interactive approach to this problem by using utility functions of electric power marketers. This approach provides an online estimation of the functions. In addition to them, it can deal with time-variable tendency of customers.

Passage-Based Document Retrieval as a Tool for Text Mining with User's Information Needs

Document retrieval can be considered as a basic but important tool for text mining that is capable of taking a users information need into account. However, document retrieval is a hard task if multitopic lengthy documents have to be retrieved with a very short description (a few keywords) of the information need. In this paper, we focus on this problem which is typical in real world applications. We experimentally validate that passage-based document retrieval is advantageous in such circumstances as compared to conventional document retrieval. Passage-based document retrieval is a kind of document retrieval which takes into account only small fractions (passages) of documents to judge the document relevance to the information need. As a passage-based method, we employ the method based on density distributions of keywords. This is compared with the following three conventional methods for document retrieval: the vector space model, pseudo-feedback, and latent semantic indexing. Experimental results show that the passage-based method is superior to the conventional methods if long documents have to be retrieved by short queries.

Adaptation to a dynamical environment by means of the environment identifying genetic algorithm

Adaptation to dynamic environments is an important application of genetic algorithms (GAs). However, there are many difficulties to apply the GA to dynamic environments. Especially, in online environments, the GAs defects become remarkable because individuals should be evaluated in the real world. We proposes a novel approach to such an online adaptation called the environment identifying genetic algorithm (EIGA). Computer simulation is carried out by taking an Nk-landscape problem as an example.

On the use of density distribution of keywords for automated generation of hypertext links from arbitrary parts of documents

This paper presents a method of automated generation of hypertext links for electronic documents. The goal is to generate links from an arbitrary part of a document (a source of a link) to its relevant parts of target documents (destinations). To achieve this goal, we assume that words are often shared by parts of documents if these parts are relevant with each other. In order to extract parts densely including words of a source (keywords), we employ density distributions of keywords. This enables us to determine destinations simply by extracting parts whose density exceeds a threshold. Experiments on generating links from figures/tables to parts of documents, as well as from texts to parts of different documents show that our method with the optimal parameters yields recall of 60% and precision of 50%.

Document image segmentation as selection of Voronoi edges

This paper presents a method of document image segmentation for pages with document components of arbitrary shape as well as any skew angles. The characteristics of the proposed method are as follows: the Voronoi diagram is constructed based on the connected components to obtain the candidates of boundaries of document components; and the candidates are utilized to estimate the inter-character, inter-line and inter-column gaps without the use of domain specific parameters so as to select the boundaries. From the experimental results for 80 images with non-Manhattan layout and the skews of 0/spl deg//spl sim/450. We have confirmed that the method is effective for extraction of column regions and as efficient as other methods based on connected component analysis.

Spotting Where to Read on Pages - Retrieval of Relevant Parts from Page Images

This paper presents a new method of document image retrieval that is capable of spotting parts of page images relevant to a users query. This enables us to improve the usability of retrieval, since a user can find where to read on retrieved pages. The effectiveness of retrieval can also be improved because the method is little influenced by irrelevant parts on pages. The method is based on the assumption that parts of page images which densely contain keywords in a query are relevant to it. The characteristics of the proposed method are as follows: (1) Two-dimensional density distributions of keywords are calculated for ranking parts of page images, (2) The method relies only on the distribution of characters so as not to be affected by the errors of layout analysis. Based on the experimental results of retrieving Japanese newspaper articles, we have shown that the proposed method is superior to a method without the function of dealing with parts, and sometimes equivalent to a method of electronic document retrieval that works on error-free text.

A communication network model of electric power trading systems using Web services

Many business models of power trading systems have been proposed to aim at load reduction by consumers cooperating with electric power suppliers in an electric power market. On the other hand, Web services are regarded as a new application paradigm in the world of the Internet. Then, we propose a network model of power trading systems using Web services in this paper. The adaptability of Web services to power trading systems was checked in the prototype of our network model and we got good results in our simulations. Each server provides functions as a SOAP (simple object access protocol) server, and it is coupled loosely with each other through SOAP. Storing SOAP message in HTTP packet can establish a penetration communication way that is not conscious of a firewall. Switching of dynamic servers is possible by means of rewriting the server point information.

Backgrounds as information carriers for printed documents

This paper presents a method of embedding and recovery of a large amount of data on printed documents. The proposed method is characterized by the following points: (1) As the medium for recording the data, backgrounds (white space) of pages are utilized. Since white space is dominant in ordinary pages, this enables us to record a large amount of data. (2) As the representation of data, an arbitrary figure (such as a logo mark of a company) is stippled to obtain a set of points, each of which is superimposed on a page image. This allows us to prevent the appearance of a page from deteriorating, since stippled figures play a role of background patterns on printed documents. The experimental results show that the method is capable of both recording 7.8KB of data and reading the embedded data with the accuracy of 99%.