Sachio Hirokawa

Testbed for information extraction from deep web

Search results generated by searchable databases are served dynamically and far larger than the static documents on the Web. These results pages have been referred to as the Deep Web. We need to extract the target data in results pages to integrate them on different searchable databases. We propose a test bed for information extraction from search results. We chose 100 databases randomly from 114,540 pages with search forms. Therefore, these databases have a good variety. We selected 51 databases which include URLs in a results pageand manually identify target information to be extracted. We also suggest evaluation measures for comparing extraction methods and methods for extending the target data.

Principal types of BCK-lambda-terms

Abstract BCK-λ-terms are the λ-terms in which each variable occurs at most once. The principal type of a λ-term is the most general type of the term. In this paper we prove that if two BCK-λ-terms in β-normal form have the same principal type then they are identical. This solves the following problem (Y. Komori, 1987) in more general form: if two closed βη-normal form BCK-λ-terms are assigned to the same minimal BCK-formula, are they identical? A minimal BCK-formula is the most general formula among BCK-provable formulas with respect to substitutions for type variables. To analyze type assignment, the notion of “connection” is introduced. A connection is a series of occurrences of a type in a type assignment figure. Connected occurrences of a type have the same meaning. The occurrences of the type in distinct connection classes can be rewritten separately; as a result, we have more general type assignment. By “formulae-as-types” correspondence, the result implies the uniqueness of the normal proof figure for principal BCK-formulas. The result is valid for BCI-logic or implicational fragment of linear logic as well.

Automatic Wrapper Generation for Multilingual Web Resources

We present a wrapper generation system to extract contents of semi-structured documents which contain instances of a record. The generation is done automatically using general assumptions on the structure of instances. It outputs a set of pairs of left and right delimiters surrounding instances of a field. In addition to input documents, our system also receives a set of symbols with which a delimiter must begin or end. Our system treats semi-structured documents just as strings so that it does not depend on markup and natural languages. It does not require any training examples which show where instances are. We show experimental results on both static and dynamic pages which are gathered from 13 Web sites, markuped in HTML or XML, and written in four natural languages. In addition to usual contents, generated wrappers extract useful information hidden in comments or tags which are ignored by other wrapper generation algorithms. Some generated delimiters contain whitespaces or multibyte characters.

Dynamic Thesaurus Construction from English-Japanese Dictionary

We propose a method that constructs a hierarchy of words from a set of given documents automatically and dynamically. Given a query, the system retrieves the set of documents that satisfy the query, and the related words are extracted automatically, according their document frequencies. Then a hypernym relations of these related words are obtained using co-occurrence frequencies. Empirical evaluation are conducted for the word hierarchy derived from an English-Japanese dictionary, where the descriptions of a word is considered as one documents. It is shown that meaningful structures are obtained. It is also shown that the proposed method generates more fine hierarchy compared to that obtained by Niwapsilas and Srinivasanpsilas method.

Feature words that classify problem sentence in scientific article

Literature review requires understanding the contents from several view points, such as the problem and the method that the articles describe. Search from these viewpoints will improve the efficiency of survey, if particular segments of articles were extracted, indexed and can be used as auxiliary query. This paper focuses on sentences that describe the problem in an abstract and the feature sets that classify such problem sentences. Classification performance are evaluated by 10-fold cross-validation for six candidate sets of feature words. It turned out that the set of all words gains the best performance if 90% of the data are used as training data. However, the set of a small number of words with positive scores outperforms other feature sets, if the training data is only 10%. In such a realistic situation, the feature words are effective in improving classification performance.

An Approach to Analyzing Correlation between Songs/Artists Using iTMS Playlists

Digital audio devices have been changing music entertainment environment. Those devices are bundled with music jukebox software, such as Apples iTunes, Sonys CONNECT player. Jukebox software not only enables us to recode, play, search, purchase music on PC, but also to manage playlist. Anybody can make his/her own playlists, and play music according to the list. In this paper, we focus on iTMS (iTunse Music Stroe) playlists and use them as the data mining resources for a music recommendation system, then developing correlation measuring methods. We have retrieved about 13,000 playlists, and analyzed the frequency of artists/songs, co-occurrence of artists/songs in the playlists. Through the result data, we found out that all graphs we drew are follow the Zipfs law. Furthermore, we have analyzed the hierarchical relation between songs according to their popularity. We proposed a basic idea of popularity measuring method

A Predictive Model to Evaluate Student Performance

In this paper we propose a new approach based on text mining techniques for predicting student performance using LSA (latent semantic analysis) and K-means clustering methods. The present study uses free-style comments written by students after each lesson. Since the potentials of these comments can reflect student learning attitudes, understanding of subjects and difficulties of the lessons, they enable teachers to grasp the tendencies of student learning activities. To improve our basic approach using LSA and k-means, overlap and similarity measuring methods are proposed. We conducted experiments to validate our proposed methods. The experimental results reported a model of student academic performance predictors by analyzing their comments data as variables of predictors. Our proposed methods achieved an average 66.4% prediction accuracy after applying the k-means clustering method and those were 73.6% and 78.5% by adding the overlap method and the similarity measuring method, respectively.

Normal Proofs and Their Grammar

We present grammatical (or equational) descriptions of the set of normal inhabitants {M|??M:A,Min?-normal form} of a given typeAunder a given basis?, both for the standard simple type system (in the partial discharge convention) and for the system in the total discharge convention (or the Prawitz-style natural deduction system). It is shown that in the latter system we can describe the set by a (finite) context-free grammar, but for the standard system this is not necessarily the case because we may need an infinite supply of fresh (bound) variables to describe the set. In both cases, however, our grammars reflect the structure of normal inhabitants in such a way that, when non-terminals are ignored, a derivation tree of the grammars yielding a?-termMcan be identified with B?hm tree ofM. We give some applications of the grammatical descriptions. Among others, we give simple algorithms for the emptiness/finiteness problem of the set of normal inhabitants of a given type (both for the standard and nonstandard systems).

Formal concept analysis of medical incident reports

It is known that a lot of incidents has happened ahead of a serious accident. Such experiences have been collected in medical sites as incident reports. The text mining is expected as a method that discovers the factors of incidents and the improvement of the situation. This paper proposes a method to analyse the co-occurrence relation of the words that appear in the medical incident reports using concept lattice.

A Hybrid Trust Degree Model in Social Network for Recommender System

Recommender system is an effective way to help users to find the required information. In the social network, the recommendation is often from one user to another user. Therefore, it is necessary to determine how the two users trust each other. However, much work has paid more attention to the one-to-one trust relationship but ignored the many-to-one relationship. In this paper, we proposed a hybrid trust degree model to describe how two users trust each other. This model not only considers the direct trust degree and indirect trust degree between the two users, but also considers the group trust degree. The group trust degree describes how a user are trusted by other users in a group. The experiment result shows that hybrid trust degree can reasonably measure and calculate the credit between two users in a group.