Tatsunori Mori

A simple but powerful automatic term extraction method

In this paper, we propose a new idea for the automatic recognition of domain specific terms. Our idea is based on the statistics between a compound noun and its component single-nouns. More precisely, we focus basically on how many nouns adjoin the noun in question to form compound nouns. We propose several scoring methods based on this idea and experimentally evaluate them on the NTCIRI TMREC test collection. The results are very promising especially in the low recall area.

Information gain ratio as term weight: the case of summarization of IR results

This paper proposes a new term weighting method for summarizing documents retrieved by IR system. Unlike query-biased summarization, our method utilizes not the information of query, but the similarity information among original documents by hierarchical clustering. To map the similarity structure of the clusters into the weight of each word, we adopt the information gain ratio of probabilistic distribution of each word as term weight.

Japanese question-answering system using A* search and its improvement

We have proposed a method to introduce A* search control in a sentential matching mechanism for Japanese question-answering systems in order to reduce the turnaround time while maintaining the accuracy of the answers. Using this method, preprocessing need not be performed on a document database and we may use any information retrieval systems by writing a simple wrapper program. However, the disadvantage is that the accuracy is not sufficiently high and the mean reciprocal rank (MRR) is approximately 0.3 in NTCIR3 QAC1, an evaluation workshop for question-answering systems. In order to improve the accuracy, we propose several measures of the degree of sentence matching and a variant of a voting method. Both of them can be integrated with our system of controlled search. Using these techniques, the system achieves a higher MRR of 0.5 in the evaluation workshop NTCIR4 QAC2.

Answering Any Class of Japanese Non-factoid Question by Using the Web and Example Q&A Pairs from a Social Q&A Website

In this paper, we propose a method of non-factoid Web question-answering that can uniformly deal with any class of Japanese non-factoid question by using a large number of example Q&A pairs. Instead of preparing classes of questions beforehand, the method retrieves already asked question examples similar to a submitted question from a set of Q&A pairs. Then, instead of preparing clue expressions for the writing style of answers according to each question class beforehand, it dynamically extracts clue expressions from the answer examples corresponding to the retrieved question examples. This clue expression information is combined with topical content information from the question to extract appropriate answer candidates. The experimental results showed that the clue expressions obtained from the set of examples improved the accuracy of answer candidate extraction.

Using web page layout for extraction of sender names

Recently, the credibility of information available on the Web has been regarded as an important issue. Sender name is one of the important indicators of the credibility of the information. In this paper, we propose a new method for extracting sender name. The proposed method use the named entity recognition method, and reducing the DOM node using Web page Layout for preprocessing. Experimental result shows that our proposed method can effectively extract sender names when the preprocessing is successful.

Constraints and defaults on zero pronouns in Japanese instruction manuals

In this paper, we propose a method for anaphora resolution of zero subjects in Japanese manual sentences based on both the nature of language expressions and the ontology of ordinary instruction manuals. In instruction manuals written in Japanese, zero subjects often introduce ambiguity into sentences. In order to resolve them, we consider the property of several types of expressions including some forms of verbal phrases and some conjunctives of clauses, and so on. As the result, we have a set of constraints and defaults for zero subject resolution. We examine the precision of the constraints and defaults with real manual sentences, and we have the result that they make a good estimate with precision of over 80%.

Answering contextual questions based on the cohesion with knowledge

In this paper, we propose a Japanese question-answering (QA) system to answer contextual questions using a Japanese non-contextual QA system. The contextual questions usually contain reference expressions to refer to previous questions and their answers. We address the reference resolution in contextual questions by finding the interpretation of references so as to maximize the cohesion with knowledge. We utilize the appropriateness of the answer candidate obtained from the non-contextual QA system as the degree of the cohesion. The experimental results show that the proposed method is effective to disambiguate the interpretation of contextual questions.

Zero pronouns and conditionals in Japanese instruction manuals

This paper proposes a method of the zero pronoun resolution, which is one of the essential processes in understanding systems for Japanese manual sentences. It is based on pragmatic properties of Japanese conditionals. We examined a number of sentences appearing in Japanese manuals according to the classification based on the types of agent and the types of verb phrase. As a result, we obtained the following pattern of usage in matrix clauses: 1) The connective particles TO and REBA have the same distribution of usage. TARA and NARA have the same distribution of usage. 2) The distribution of usage of TO and REBA, and that of TARA and NARA are complementary to each other. We show that these distributions of usage can be used for resolution of zero subjects.

Answering Contextual Questions Based on the Cohesion with Knowledge

In this paper, we propose a Japanese question-answering (QA) system to answer contextual questions using a Japanese non-contextual QA system. The contextual questions usually contain reference expressions to refer to previous questions and their answers. We address the reference resolution in contextual questions by finding the interpretation of references so as to maximize the cohesion with knowledge, i.e., information source like document collections. We utilize the appropriateness of the answer candidate obtained from the non-contextual QA system as the degree of the cohesion. The experimental results show that the proposed method is effective to disambiguate the interpretation of contextual questions.

Preface to the special issues on NTCIR-4

The ACM TALIP Special Issues on NTCIR-4 contain fourteen papers selected from the papers submitted by researchers involved in the Fourth NTCIR Workshop (NTCIR-4). The NTCIR Workshops are a series of evaluation workshops designed to enhance research in information access technologies such as information retrieval (IR), question answering, summarization, text mining, and so on, by providing large-scale evaluation infrastructures and a forum for researchers interested in cross-system comparisons and in exchanging research ideas in an informal atmosphere. Because fundamental text processing, such as indexing, includes language-dependent procedures, the NTCIR project began in late 1997, and has placed emphasis on East Asian languages such as Japanese, Chinese, Korean, (and English documents published in Asia), and its series of workshops has attracted international participation. An NTCIR workshop is held about once every one and a half years. Because we respect all the interactions among the participants, we consider the entire process from initial document release to the final meeting as the workshop. Each workshop selects several research areas, called “tasks,” or “challenges” for the more challenging tasks. A task may consist of more than one subtask. From the beginning of the project, the tasks were selected with a focus along two directions: (a) laboratory-type testing of IR systems, and (b) evaluating challenging technologies. For testing IR systems, we placed emphasis on East Asian languages and on testing various document genres. For the challenging issues, we looked at the technologies that utilize “information” in documents; the intersection of IR and natural language processing; and at the methodologies and metrics that provide more realistic and reliable evaluations, with special attention to users’ information-seeking tasks. The NTCIR-4 focused on five tasks: Cross-Lingual Information Retrieval (CLIR); Patent Retrieval (PATENT); Question Answering (QAC); Text Summarization (TSC);