Fumito Masui

WoZ Simulation of Interactive Question Answering

QACIAD (Question Answering Challenge for Information Access Dialogue) is an evaluation framework for measuring interactive question answering (QA) technologies. It assumes that users interactively collect information using a QA system for writing a report on a given topic and evaluates, among other things, the capabilities needed under such circumstances. This paper reports an experiment for examining the assumptions made by QACIAD. In this experiment, dialogues under the situation that QACIAD assumes are collected using WoZ (Wizard of Oz) simulating, which is frequently used for collecting dialogue data for designing speech dialogue systems, and then analyzed. The results indicate that the setting of QACIAD is real and appropriate and that one of the important capabilities for future interactive QA systems is providing cooperative and helpful responses.

Detecting article errors based on the mass count distinction

This paper proposes a method for detecting errors concerning article usage and singular/plural usage based on the mass count distinction. Although the mass count distinction is particularly important in detecting these errors, it has been pointed out that it is hard to make heuristic rules for distinguishing mass and count nouns. To solve the problem, first, instances of mass and count nouns are automatically collected from a corpus exploiting surface information in the proposed method. Then, words surrounding the mass (count) instances are weighted based on their frequencies. Finally, the weighted words are used for distinguishing mass and count nouns. After distinguishing mass and count nouns, the above errors can be detected by some heuristic rules. Experiments show that the proposed method distinguishes mass and count nouns in the writing of Japanese learners of English with an accuracy of 93% and that 65% of article errors are detected with a precision of 70%.

Are open-domain question answering technologies useful for information access dialogues?---an empirical study and a proposal of a novel challenge

There are strong expectations for the use of question answering technologies in information access dialogues, such as for information gathering and browsing. In this paper, we empirically examine what kinds of abilities are needed for question answering systems in such situations, and propose a challenge for evaluating those abilities objectively and quantitatively. We also show that existing technologies have the potential to address this challenge. From the empirical study, we found that questions that have values and names as answers account for a majority in realistic information-gathering situations and that those sequences of questions contain a wide range of reference expressions and are sometimes complicated by the inclusion of subdialogues and focus shifts. The challenge proposed is not only novel as an evaluation of the handling of information access dialogues, but also includes several valuable ideas such as categorization and characterization of information access dialogues, and introduces three measures to evaluate various aspects in addressing list-type questions and reference test sets for evaluating context-processing ability in isolation.

A Statistical Model Based on the Three Head Words for Detecting Article Errors

In this paper, we propose a statistical model for detecting article errors, which Japanese learners of English often make in English writing. It is based on the three head words --- the verb head, the preposition, and the noun head. To overcome the data sparseness problem, we apply the backed-off estimate to it. Experiments show that its performance (F-measure=0.70) is better than that of other methods. Apart from the performance, it has two advantages: (i) Rules for detecting article errors are automatically generated as conditional probabilities once a corpus is given; (ii) Its recall and precision rates are adjustable.

An evaluation of question answering challenge (QAC-1) at the NTCIR workshop 3

The Question Answering Challenge (QAC) was carried out as the first evaluation task on question answering of the NTCIR Workshop 3 [Fukumoto2002] [NTCIR]. Question answering in an open domain is a task for obtaining appropriate answers to given domain independent questions written in natural language from a large corpus. The purpose of the QAC was to develop practical QA systems in an open domain focusing on research of user interaction and information extraction. A further objective was to develop an evaluation method for the question answering system and information resources for evaluation. To evaluate QA technologies, there are several technical aspects to consider for the extraction of answer expressions from knowledge sources. Question type is one aspect of the QA system evaluation. In QAC-1, question types are defined as a noun or noun phrase which indicates names of persons, organizations, and various artifacts and facts, such as money, size, date and so on. Moreover, information related to these can also be considered as answer candidates: for example, names of persons, their affiliations, age and status can be an answer; and for names of organizations, their annual profit, year of establishment and so on. Another aspect to consider is how many answer expressions exist in the knowledge sources. In QAC-1, there may be multiple answers or no answers to questions in general. This aspect makes development of a QA system difficult because, QA system has to check all answer candidates very carefully. User interaction technology requires actual interaction between the computer and person. In actual QA between people, there are typically several interactions which take place in order to confirm the intention of the questions and so on. In QAC-1, we gave one follow-up question for the first question. It will be necessary to resolve an ellipsis in the follow-up question which is frequently occurs in Japanese sentences.

Detecting Emotive Sentences with Pattern-based Language Modelling

Abstract This paper presents our research in detection of emotive (emotionally loaded) sentences. The task is defined as a text classification problem with an assumption that emotive sentences stand out both lexically and grammatically. The assumption is verified exper- imentally. The experiment is based on n-grams as well as more sophisticated patterns with disjointed elements. To deal with the sophisticated patterns a novel language modelling algorithm based on the idea of language combinatorics is applied. The results of experiments are explained with the standard means of Precision, Recall and balanced F-score. The algorithm also provides a refined list of most frequent sophisticated patterns typical for both emotive and non-emotive context.

Toward curling informatics — Digital scorebook development and game information analysis

This paper introduces the concept of curling informatics and the digital scorebook system iCE which was developed to support strategies and tactics in curling - a winter team sport played on ice. Our project aims to develop an environment to support curling strategies and tactics by realizing methods to record game information, analyzing, visualizing and sharing the information. We developed the digital scorebook system iCE (intelligent Curling Elicitator) as the first step of our concept. We tested our system by recording real game information at the top-level championships. Consequently, we confirmed that the system performs adequately for practical use. Moreover, we analyzed the recorded information in detail, and the results suggest differences in shot accuracies relating to the differences in game scores. This suggests that our proposed method can effectively support strategic/tactical planning in curling games.

Emotive or Non-emotive: That is The Question

In this research we focus on discriminating between emotive (emotionally loaded) and non-emotive sentences. We define the problem from a linguistic point of view assuming that emotive sentences stand out both lexically and grammatically. We verify this assumption experimentally by comparing two sets of such sentences in Japanese. The comparison is based on words, longer n-grams as well as more sophisticated patterns. In the classification weuse a novelunsupervised learning algorithm based on the idea of language combinatorics. The method reached results comparable to the state of the art, while the fact that it is fully automatic makes it more efficient and language independent.

Advances in Curling Game Information Analysis by Considering Starting Position.

Japanese curling teams have been recently preparing for Pyeongchang Winter Olympics in 2018. In curling, there are three factors influencing game performance: physical factor, human factor, and strategic/tactical factor. The strategic/tactical factor is considered as the most important at top level. To support the strategic/tactical factor, we proposed the concept of Curling Informatics. As the first step of Curling Informatics we developed a digital scorebook iCE for digital collection of game information, storing it in a database and performing further analysis to improve player performance. In this article, we further contribute to this project by analyzing game information of world national top level teams. We have previously confirmed that correlation between shot accuracy and game score could differ with the team level or position. We also found out that selected tactics and psychological pressure on opponent team has impact on game result. However, previous analyses disregarded the order of teams in play, which could result in confusion of strategic tendencies or play characteristics. In this paper, we carried out analysis of correlations of shot scores considering whether the teams started as the first or the second. We did this to specify the process of how the team strategy/tactics influences game results.

Automatic extraction of future references from news using morphosemantic patterns with application to future trend prediction

In everyday life people use past events and their own knowledge to predict future events. In such everyday predictions people use widely available resources (newspapers, Internet). This study focused on sentences referring to the future, such as the one below, as one of such resource.