Tsuyoshi Kitani

BMIR-J2: a test collection for evaluation of Japanese information retrieval systems

BMIR-J2 is the first complete test collection generally available for evaluating Japanese information retrieval systems. BMIR-J2 features include a novel division of search requests based on various functions required to perform successful retrieval. BMIR-J2 and its smaller predecessor BMIR-J1 were constructed by a volunteer-based working group under the Information Processing Society of Japan. We hope that BMIR-J2 will come into wide use and that it will foster the development of Japanese IR systems.

Lessons from BMIR-J2: a test collection for Japanese IR systems

BMIR-JP is the lirat complete Japanese test collection available for use in evaluating information retrieval systems. It contains sixty queries and the IDS of 5080 newspaper articles in the fields of economics and engineering. The queries are classified into five categories, based on the functions the system is likely to use to interpret them correctly and retrieve relevant texts. This collection has two levels of relevance, topically relevant and partially relevant. Also discussed are design issues such as collection types and size. This collection and the principles derived in designing it should be helpful in the future development of new test collections.

Requirements clinic: Third party inspection methodology and practice for improving the quality of software requirements specifications

We have been involved in a number of large-scale software development projects, which might lead to loss of millions of dollars if failed. The quality of SRS (Software Requirements Specification) is the key to success of the software development. Review and inspection are common practices for the verification and validation of SRS. However, verification techniques used in projects might be characterized as ad hoc. In this article, we propose requirements clinic, a third party inspection methodology for improving the quality of the SRS. In order to systematically inspect a SRS, we developed a perspective-based inspection methodology based on PQM (Pragmatic Quality Model) of SRS. PQM is derived from IEEE Std. 830 from the perspective of pragmatic quality. To inspect a SRS according to PQM, we identified 198 inspection points, which lead to a quality score between 0 and 100. The requirements clinic advises to the requirements engineering team by a comprehensive quality inspection report including quality score, benchmark and SRS patterns for improvement. Since 2010, we have been practicing the methodology to a variety of development projects, and revealed an average of 10.6 ROI in 12 projects. We also discuss the feasibility of the methodology and lessons learned from the practices.

NTT DATA: DESCRIPTION OF THE ERIE SYSTEM USED FOR MUC-6

Erie is a name recognition system developed for the Multilingual Entity Task (MET) in MUC-6. The pattern matching engine recognizes organization, person, and place names along with time and numeric expressions in Japanese text. Although our previous information extraction system Tex-tract performed well in MUC-5, the pattern matching engine, which was written in AWK language, was slow[2]. System maintenance was also difficult, since the patterns were defined in both the matching engine and the pattern files. Erie solves these problems by generating a pattern matching engine in C language directly from the defined patterns.

Empirical Analysis of the Impact of Requirements Traceability Quality to the Productivity of Enterprise Applications Development

The aim of our research is to empirically analyze the impact of requirements trace ability quality to the productivity of the enterprise applications development. We analyze trace ability links of Requirements Specification Documents (RSDs) collected from the enterprise applications development. These RSDs are classified into two groups in terms of project performance. One group includes RSDs of significant cost overrun projects, another group includes RSDs of normal range cost projects. Our analysis revealed that high quality of trace ability in RSDs significantly reduces the cost overrun in the enterprise applications development.

Efficient search server assignment in a disproportionate system environment

An efficient method is proposed for assigning jobs to search servers in cluster-type system environments when search server performance levels diier. The assignment is done by estimating the search time for each query, based on factors such as the number of databases to be retrieved by the query and the performance of the servers. Search processing efficiency is improved by assigning time-consuming jobs to high-performance servers, regardless of the order of request arrival. Experimental results showed a 10 to 50% shortening of the overall response time.