Masaki Oono

Word classification and hierarchy using co-occurrence word information

By the development of the computer in recent years, calculating a complex advanced processing at high speed has become possible. Moreover, a lot of linguistic knowledge is used in the natural language processing (NLP) system for improving the system. Therefore, the necessity of co-occurrence word information in the natural language processing system increases further and various researches using co-occurrence word information are done. Moreover, in the natural language processing, dictionary is necessary and indispensable because the ability of the entire system is controlled by the amount and the quality of the dictionary. In this paper, the importance of co-occurrence word information in the natural language processing system was described. The classification technique of the co-occurrence word (receiving word) and the co-occurrence frequency was described and the classified group was expressed hierarchically. Moreover, this paper proposes a technique for an automatic construction system and a complete thesaurus. Experimental test operation of this system and effectiveness of the proposal technique is verified.

A compact static double-array keeping character codes

A trie represented by a double-array enables us to search a key fast with a small space. However, the double-array uses extra space to be updated dynamically. This paper presents a compact structure for a static double-array. The new structure keeps character codes instead of indices in order to compress elements of the double-array. In addition, the new structure unifies common suffixes and consists of less elements than the old structure. Experimental results for English keys show that the new structure reduces space usage of the double-array up to 40%.

A sentence classification technique using intention association expressions

Although there are many text classification techniques depending on the vector space, it is difficult to detect the meaning related to the user’s intention (complaint, encouragement, request, invitation, etc.). The approach be discussed in this paper is very useful for understanding focus points in conversation. We present a technique for determining the speaker’s intention for sentences in conversation. Intention association expressions are introduced, and formal descriptions with weights are defined using these expressions to construct an intention classification. A deterministic multi-attribute pattern-matching algorithm is used to determine the intention class efficiently. In simulation results for 681 email messages of 5859 sentences, the multi-attribute pattern-matching algorithm is about 44.5 times faster than the Aho and Corasick method. The precision and recall of intention classification of sentences are 91% and 95%, respectively. The precision and recall of extraction of unnecessary sentences are 98% and 96%, respectively. The precision and recall of the classification of each email are 88% and 89%, respectively.

An efficient deletion method for a minimal prefix double array

A minimal prefix (MP) double array is an efficient data structure for a trie. The MP double array only requires a small amount of space and enables fast retrieval. However, the space efficiency of the MP double array is degraded by deletion. This paper presents a fast and compact adaptive deletion method for the MP double array. The presented method is implemented with C. Simulation results for English and Japanese keys show that the adaptive method is faster than the conventional method and maintains higher space efficiency. Copyright

A fast and compact elimination method of empty elements from a double‐array structure

A double‐array is a well‐known data structure to implement the trie. However, the space efficiency of the double‐array degrades with the number of key deletions because the double‐array keeps empty elements produced by the key deletion. This paper presents a fast and compact elimination method of empty elements using properties of the trie nodes that have no siblings. The present elimination method is implemented by C language. From simulation results for large sets of keys, the present elimination method is about 30–330 times faster than the conventional elimination method and maintains high space efficiency. Copyright

A compression method using link-trie structure for natural language dictionaries

A trie structure is a key search method that is frequently used in the field of natural language systems and database systems. Morita presented an efficient data structure for collocation information using trie structure. This structure stores two basic words into a trie and defines link information by a link function. This paper presents how to apply Moritas method into general key-set and compress trie. Presented method divides a key into several parts and defines link information between keys. From simulation results, it turned out that presented method is 30% smaller than Conventional method in space capacity.

Improvement of the LR parsing table and its application to grammatical error correction

A LR parsing table is generally made use of the parsing process based on the context free grammar for natural languages. Besides the parsing process, it can be used as the index of approximate pattern matching and error correction, because it has the characteristic to be able to predict the next character in the sentence. As for the issue of the traditional LR parsing table, however we can mention if the number of sequences to be processed becomes large, many reduce actions will be created in the parsing table, as a result, it takes a great deal of time to parse the sentence. In this paper, we propose the method to construct a new LR parsing table without reduce actions from the generalized context free grammar. This new parsing table denotes the states to be transited after accepting each symbol. Moreover, we applied this new parsing table to detect and correct erroneous sentences which include the syntax errors, unknown words and misspelling. By using this table, the symbol which is allocated just after the error position can be utilized for selecting correction symbols, as a result, the number of candidates produced on the correction process is reduced, and fast system can be realized. The experiment results, using 1050 sentences including error characters, show that this method can correct error points 69 times faster than the traditional method, also keep the almost same correction accuracy as the traditional method.

Double-array compression by pruning twin leaves and unifying common suffixes

A minimal prefix (MP) trie is a tree structure for key retrieval, and a double-array is an efficient data structure for the MP trie. This paper presents two compression methods for the double-array. One method removes leaf nodes following two-way arcs (named twin leaves) from the MP trie. The other method unifies common suffixes. Experimental results show that space usage of the double-array is reduced to about 60% by the two methods.

A fast and compact technique of implementing transition tables for finite state automata

Abstract A finite state automaton (FA) is a typical and basic machine for computer sciences and how to make the transition tables fast and compact is very important for the whole performance of modules by using the machine. Although a two-dimensional array and a list form are well known as for their representation, which have contrasted features for retrieval time and space efficiency. A.V. Aho and J.D. Ullman [Principles of Compiler Design, Addison-Wesley, Reading, MA, 1986, pp. 74–118] proposed a triple-array method for combining the compactness of the list with the fast retrieval of the array, but there is no discussion to make the structure compact without degrading the fast retrieval. This paper proposes a method of compressing the triple-array structures. Although the approach is not optimal, it does realize a reasonable reduction in size. From the simulation results, it is verified that the presented method can produce fast and compact structures.

An improvement key deletion method for double-array structure using single-nodes

A trie is a well known method for various dictionaries, such as spelling check and morphological analysis. A double-array structure is an efficient data structure combining fast access of a matrix form with the compactness of a list form. The drawback of the double-array is that the space efficiency degrades by empty elements produced in key deletion. Morita presented a key deletion method eliminating empty elements. However, the space efficiency of this method is low for high frequent deletion and deletion takes much time because the cost depends on the number of the empty elements. This paper presents a fast and compact deletion method by using the property of nodes that have no brothers. From simulation results for 100,000 keys, the present method is about 330 times faster than Moritas method and keeps high space efficiency.