Elmarhomy Ghada

Improvement of building field association term dictionary using passage retrieval

Field Association (FA) terms are a limited set of discriminating terms that can specify document fields. Document fields can be decided efficiently if there are many relevant FA terms in that documents. An earlier approach built FA terms dictionary using a WWW search engine, but there were irrelevant selected FA terms in that dictionary because that approach extracted FA terms from the whole documents. This paper proposes a new approach for extracting FA terms using passage (portions of a document text) technique rather than extracting them from the whole documents. This approach extracts FA terms more accurately than the earlier approach. The proposed approach is evaluated for 38,372 articles from the large tagged corpus. According to experimental results, it turns out that by using the new approach about 24% more relevant FA terms are appending to the earlier FA term dictionary and around 32% irrelevant FA terms are deleted. Moreover, precision and recall are achieved 98% and 94% respectively using the new approach.

A new algorithm for construction specific field terms using co-occurrence words information

Readers can know the subject of many document fields by reading only some specific words called field association (FA) terms. It is very important to construct these FA terms to decide correctly the document fields from few words information in part of file. The field can be decided efficiency if the number of these FA terms is many and the frequency rate is high. If the number of level I (words that direct connect to terminal fields) FA word is limited, old methods can not determine the documents tiled easily and fast, special when there is a small number of corpus documents. This paper proposes a new method for deciding FA terms using the weight of co-occurrence words and declinable words which related to a narrow association category with eliminating FA terms ambiguity. Moreover, efficient FA terms are difficult to be extracted only by the information of the frequency of them. This paper proposed a new efficient method using new cooccurrence words weight which makes precision and recall are higher than the case of degree of frequency.

A compact memory space of dynamic full-text search using Bi-gram index

Full-text search is widely used for various services of the Internet. A more high-speed and a more efficient full-text search technology are necessary because of the amount of increasing handled document and corresponding document data every day. This work proposes an adaptive block management algorithm that is efficient for dynamic, data management method. This algorithm is applied for inverted file searching. The new method is speeding up character string retrieval by first making the full-text search of Uni-gram and by the full-text search of Bi-gram. This work proposes a method of enhancing the static full-text search system of Bi-gram to the dynamic full-text search system of Bi-gram. Moreover, This work presents an efficient achievement method of the dynamic full-text search system of Bi-gram using effectiveness of the adaptive block management structure.

An automatic filtering method for field association words by deleting unnecessary words

Document classification and summarization are very important for document text retrieval. Generally, humans can recognize fields such as ⟨Sports⟩ or ⟨Politics⟩ based on specific words called Field Association (FA) words in those document fields. The traditional method causes misleading redundant words (unnecessary words) to be registered because the quality of the resulting FA words depends on learning data pre-classified by hand. Therefore recall and precision of document classification are degraded if the classified fields classified by hand are ambiguous. We propose two criteria: deleting unnecessary words with low frequencies, and deleting unnecessary words using category information. Moreover, using the proposed criteria unnecessary words can be deleted from the FA words dictionary created by the traditional method. Experimental results showed that 25% of 38 372 FA word candidates were identified as unnecessary and deleted automatically when the presented method was used. Furthermore, precision and F-measure were improved by 26% and 15%, respectively, compared with the traditional method.

Knowledge discovery method to accomplish English document classification

Although there is much research of text classification based on vector spaces using word information in the whole text, generally humans can recognize the field by finding the specific words. This paper describes what is field-associated term and how to discover field-associated terms, which exist in any text. In this paper, such words are called a field association (FA) word that can be directly related to the field classification. Five criteria of FA terms are defined for hierarchical fields. All of them are stored to field tree to make use of extraction of field-coherent passages for document classification. The presented approach is estimated by the simulation results of 140 fields text files of sports field and extended by 197 text field of civil engineering.

Building new field association term candidates automatically by search engine

With increasing popularity of the Internet and tremendous amount of on-line text, automatic document classification is important for organizing huge amounts of data. Readers can know the subject of many document fields by reading only some specific Field Association (FA) words. Document fields can be decided efficiently if there are many FA words and if the frequency rate is high. This paper proposes a method for automatically building new FA words. A WWW search engine is used to extract FA word candidates from document corpora. New FA word candidates in each field are automatically compared with previously determined FA words. Then new FA words are appended to an FA word dictionary. From the experiential results, our new system can automatically appended around 44% of new FA words to the existence FA word Dictionary. Moreover, the concentration ratio 0.9 is also effective for extracting relevant FA words that needed for the system design to build FA words automatically.

A new approach for automatic building field association words using selective passage retrieval

Large collections of full-text document are now commonly used in automated information retrieval. When the stored document texts are long, the retrieval of complete documents may not be in the users’ best interest and extract Filed Association (FA) words is not accurate. In such circumstances, efficient and effective retrieval FA words may be obtained by using passage retrieval strategies designed to retrieve text excerpts of varying size in response to statements of user interest. New approaches are described in this study for implementing selective passage retrieval systems, and identifying text passage response to particular user needs. Moreover an automated system is using for extract accurate FAwords from that passage and evaluate the usefulness of the proposed method. From the experimental results, when passage retrieval are accessible leading to the retrieval of additional extracted relevant FA word with corresponding improvements in Recall and Precision. Therefore, Recall and Precision improved by 30% than using whole texts and traditional methods.

A new approach for improving field association term dictionary using passage retrieval

Large collections of full-text document are now commonly used in automated information retrieval Readers generally identify the subject of a text when they notice specific terms, called Field Association (FA) terms, in that text. Previous researches showed that evidence from passage can improve retrieval results by dividing documents into coherent units with each unit corresponding to a subtopic. Moreover, many current researchers are extracting FA terms candidates from the whole documents to build FA term dictionary automatically. This paper proposes a method for automatically building new FA term dictionary from documents after using passage retrieval. A WWW search engine is used to extract FA terms candidates from passage document corpora. Then, new FA terms candidates in each field are automatically compared with previously determined FA terms dictionary. Finally, new FA terms from extracted term candidates are appended automatically to the existence FA terms dictionary. From experimental results the new technique using passage documents can automatically append about 15% of FA terms from terms candidates to the existence FA term dictionary over the old method. Moreover, Recall and Precision significantly improved by 20% and 32% over the traditional method. The proposed methods are applied to 38,372 articles from the large tagged corpus.

New Hierarchy Technique Using Co-Occurrence Word Information

By the development of the computer in recent years, calculating a complex advanced processing at high speed has become possible. Moreover, a lot of linguistic knowledge is used in the natural language processing system for improving the system. Therefore, the necessity of co-occurrence word information in the natural language processing system increases further and various researches using co-occurrence word information are done. Moreover, in the natural language processing, dictionary is necessary and indispensable because the ability of the entire system is controlled by the amount and the quality of the dictionary. In this paper, the importance of co-occurrence word information in the natural language processing system was described. The classification technique of the co-occurrence word (receiving word) and the co-occurrence frequency was described and the classified group was expressed hierarchically. Moreover, this paper proposes a technique for an automatic construction system and a complete thesaurus. Experimental test operation of this system and effectiveness of the proposal technique is verified.