Alexander M. Robertson

Applications of N-Grams in Textual Information Systems.

This paper provides an introduction to the use of n‐grams in textual information systems, where an n‐gram is a string of n, usually adjacent, characters extracted from a section of continuous text. Applications that can be implemented efficiently and effectively using sets of n‐grams include spelling error detection and correction, query expansion, information retrieval with serial, inverted and signature files, dictionary look‐up, text compression, and language identification.

Effectiveness of query expansion in ranked-output document retrieval systems

This paper reports an evaluation of three methods for the expansion of natural language queries in ranked-out put retrieval systems. The methods are based on term co-oc currence data, on Soundex codes, and on a string similarity measure. Searches for 110 queries in a database of 26,280 titles and abstracts suggest that there is no significant differ ence in retrieval effectiveness between any of these methods and unexpanded searches.

Searching for historical word-forms in a database of 17th-century English text using spelling-correction methods

This paper discusses the application of algorithmic spelling-correction techniques to the identification of those words in a database of 17th century English text that are most similar to a query word in modern English. The experiments have used n-gram matching, non-phonetic coding and dynamic programming methods for spelling correction, and have demonstrated that high-recall searches can be carried out, although some of the searches are very demanding of computational resources. The methods are, in principle, applicable to historical texts in many languages and from many diffeent periods.

Retrieval Of Morphological Variants In Searches Of Latin Text Databases

This paper reports a detailed evaluation of the effectiveness of a system that has been developed for the identification and retrieval of morphological variants in searches of Latin text databases. A user of the retrieval system enters the principal parts of the search term (two parts for a noun or adjective, three parts for a deponent verb, and four parts for other verbs), this enabling the identification of the type of word that is to be processed and of the rules that are to be followed in determining the morphological variants that should be retrieved. Two different search algorithms are described. The algorithms are applied to the Latin portion of the Hartlib Papers Collection and to a range of classical, vulgar and medieval Latin texts drawn from the Patrologia Latina and from the PHI Disk 5.3 datasets. The effectiveness of these searches demonstrates the effectiveness of our procedures in providing access to the full range of classical and post-classical Latin text databases.

Evaluation Of Techniques For The Conflation Of Modern And Seventeenth Century English Spelling

This paper discusses a range of techniques for the identification of those words in a database of 17th-century English text that are most similar to a query word in modern English. The experiments have used n-gram matching, non-phonetic coding and dynamic-programming methods for spelling correction, and have demonstrated that high-recall searches can be carried out, although some of the searches are very demanding of computational resources. It is also shown that neural networks, which seem to be well-suited to use in this context, cannot, in fact, be employed for this application.

‘PhiloFacs’: a Tool for Searching Latin Text Databases

This article discusses the design, production and use of a British Library funded project to provide rs a stemming search engine for Latin text. The resulting software has been christened ‘PhiloFacs’ — short for ‘PhilologiaFacilis’, itself, appropriately enough, a conflation of stems.