Gordon W. Paynter

Scalable browsing for large collections: a case study

Phrase browsing techniques use phrases extracted automatically from a large information collection as a basis for browsing and accessing it. This paper describes a case study that uses an automatically constructed phrase hierarchy to facilitate browsing of an ordinary large Web site. Phrases are extracted from the full text using a novel combination of rudimentary syntactic processing and sequential grammar induction techniques. The interface is simple, robust and easy to use. To convey a feeling for the quality of the phrases that are generated automatically, a thesaurus used by the organization responsible for the Web site is studied and its degree of overlap with the phrases in the hierarchy is analyzed. Our ultimate goal is to amalgamate hierarchical phrase browsing and hierarchical thesaurus browsing: the latter provides an authoritative domain vocabulary and the former augments coverage in areas the thesaurus does not reach.

Automatic extraction of document keyphrases for use in digital libraries: evaluation and applications

This article describes an evaluation of the Kea automatic keyphrase extraction algorithm. Document keyphrases are conventionally used as concise descriptors of document content, and are increasingly used in novel ways, including document clustering, searching and browsing interfaces, and retrieval engines. However, it is costly and time consuming to manually assign keyphrases to documents, motivating the development of tools that automatically perform this function. Previous studies have evaluated Keas performance by measuring its ability to identify author keywords and keyphrases, but this methodology has a number of well-known limitations. The results presented in this article are based on evaluations by human assessors of the quality and appropriateness of Kea keyphrases. The results indicate that, in general, Kea produces keyphrases that are rated positively by human assessors. However, typical Kea settings can degrade performance, particularly those relating to keyphrase length and domain specificity. We found that for some settings, Keas performance is better than that of similar systems, and that Keas ranking of extracted keyphrases is effective. We also determined that author-specified keyphrases appear to exhibit an inherent ranking, and that they are rated highly and therefore suitable for use in training and evaluation of automatic keyphrasing systems.

Lexically-generated subject hierarchies for browsing large collections

Developing intuition for the content of a digital collection is difficult. Hierarchies of subject terms allow users to explore the space of topics that a collection covers, to form and specialize useful query terms, and to directly identify interesting documents. We describe two interfaces for navigating such hierarchies, and present a technique for inferring hierarchies automatically from large corpora. We also discuss scalability issues for the techniques involved, and our solutions to these problems.

Human evaluation of Kea, an automatic keyphrasing system

This paper describes an evaluation of the Kea automatic keyphrase extr action algorithm. Tools that automatically identify keyphrases are desirable because document keyphrases have numerous applications in digital library systems, but are costly and time consuming to manually assign. Keyphrase extraction algorithms are usually evaluated by comparison to author-specified keywords, but this methodology has several well-known shortcomings. The results presented in this paper are based on subjective evaluations of the quality and appropriateness of keyphrases by human assessors, and make a number of contributions. First, they validate previous evaluations of Kea that rely on author keywords. Second, they show Keas performance is comparable to that of similar systems that have been evaluated by human assessors. Finally, they justify the use of author keyphrases as a performance metric by showing that authors generally choose good keywords.

A Combined Phrase and Thesaurus Browser for Large Document Collections

A browsing interface to a document collection can be constructed automatically by identifying the phrases that recur in the full text of the documents and structuring them into a hierarchy based on lexical inclusion. This provides a good way of allowing readers to browse comfortably through the phrases (all phrases) in a large document collection. A subject-oriented thesaurus provides a different kind of hierarchical structure, based on deep knowledge of the subject area. If all documents, or parts of documents, are tagged with thesaurus terms, this provides a very convenient way of browsing through a collection. Unfortunately, manual classification is expensive and infeasible for many practical document collections. This paper describes a browsing scheme that gives the best of both worlds by providing a phrase-oriented browser and a thesaurus browser within the same interface. Users can switch smoothly between the phrases in the collection, which give access to the actual documents, and the thesaurus entries, which suggest new relationships and new terms to seek.

Generalising programming by demonstration

Programming by demonstration (PBD) is a term that describes a wide range of end-user programming techniques that generate code from the users examples. Most systems that use PBD rely on knowledge of a specific application (domain knowledge) and on ad hoc inferencing techniques to make sense of the users actions. This paper argues that it is possible and practical to build a PBD system that incorporates domain knowledge without being restricted to a single application and that uses general purpose learning algorithms. A methodology for designing such a system and the authors current work in the area are described.

Domain-independent programming by demonstration in existing applications

Publisher Summary Domain independence is of huge benefit because it eliminates the difficult, time-consuming, and error-prone job of encoding domain knowledge in a satisfactory way, and it eliminates the brittleness that is associated with unexpected interactions between different pieces of domain knowledge. The ability to work with existing application programs is highly desirable for any interface agent because it eliminates the restriction that users can only use selected applications and removes the need to re-implement applications. This chapter describes “Familiar,” a domain-independent programming-by demonstration system for automating iterative tasks in existing, unmodified applications on a popular commercial platform. Familiar requires no domain knowledge from the developer and works immediately with new applications as soon as they are installed. Based on the Apple Script language, the system demonstrates that commercial operating systems are mature enough to support practical, domain-independent programming by demonstration- but only just, for the work exposes many deficiencies.