Carol C. Lochbaum

Formative design evaluation of superbook

SuperBook is a hypertext browsing system designed to improve the usability of conventional documents. Successive versions of SuperBook were evaluated in a series of behavioral studies. Students searched for information in a statistics text. presented either in conventional printed form or in SuperBook form. The best version of SuperBook enabled students to answer search questions more quickly and accurately than they could with the conventional text. Students wrote higher quality “open-book” essays using SuperBook than they did with the conventional text, and their subjective ratings of the documentation strongly favored SuperBook. This work is a case study of formative design-evaluation. Behavioral evaluation of the first version of SuperBook showed how design factors and user strategies affected search and established baseline performance measures with printed text. The second version of SuperBook was implemented with the goal of improving search accuracy and speed. User strategies that had proved effective in the first study were made very easy and attractive to use. System response time for common operations was greatly improved. Behavioral evaluation of the new SuperBook demonstrated its superiority to printed text and suggested additional improvements that were incorporated into “MiteyBook,” a SuperBook implementation for PC-size screens. Search with MiteyBook proved to be approximately 25 percent faster and 25 percent more accurate than that obtained with a conventional printed book.

Behavioral evaluation and analysis of a hypertext browser

Students performed a variety of tasks using a statistics text presented either in conventional printed form or via the text browser “SuperBook” (Remde, Gomez and Landauer [18]). Students using SuperBook answered more search questions correctly, wrote higher quality “open-book” essays, and recalled certain incidental information better than students using the conventional text. Subjective ratings overwhelmingly favored SuperBook. The advantage of SuperBook appears to be particularly strong for questions that are not anticipated by the authors organization of a text.

Hypertext for the electronic library?: CORE sample results

The Chemistry Online Retrieval Experiment, or CORE project, is studying the possibility of creating a useful, usable electronic library for chemistry researchers. In a preliminary study, chemists were observed performing five different tasks representative of typical uses of the scientific journal literature. The tasks simulated browsing journals, answering specific questions given a citation to an article, answering specific questions given no citation, writing essays to summarize and integrate information, and finding “analogous transformations” for chemical reactions. Chemists carried out these tasks using one of three systems: (a) t.lheprinted journals supplemented with a widely used printed index system, (b) a hypertext system (the SuperBook@ document browser), or (c) a new electronic system (Pixlook) that incorporates traditional document retrieval methods plus full text indexing and delivers bitmap images of journal pages. Both electronic systems had a large advantage over the printed system for search and essay tasks. SuperBook users were faster and more accurate than Pixlook users at finding information relevant to browsing and search topics. Certain SuperBook hypertext features, however, did not work as well as Pixlook for displaying target articles. The patterns of data and log files of subjects suggest how SuperBook, Pixlook and related systems might be improved.

All the Right Words: Finding What You Want as a Function of Richness of Indexing Vocabulary.

The implications of index‐word selection strategies for user success in interactive searching were investigated in two experiments. People were asked to find target information objects using a simple interactive keyword information retrieval system in which the number of referent terms assigned to each object was systematically varied. The results demonstrate that searcher success is markedly improved by greatly increasing the number of names per object.

Recommending from content: preliminary results from an e-commerce experiment

We are conducting an ongoing experiment into the effects of various forms of recommendations on consumer behavior at a web site. In this paper, we report on measures of the usefulness and effectiveness of recommendations based on content. During a three month period, we provided recommendations on over 2000 products at an e-commerce web site. To evaluate the effectiveness of recommendations on customer behavior, we collected three sets of metrics. First, we measured the rate at which recommendations were actually viewed by visitors. Second, we analyzed the paths visitors took through the recommendations. Finally, we measured the impact of recommended items on number of items purchased and on revenue.