Brian Amento

PHOAKS: a system for sharing recommendations

The feasibility of automatic recognition of recommendations is supported by empirical results. First, Usenet messages are a significant source of recommendations of Web resources: 23% of Usenet messages mention Web resources, and ?>0% of these mentions are recommendations. Second, recommendation instances can be machine-recognized with nearly 90% accuracy. Third, some resources are recommended by more than one person. These multiconfirmed recommendations appear to be significant resources for the relevant community. Finally, the number of distinct recommenders of a resource is a tallying, and redistributing recom-

Does “authority” mean quality? predicting expert quality ratings of Web documents

For many topics, the World Wide Web contains hundreds or thousands of relevant documents of widely varying quality. Users face a daunting challenge in identifying a small subset of documents worthy of their attention. Link analysis algorithms have received much interest recently, in large part for their potential to identify high quality items. We report here on an experimental evaluation of this potential. We evaluated a number of link and content-based algorithms using a dataset of web documents rated for quality by human topic experts. Link-based metrics did a good job of picking out high-quality items. Precision at 5 is about 0.75, and precision at 10 is about 0.55; this is in a dataset where 0.32 of all documents were of high quality. Surprisingly, a simple content-based metric performed nearly as well; ranking documents by the total number of pages on their containing site.

Constructing, organizing, and visualizing collections of topically related Web resources

For many purposes, the Web page is too small a unit of interaction and analysis. Web sites are structured multimedia documents consisting of many pages, and users often are interested in obtaining and evaluating entire collections of topically related sites. Once such a collection is obtained, users face the challenge of exploring, comprehending and organizing the items. We report four innovations that address these user needs: (1) we replaced the Web page with the Web site as the basic unit of interaction and analysis;(2) we defined a new informationstructure, the clan graph, that groups together sets of related sites; (3) we augment the representation of a site with a site profile, information about site structure and content that helps inform user evaluation of a site; and (4) we invented a new graph visualization, the auditorium visualization, that reveals important structural and content properties of sites within a clan graph. Detailed analysis and user studies document the utility of this approach. The clan graph construction algorithm tends to filter out irrelevant sites and discover additional relevant items. The auditorium visualization, augmented with drill-down capabilities to explore site profile data, helps users to find high-quality sites as well as sites that serve a particular function.

SCANMail: a voicemail interface that makes speech browsable, readable and searchable

Increasing amounts of public, corporate, and private speech data are now available on-line. These are limited in their usefulness, however, by the lack of tools to permit their browsing and search. The goal of our research is to provide tools to overcome the inherent difficulties of speech access, by supporting visual scanning, search, and information extraction. We describe a novel principle for the design of UIs to speech data: What You See Is Almost What You Hear (WYSIAWYH). In WYSIAWYH, automatic speech recognition (ASR) generates a transcript of the speech data. The transcript is then used as a visual analogue to that underlying data. A graphical user interface allows users to visually scan, read, annotate and search these transcripts. Users can also use the transcript to access and play specific regions of the underlying message. We first summarize previous studies of voicemail usage that motivated the WYSIAWYH principle, and describe a voicemail UI, SCANMail, that embodies WYSIAWYH. We report on a laboratory experiment and a two-month field trial evaluation. SCANMail outperformed a state of the art voicemail system on core voicemail tasks. This was attributable to SCANMails support for visual scanning, search and information extraction. While the ASR transcripts contain errors, they nevertheless improve the efficiency of voicemail processing. Transcripts either provide enough information for users to extract key points or to navigate to important regions of the underlying speech, which they can then play directly

CollaboraTV: making television viewing social again

With the advent of video-on-demand services and digital video recorders, the way in which we consume media is undergoing a fundamental change. People today are less likely to watch shows at the same time, let alone the same place. As a result, television viewing, which was once a social activity, has been reduced to a passive and isolated experience. To study this issue, we developed a system called CollaboraTV and demonstrated its ability to support the communal viewing experience through a month-long field study. Our study shows that users understand and appreciate the utility of asynchronous interaction, are enthusiastic about CollaboraTVs engaging social communication primitives and value implicit show recommendations from friends. Our results both provide a compelling demonstration of a social television system and raise new challenges for social television communication modalities.

Experiments in social data mining: The TopicShop system

Social data mining systems enable people to share opinions and benefit from each others experience. They do this by mining and redistributing information from computational records of social activity such as Usenet messages, system usage history, citations, or hyperlinks. Some general questions for evaluating such systems are: (1) is the extracted information valuable? and (2) do interfaces based on the information improve user task performance? We report here on TopicShop, a system that mines information from the structure and content of Web pages and provides an exploratory information workspace interface. We carried out experiments that yielded positive answers to both evaluation questions. First, a number of automatically computable features about Web sites do a good job of predicting expert quality judgments about sites. Second, compared to popular Web search interfaces, the TopicShop interface to this information lets users select significantly more high-quality sites, in less time and with less effort, and to organize the sites they select into personally meaningful collections more quickly and easily. We conclude by discussing how our results may be applied and considering how they touch on general issues concerning quality, expertise, and consensus.

An empirical evaluation of user interfaces for topic management of Web sites

Topic management is the task of gathering, evaluating,organizing, and sharing a set of web sites for a specific topic.Current web tools do not provide adequate support for this task. Wecreated the TopicShop system to address this need. TopicShopincludes (1) a webcrawler that discovers relevant web sites andbuilds site profiles, and (2) user interfaces for exploring andorganizing sites. We conducted an empirical study comparing userperformance with TopicShop vs. YahooTM. TopicShop subjects foundover 80% more high-quality sites (where quality was determined byindependent expert judgements) while browsing only 8 1% as manysites and completing their task in 89% of the time. The siteprofile data that TopicShop provides - in particular, the number ofpages on a site and the number of other sites that link to it - wasthe key to these results, as users exploited it to identify themost promising sites quickly and easily.

TopicShop: enhanced support for evaluating and organizing collections of Web sites

TopicShop is an interface that helps users evaluate and organize collections of web sites. The main interface components are site profiles, which contain information that helps users select high-quality items, and a work area, which offers thumbnail images, annotation, and lightweight grouping techniques to help users organize selected sites. The two components are linked to allow task integration. Previous work [2] demonstrated that subjects who used TopicShop were able to select significantly more highquality sites, in less time and with less effort. We report here on studies that confirm and extend these results. We also show that TopicShop subjects spent just half the time organizing sites, yet still created more groups and more annotations, and agreed more in how they grouped sites. Finally, TopicShop subjects tightly integrated the tasks of evaluating and organizing sites. INTRODUCTION In previous work [2], we motivated an important task for web users – gathering, evaluating, and organizing information resources for a given topic. Current web tools do not support this task well; specifically, they do not make it easy to evaluate collections of web sites to select the best ones or to organize sites for future reuse and sharing. Users have to browse and view sites one after another until they are satisfied they have a good set, or, more likely, they get tired and give up. Browsing a web site is an expensive operation, both in time and cognitive effort. And bookmarks, the most common form of keeping track of web sites, are a fairly primitive organizational technique. We designed and implemented the TopicShop system to provide comprehensive, integrated support for this task. TopicShop aids users in finding a set of relevant sites, in narrowing down the set into a smaller set of high quality sites, and in organizing sites for future use. TopicShop has evolved through a number of design iterations, driven by extensive user testing. We report here on lessons we learned from a pilot study, how these lessons re-shaped our understanding of the task and led to a significant re-design, and the results of a second, larger user study. RELATED WORK Our research program investigates the major information problems faced by users of the World Wide Web: • finding collections of items relevant to their interests; • identifying high-quality items within a collection; • finding items that contain a certain category of information, e.g., episode guides (for a television show) or song lyrics (for a musician); • organizing personalized subsets of items. We have addressed these problems by developing algorithms, implementing them in web crawling and analysis tools, creating interfaces to support users in exploring, comprehending, and organizing collections of web documents, and performing user studies [2, 3, 4, 15]. The work reported here focuses on understanding the user tasks of evaluating and organizing collections of web sites, as illuminated by the design, evaluation, and re-design of interfaces to support these tasks. Other researchers have investigated these issues. Much recent work has been devoted to algorithms for adding meta-information to collections of web sites to enhance user comprehension, typically by analyzing the structure of links between sites. This approach builds on the intuition that when the author of one site chooses to link to another, this often implies both that the sites have similar content and that the author is endorsing the content of the linked-to site. Pirolli, Pitkow and colleagues [12, 13] experimented with link-based algorithms for clustering and categorizing web pages. Kleinberg’s HITS algorithm [8] defines authoritative and hub pages within a hypertext collection. Authorities and hubs are mutually dependent: a good authority is a page that is linked to by many hubs, and a good hub is one that links to many authorities. After evaluating items and selecting the interesting ones, users must organize the items for future use. Card, Robertson, and Mackinlay [5] introduced the concept of information workspaces to refer to environments in which information items can be stored and manipulated. A departure point for most such systems is the file manager popularized by the Apple Macintosh and then in Microsoft Windows. Such systems typically include a list view, which shows various properties of items, and an icon view, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. UIST ’00. San Diego, CA USA  2000 ACM 1-58113-212-3/00/11...

Building task-specific interfaces to high volume conversational data

5.00 CHI Letters vol 2, 2 201 which lets users organize icons representing the items in a 2D space. Mander, Salomon, and Wong [10] enhanced the basic metaphor with the addition of “piles”. Users could create and manipulate piles of items. Interesting interaction techniques for displaying, browsing, and searching piles were designed and tested. Bookmarks are the most popular way to create personal information workspaces of web resources. Bookmarks consist of lists of URLs; typically the title of the web page is used as the label for the URL. Users may organize their bookmarks into a hierarchical category structure. Abrams, Baecker, and Chignell [1] carried out an extensive study of how several hundred web users used bookmarks. They observed a number of strategies for organizing bookmarks, including a flat ordered list, a single level of folders, and hierarchical folders. They also made four design recommendations to help users manage their bookmarks more effectively. First, bookmarks must be easy to organize, e.g., via automatic sorting techniques. Second, visualization techniques are necessary to provide comprehensive overviews of large sets of bookmarks. Third, rich representations of sites are required; many users noted that site titles are not accurate descriptors of site content. Finally, tools for managing bookmarks must be well integrated with web browsers. Many researchers have created experimental information workspace interfaces, often designed expressly for web documents. Card, Robertson, and York [5] describe the WebBook, which uses a book metaphor to group a collection of related web pages for viewing and interaction, and the WebForager, an interface that lets users view and manage multiple WebBooks. Mackinlay, Rao, and Card [9] developed a novel user interface for accessing articles from a citation database. The central UI object is a “Butterfly”, which represents an article, its references, and its citers. The interface makes it easy for users to browse among related articles, group articles, and generate queries to retrieve articles that stand in a particular relationship to the current article. The Data Mountain of Robertson et al [14] represents documents as thumbnail images in a 3D virtual space. Users can move and group the images freely, with various interesting visual and audio cues used to help users arrange the documents. In a study comparing the use of Data Mountain to Internet Explorer Favorites, Data Mountain users retrieved items more quickly, with fewer incorrect or failed retrievals. Our research shares goals with much of the previous work. We focus on designing interfaces that make automatically extracted information about web sites readily accessible to users. We show that this increases users’ ability to select high-quality sites. Through ongoing user studies and redesign, we developed easy-to-use annotation and grouping techniques that let users organize items better and more quickly. Finally, we learned how users interleave work on various tasks and re-designed our interface to support such task interleaving. TOPICSHOPEXPLORER, VERSION 1 The TopicShop Explorer is implemented in C++ and runs on Microsoft Windows platforms. Version 1 was based directly on the Macintosh file manager / MS Windows Explorer metaphor (see [3] for detail of TopicShop Version 1 and the pilot study). Accordingly, users could view collections in either a details (Figure 1) or icons (Figure 2) view. The details view showed site profile information (see below) to help users evaluate sites, and the icons view let users organize sites spatially. Figure 1: TopicShop Explorer (version 1), details view. Each web site is represented by a small thumbnail image, the site title, and profile data including the links to/from other sites in the collection, and the number of pages, images, and audio files on the site. Users can sort by a property by clicking on the appropriate column. Figure 2: TopicShop Explorer (version 1), icons view. Each site is represented by a large thumbnail image and the site title. Users can organize sites by arranging them spatially, a technique especially useful in the early stages of exploration. The collections of sites and site profile data used in TopicShop are obtained by running a webcrawler/analyzer. The crawler takes a user-specified set of seed sites as input, and follows links from the seeds to construct a graph of the seed sites, pages contained on these sites, and, optionally, sites determined to be related based on their textual and hyperlink connections to the seeds. CHI Letters vol 2, 2 202 Site profiles are built by fetching a large number of pages from each site. Profiles contain content data, including the page title, an estimate of the page count, and a roster of audio files, movie files, and images; they also record links between sites in the collection. In addition, a thumbnail image of each site’s root page is constructed. The first goal of TopicShop is to help users evaluate and identify high quality sites. We sought to achieve this goal by providing site profile data and int

Specifying preferences based on user history

As people participate in the thousands of global conversations that comprise Usenet news, one thing they do is post their opinions of web resources. Phoaks is a collaborative filtering system that continuously parses, classifies, abstracts and tallies those opinions. About 3,500 users per day consult Phoaks web pages that reflect the results. Phoaks also features a general architecture for building similar collaborative filtering interfaces to conversational data. We report here on the Phoaks resource recommendation interface, the architecture, and the issues and experience that make up its rationale.