Sharon J. Laskowski

Visualization of search results: a comparative evaluation of text, 2D, and 3D interfaces

Although there have been many prototypes of visualization in support of information retrieval, there has been little systematic evaluation that distinguishes the benefits of the visualization per se from that of various accompanying features. The current study focuses on such an evaluation of NIRVE, a tool that supports visualization of search results. Insofar as possible, functionally equivalent 3D, 2D, and text versions of NIRVE were implemented. Nine novices and six professional users completed a series of information-seeking tasks on a set of retrieved documents. There were high interface costs for the 3D visualization, although those costs decreased substantially with experience. Performance was best when the tool’s properties matched task demands; only under the right combination of task, user, and interface did 3D visualization result in performance comparable to functionally matched 2D and textual tools.

Design of 3D visualization of search results: evolution and evaluation

We discuss the evolution of the NIST Information Retrieval Visualization Engine. This prototype employs modern interactive visualization techniques to provide easier access to a set of documents resulting from a query to a search engine. The motivation and evaluation of several design features, such as keywork to concept mapping, explicit clustering, the use of 3D vs. 2D, and the relationship of visualization to logical structure are described. In particular, the result of an extensive usability experiment show how visualization may lead to either increased or decreased cognitive load.

Evaluation for collaborative systems

The Evaluation Working Group (EWG) in the Defense Advanced Research Projects Agency (DARPA) Intelligent Collaboration and Visualization (IC&V) program has developed a methodology for evaluating collaborative systems. This methodology consists of a framework for classification of CSCW (Computer Supported Cooperative Work) systems, metrics and measures related to the various components in the framework, and a scenario-based evaluation approach. This paper describes the components of this methodology. Two case studies of evaluations based on this methodology are also described.

VAST 2008 Challenge: Introducing mini-challenges

Visual analytics experts realize that one effective way to push the field forward and to develop metrics for measuring the performance of various visual analytics components is to hold an annual competition. The VAST 2008 Challenge is the third year that such a competition was held in conjunction with the IEEE Visual Analytics Science and Technology (VAST) symposium. The authors restructured the contest format used in 2006 and 2007 to reduce the barriers to participation and offered four mini-challenges and a Grand Challenge. Mini Challenge participants were to use visual analytic tools to explore one of four heterogeneous data collections to analyze specific activities of a fictitious, controversial movement. Questions asked in the Grand Challenge required the participants to synthesize data from all four data sets. In this paper we give a brief overview of the data sets, the tasks, the participation, the judging, and the results.

VAST 2006 Contest - A Tale of Alderwood

Visual analytics experts realize that one effective way to push the field forward and to develop metrics for measuring the performance of various visual analytics components is to hold an annual competition. The first visual analytics science and technology (VAST) contest was held in conjunction with the 2006 IEEE VAST Symposium. The competition entailed the identification of possible political shenanigans in the fictitious town of Alderwood. A synthetic data set was made available as well as tasks. We summarize how we prepared and advertised the contest, developed some initial metrics for evaluation, and selected the winners. The winners were invited to participate at an additional live competition at the symposium to provide them with feedback from senior analysts

Universal remote console standard: toward natural user interaction in ambient intelligence

The draft standard on a Universal Remote Console (URC) framework is on its way to be reviewed and released by ANSI in 2004. This standard will contribute to the goal of Ambient Intelligence by allowing users to interact with networked devices and services in their environments in universal and natural ways, utilizing technologies such as natural language interaction and wearable computing. This SIG will follow up on last years successful SIG, whose contributions helped to shape the URC draft standard.Participants in this SIG will present and discuss the impact of the URC draft standard on the field of Ambient Intelligence. The goal of this SIG is to build a network of people from industry and academia who are interested in moving the Universal Remote Console standard forward, and in pursuing its wide-spread adoption in order to implement adaptable and usable user interfaces for networked devices and services.

VAST 2007 Contest - Blue Iguanodon

Visual analytics experts realize that one effective way to push the field forward and to develop metrics for measuring the performance of various visual analytics components is to hold an annual competition. The second visual analytics science and technology (VAST) contest was held in conjunction with the 2007 IEEE VAST symposium. In this contest participants were to use visual analytic tools to explore a large heterogeneous data collection to construct a scenario and find evidence buried in the data of illegal and terrorist activities that were occurring. A synthetic data set was made available as well as tasks. In this paper we describe some of the advances we have made from the first competition held in 2006.

Implications of Graphics on Usability and Accessibility for the Voter

This paper explores the impact of graphics on the usability and accessibility of voting systems. Graphical elements, as part of voting systems, include both photographs and party logos that indicate specific candidates or political parties, informational icons such as arrows and alert symbols, and animations or other video. After an overview of the history of graphics on ballots, usability and accessibility issues concerning graphics are discussed in detail. The question of whether certain types of graphics would help people with cognitive disabilities vote is then considered in light of research and best practices for usability and accessibility.

Quantifying Usability: The Industry Usability Reporting Project

The Industry US ability Reporting (IUSR) Project seeks to help potential corporate consumers of software obtain information about the usability of supplier products, to measure the benefit of more usable software, and to increase communication about usability needs between consumers and suppliers. Human factors and software engineers have developed a Common Industry Format (ANSI/NCITS 354–2001) for sharing usability information. Four pilot studies were conducted by industry which verify its usefulness in procurement and assess the costs and benefits of including usability test results in the software purchase process. Use of the Common Industry Format can increase communications across corporate boundaries and help improve the usability of software for consumers. The standard may also be applicable to setting usability requirements, and measuring usability of websites, hardware, and universal access.

Automatic capture, representation, and analysis of user behavior

The goal of this workshop is to explore the implications of automated capture and analysis of user behaviors on HCI and UE research.