Marshall Ramsey

A smart itsy bitsy spider for the web

As part of the ongoing Illinois Digital Library Initiative project, this research proposes an intelligent agent approach to Web searching. In this experiment, we developed two Web personal spiders based on best first search and genetic algorithm techniques, respectively. These personal spiders can dynamically take a users selected starting homepages and search for the most closely related homepages in the Web, based on the links and keyword indexing. A graphical, dynamic, Java-based interface was developed and is available for Web access. A system architecture for implementing such an agent-based spider is presented, followed by detailed discussions of benchmark testing and user evaluation results. In benchmark testing, although the genetic algorithm spider did not outperform the best first search spider, we found both results to be comparable and complementary. In user evaluation, the genetic algorithm spider obtained significantly higher recall value than that of the best first search spider. However, their precision values were not statistically different. The mutation process introduced in genetic algorithm allows users to find other potential relevant homepages that cannot be explored via a conventional local search process. In addition, we found the Java-based interface to be a necessary component for design of a truly interactive and dynamic Web agent.

An intelligent personal spider (agent) for dynamic Internet/intranet searching

Abstract As Internet services based on the World-Wide Web become more popular, information overload has become a pressing research problem. Difficulties with search on Internet will worsen as the amount of on-line information increases. A scalable approach to Internet search is critical to the success of Internet services and other current and future National Information Infrastructure (NII) applications. As part of the ongoing Illinois Digital Library Initiative project, this research proposes an intelligent personal spider (agent) approach to Internet searching. The approach, which is grounded on automatic textual analysis and general-purpose search algorithms, is expected to be an improvement over the current static and inefficient Internet searches. In this experiment, we implemented Internet personal spiders based on best first search and genetic algorithm techniques. These personal spiders can dynamically take a users selected starting homepages and search for the most closely related homepages in the web, based on the links and keyword indexing. A plain, static CGI/HTML-based interface was developed earlier, followed by a recent enhancement of a graphical, dynamic Java-based interface. Preliminary evaluation results and two working prototypes (available for Web access) are presented. Although the examples and evaluations presented are mainly based on Internet applications, the applicability of the proposed techniques to the potentially more rewarding Intranet applications should be obvious. In particular, we believe the proposed agent design can be used to locate organization-wide information, to gather new, time-critical organizational information, and to support team-building and communication in Intranets.

Creating a large-scale content-based airphoto image digital library

This paper describes a content-based image retrieval digital library that supports geographical image retrieval over a testbed of 800 aerial photographs, each 25 megabytes in size. In addition, this paper also introduces a methodology to evaluate the performance of the algorithms in the prototype system. There are two major contributions: we suggest an approach that incorporates various image processing techniques including Gabor filters, image enhancement and image compression, as well as information analysis techniques such as the self-organizing map (SOM) into an effective large-scale geographical image retrieval system. We present two experiments that evaluate the performance of the Gabor-filter-extracted features along with the corresponding similarity measure against that of human perception, addressing the lack of studies in assessing the consistency between an image representation algorithm or an image categorization method and human mental model.

Information forage through adaptive visualization

Marshall Ramsey Department of MIS University of Arizona, McClelland Hall 430 Tucson, AZ 85721 E-mail: mramsey@bpa.arizona.edu Automatically created maps of concepts improve navigation in a collection of text documents. We report our research on leveraging navigation by providing interactively the ability to modify the maps themselves. We believe that this functionality leads to better responsiveness to the user and a more effective search. For this purpose we have created and tested a prototype system that builds and refines in real-time a map of concepts found in Web documents returned by a commercial search engine.

Tools for distributed facilitation

We have extensive experience with distributed facilitation of synchronous virtual meetings since 1992. More recently, we have gained experience in asynchronous virtual meetings and have learned lessons from which best practices have been distilled. Best practices for synchronous and asynchronous distributed facilitation give insight into requirements for toolsets. Our experience with toolsets, including (1) widely available unstructured toolsets, (2) virtual reality offices, and (3) persistent visualization, provide direction for further tool development.

A geographic knowledge representation system for multimedia geospatial retrieval and analysis

Abstract. Digital libraries serving multimedia information that may be accessed in terms of geographic content and relationships are creating special challenges and opportunities for networked information systems. An especially challenging research issue concerning collections of geo-referenced information relates to the development of techniques supporting geographic information retrieval (GIR) that is both fuzzy and concept-based. Viewing the meta-information environment of a digital library as a heterogeneous set of services that support users in terms of GIR, we define a geographic knowledge representation system (GKRS) in terms of a core set of services of the meta-information environment that is required in supporting concept-based access to collections of geospatial information. In this paper, we describe an architecture for a GKRS and its implementation in terms of a prototype system. Our GKRS architecture loosely couples a variety of multimedia knowledge sources that are in part represented in terms of the semantic network and neural network representations developed in artificial intelligence research. Both textual analysis and image processing techniques are employed in creating these textual and iconic geographical knowledge structures. The GKRS also employs spreading activation algorithms in support of concept-based knowledge retrieval. The paper describes implementational details of several of the components of the GKRS as well as discussing both the lessons learned from, and future directions of, our research.

A collection of visual thesauri for browsing large collections of geographic images

Digital libraries of geo-spatial multimedia content are currently deficient in providing fuzzy, concept-based retrieval mechanisms to users. The main challenge is that indexing and thesaurus creation are extremely laborintensive processes for text documents and especially for images. Recently, 800,000 declassified satellite photographs were made available by the United States Geological Survey. Additionally, millions of satellite and aerial photographs are archived in national and local map libraries. Such enormous collections make human indexing and thesaurus generation methods impossible to utilize. In this article we propose a scalable method to automatically generate visual thesauri of large collections of geo-spatial media using fuzzy, unsupervised machine-learning techniques.

Visualizing Internet search results with adaptive self-organizing maps (demonstration abstract)

Summarization and visualization tools are believed to be helpful in navigating through large volumes of data since a visual representation may elicit more deliberate query reformulation and better feedback to the retrieval system. Results from improved feedback explain the growing interest in visualization tools in academic and industrial research (for example, Scatter/Gather by Xerox, ThemeMedia by Batelle, Live Topics by AltaVista).

Interactive Internet search through automatic clustering (poster abstract): an empirical study

We have developed and empirically evaluated a method of information seeking (called Adaptive Search) that combines automatic document clustering and user feedback in a novel way. In this approach, the user starts with a natural text description of the needed information and goes through a sequence of interactions with the system in order to find documents of interest. Adaptive Search utilizes Kohonen Self-Organizing maps and acts as a layer between the user and a commercial search engine. In a laboratory experiment, subjects searched the World Wide Web for answers to a given set of questions. Our results indicate that the subjects spent less time finding correct answers using Adaptive Search than using the search engine directly. In addition, the Adaptive Search-suggested documents contained answers that were positioned consistently higher in the rank-ordered lists than those suggested by the Internet search engine. This suggests that document clustering can be integrated into an interactive search system in such a way that it substantially helps information seekers.

Create a large-scale digital library for geo-referenced information

One of the most challenging research issues for geospatial collections is to develop techniques to support fuzzy, conceptbased, geographic information retrieval (GIR). Based on an artificial intelligence approach, this project presents a Geospatial Knowledge Representation System (GKRS) prototype system that integrates multiple multimedia (textual and image) knowledge sources to support concept-based GIR. Based on semantic network and neural network representations, GKRS loosely couples different knowledge sources and adopts spreading activation algorithms for concept-based knowledge inferencing. Both textual analysis and image processing techniques have been employed to create textual and visual geographical knowledge structures. This project suggests a framework for developing a complete GKRS-based system and describes in detail the prototype.