Qunhua Zhao

Empirical evaluation of adaptive user modeling in a medical information retrieval application

A comprehensive methodology for evaluating a user model presents challenges in choosing metrics and in assessing usefulness from both user and system perspectives. In this paper, we describe such a methodology and use it to assess the effectiveness of an adaptive user model embedded in a medical information retrieval. We demonstrate that the user model helps to improve the retrieval quality without degrading the system performance and identify usability problems overlooked in the user model architecture. Empirical data help us in analyzing drawbacks in our user model and develop solutions.

User Modelling for Intent Prediction in Information Analysis

User modelling is a key element in successfully assisting intelligence analysts who must gather information and make decisions without being overloaded by the massive amounts of data available on a daily basis most of which are irrelevant. Furthermore, with user modelling, we can predict the goals and intentions of the analyst in order to better serve their information seeking tasks by providing better re-organization and presentation of data as well as pro-actively retrieve novel and relevant information as it arises. Our goal is to provide a dynamic user model of an analyst and work with him as he goes about his daily tasks.

OmniSeer: A Cognitive Framework for User Modeling, Reuse of Prior and Tacit Knowledge, and Collaborative Knowledge Services

This paper describes the current state of the OmniSeer system. OmniSeer supports intelligence analysts in the handling of massive amounts of data, the construction of scenarios, and the management of hypotheses. OmniSeer models analysts with dynamic user models that capture an analysts context, interests, and preferences, thus enabling more efficient and effective information retrieval. OmniSeer explicitly represents the prior and tacit knowledge of analysts, thus enabling transfer and reuse of such knowledge. Both the user and cognitive models employ a Bayesian network fragment representation, which supports principled probabilistic reasoning and analysis. An independent evaluation of OmniSeer was carried out at NIST and will be used to guide further development.

I-FGM: information retrieval in highly dynamic search spaces

Intelligent foraging, gathering and matching (I-FGM) has been shown to be an effective tool for intelligence analysts who have to deal with large and dynamic search spaces. I-FGM introduced a unique resource allocation strategy based on a partial information processing paradigm which, along with a modular system architecture, makes it a truly novel and comprehensive solution to information retrieval in such search spaces. This paper provides further validation of its performance by studying its behavior while working with highly dynamic databases. Results from earlier experiments were analyzed and important changes have been made in the system parameters to deal with dynamism in the search space. These changes also help in our goal of providing relevant search results quickly and with minimum wastage of computational resources. Experiments have been conducted on I-FGM in a realistic and dynamic simulation environment, and its results are compared with two other control systems. I-FGM clearly outperforms the control systems.

What Makes a Good Summary

One of the biggest challenges for intelligence analysts who participate in prevention or response to a terrorism act is to quickly find relevant information from massive amounts of data. Along with research on information retrieval and filtering, text summarization is an effective technique to help intelligence analysts shorten their time to find critical information and make timely decisions. Multi-document summarization is particularly useful as it serves to quickly describe a collection of information. The obvious shortcoming lies in what it cannot capture especially in more diverse collections. Thus, the question lies in the adequacy and/or usefulness of such summarizations to the target analyst. In this chapter, we report our experimental study on the sensitivity of users to the quality and content of multi-document summarization. We used the DUC 2002 collection for multi-document summarization as our testbed. Two groups of document sets were considered: (I) the sets consisting of closely correlated documents with highly overlapped content; and (II) the sets consisting of diverse documents covering a wide scope of topics. Intuitively, this suggests that creating a quality summary would be more difficult for the latter case. However, human evaluators were discovered to be fairly insensitive to this difference. This occurred when they were asked to rank the performance of various automated summarizers. In this chapter, we examine and analyze our experiments in order to better understand this phenomenon and how we might address it to improve summarization quality. In particular, we present a new metric based on document graphs that can distinguish between the two types of document sets.

Adaptivity modeling for complex adaptive systems with application to biology

Modeling or simulating complex adaptive systems (CASs) is a very important and challenging endeavor. Previously, we introduced a generic framework for addressing this problem, and included a number of critical criteria including emergence, self-organization, adaptivity, and others. In this paper, we present the methodology used for designing a particularly key component of our framework: the short-term adaptivity model. We test our short-term adaptivity model and framework within the biological science application domain, which have a number of critical CASs. In particular we model the aggregation process of Dictyostelium. The comparison between established biological experimental results and our simulation results validate the effectiveness of our model and framework.

Applying I-FGM to image retrieval and an I-FGM system performance analyses

Intelligent Foraging, Gathering and Matching (I-FGM) combines a unique multi-agent architecture with a novel partial processing paradigm to provide a solution for real-time information retrieval in large and dynamic databases. I-FGM provides a unified framework for combining the results from various heterogeneous databases and seeks to provide easily verifiable performance guarantees. In our previous work, I-FGM had been implemented and validated with experiments on dynamic text data. However, the heterogeneity of search spaces requires our system having the ability to effectively handle various types of data. Besides texts, images are the most significant and fundamental data for information retrieval. In this paper, we extend the I-FGM system to incorporate images in its search spaces using a region-based Wavelet Image Retrieval algorithm called WALRUS. Similar to what we did for text retrieval, we modified the WALRUS algorithm to partially and incrementally extract the regions from an image and measure the similarity value of this image. Based on the obtained partial results, we refine our computational resources by updating the priority values of image documents. Experiments have been conducted on I-FGM system with image retrieval. The results show that I-FGM outperforms its control systems. Also, in this paper we present theoretical analysis of the systems with a focus on performance. Based on probability theory, we provide models and predictions of the average performance of the I-FGM system and its two control systems, as well as the systems without partial processing.

Modeling, analysis and visualization of uncertainty in the battlespace

Our goal is to develop a technology that enables information uncertainty portrayal, which provides battlespace decision makers with effective means to incorporate uncertainty into their decision making process. In This work, we describe the prototype uncertainty prediction system, which supports visualization and assessment of event uncertainty within the context of predictive battlespace awareness.