Giorgio Brajnik

A shell for developing non-monotonic user modeling systems

This paper first presents a general structured framework for user modeling, which includes a set of basic user modeling purposes exploited by a user modeling system when providing a set of services to other components of an application. At a higher level of abstraction such an application may perform a generic user modeling task , which results from an appropriate combination of some basic user modeling purposes. The central aim of the paper is to present, within the proposed framework, a flexible general-purpose shell, called UMT (User Modeling Tool), which supports the development of user modeling applications. UMT features a non-monotonic approach for performing the modeling activity: more specifically, it utilizes a modeling approach called assumption-based user modeling, which exploits a truth maintenance mechanism for maintaining the consistency of the user model. The modeling task is divided into two separate activities, one devoted to user classification and user model management, and the other devoted to consistency maintenance of the model. The modeling knowledge exploited by UMT is represented by means of stereotypes and production rules. UMT is capable of identifying, at any given moment during an interaction, all the possible alternative models which adequately describe the user and are internally consistent. The choice of the most plausible one among them is then performed using an explicit programmable preference criterion. UMT is also characterized by a very well defined and simple interface with the hosting application, and by a specialized development interface which supports the developer during the construction of specific applications. This paper includes an example application in the field of information-providing systems. UMT has been developed in Common LISP.

Evaluating user interfaces to information retrieval systems: a case study on user support

Designing good user interfaces to information retrieval systems is a complex activity. The design space is large and evaluation methodologies that go beyond the classical precision and recall figures are not well established. In this paper we present an evaluation of an intelligent interface that covers also the user-system interaction and measures users satisfaction. More specifically, we describe an experiment that evaluates: (i) the added value of the semiautomatic query reformulation implemented in a prototype system; (ii) the importance of technical, terminological, and strategic supports and (iii) the best way to provide them. The interpretation of results leads to guidelines for the design of user interfaces to information retrieval systems and to some observations on the evaluation issue.

User modeling in intelligent information retrieval

Abstract The issue of exploiting user modeling techniques in the framework of cooperative interfaces to complex artificial systems has recently received increasing attention. In this paper we present the IR-NLI II system, an expert interface that allows casual users to access online information retrieval systems and encompasses user modeling capabilities. More specifically, an illustration of the user modeling subsystem is given by describing the organization of the user model proposed for the particular application area, together with its use during system operation. The techniques utilized for the construction of the model are presented as well. They are based on the use of stereotypes, which are descriptions of typical classes of users. More specifically, they include both declarative and procedural knowledge for describing the features of the class to which the stereotype is related, for assigning a user to that class, and for acquiring and validating the necessary information during system operation.

Quantitative metrics for measuring web accessibility

This paper raises the need for quantitative accessibility measurement and proposes three different application scenarios where quantitative accessibility metrics are useful: Quality Assurance within Web Engineering, Information Retrieval and accessibility monitoring. We propose a quantitative metric which is automatically calculated from reports of automatic evaluation tools. In order to prove the reliability of the metric, 15 websites (1363 web pages) are measured based on results yielded by 2 evaluation tools: EvalAccess and LIFT. Statistical analysis of results shows that the metric is dependent on the evaluation tool. However, Spearmans test produces high correlation between results of different tools. Therefore, we conclude that the metric is reliable for ranking purposes in Information Retrieval and accessibility monitoring scenarios and can also be partially applied in a Web Engineering scenario.

User modeling in expert man-machine interfaces: a case study in intelligent information retrieval

The requirements of a user modeling component for an expert interface are analyzed, and the main points of a proposed approach to user modeling are stated. The authors focus on a knowledge-based system, called UM-tool, devoted to creating, maintaining, and using explicit user models within an expert interface. UM-tool supports a novel approach to user modeling, which is based both on the use of stereotypes and on a dynamic reclassification scheme. The architecture of the system is described, the organization and content of its knowledge bases are illustrated, and the modeling mechanisms utilized are presented in detail. An example of the use of UM-tool in the frame of the information-retrieval-natural-language-interface (IR-NLI II) expert interface devoted to supporting end users in accessing online information retrieval services is discussed, focusing on the specific role of the user modeling component. An evaluation of the proposed approach and a critical comparison with related work are presented. Future research directions are outlined. >

Comparing accessibility evaluation tools: a method for tool effectiveness

This paper claims that effectiveness of automatic tools for evaluating web site accessibility has to be itself evaluated, given the increasingly important role that these tools play. The paper presents a comparison method for a pair of tools that takes into account correctness, completeness and specificity in supporting the task of assessing the conformance of a web site with respect to established guidelines. The paper presents data acquired during a case study based on comparing LIFT Machine with Bobby. The data acquired from the case study is used to assess the strengths and weaknesses of the comparison method. The conclusion is that even though there is room for improvement of the method, it is already capable of providing accurate and reliable conclusions.

Automatic web accessibility metrics: Where we are and where we can go

The fact that several web accessibility metrics exist may be evidence of a lack of a comparison framework that highlights how well they work and for what purposes they are appropriate. In this paper we aim at formulating such a framework, demonstrating that it is feasible, and showing the findings we obtained when we applied it to seven existing automatic accessibility metrics. The framework encompasses validity, reliability, sensitivity, adequacy and complexity of metrics in the context of four scenarios where the metrics can be used. The experimental demonstration of the viability of the framework is based on applying seven published metrics to more than 1500 web pages and then operationalizing the notions of validity-as-conformance, adequacy and complexity. Our findings lead us to conclude that the Web Accessibility Quantitative Metric, Page Measure and Web Accessibility Barrier are the metrics that achieve the highest levels of quality (out of the seven that we examined). Finally, since we did not analyse reliability, sensitivity and validity-in-use, this paper provides guidance to address them in what are new research avenues.

Beyond Conformance: The Role of Accessibility Evaluation Methods

The topic I want to address is the role that accessibility evaluation methods can play in helping the transition from accessibility viewed as standard conformance, to a user-centered accessibility. As we will see, this change sets additional requirements on how evaluations of websites should be carried out. This paper first discusses different problems that occur while dealing with accessibility. We will see that different people have radically different views of accessibility and how it should be assessed. The first requirement is a clear definition of what accessibility is and how it should be assessed. The accessibility model discussed in Section 2.1 has precisely this role. Several existing evaluation methods are then reviewed and discussed, a simple taxonomy is presented, and differences that occur when evaluating accessibility rather than usability are pinpointed.

Web accessibility testing: when the method is the culprit

Testing accessibility of a web site is still an art. Lack of appropriate definitions of accessibility and of standard testing methods are some of the reasons why Web accessibility is so difficult to achieve The paper describes a heuristic walkthrough method based on barriers; it then discusses how methods like this can be evaluated, and it shows experimental data about validity and usefulness of the method when compared to standards review

Strategic help in user interfaces for information retrieval

Brajnik et alia describe their view of an effective retrieval interface, one which coaches the searcher using stored knowledge not only of database structure, but of strategic situations which are likely to occur, such as repeating failed tactics in a low return search, or failing to try relevance feedback techniques. The emphasis is on the system suggesting search strategy improvements by relating them to an analysis of work entered so far and selecting and ranking those found relevant. FIRE is an interface utilizing these techniques. It allows the user to assign documents to useful, topical and trash folders, maintains thesauri files automatically searchable on query terms, and it builds, using user entries and a rule system, a picture of the retrieval situation from which it generates suggestions.