András Micsik

Evaluation of digital libraries

Digital libraries (DLs) are new and innovative information systems, under constant development and change, and therefore evaluation is of critical importance to ensure not only their correct evolution but also their acceptance by the user and application communities. The Evaluation activity of the DELOS Network of Excellence has performed a large-scale survey of current DL evaluation activities. This study has resulted in a description of the state of the art in the field, which is presented in this paper. The paper also proposes a new framework for the evaluation of DLs, as well as for recording, describing and analyzing the related research field. The framework includes a methodology for the classification of current evaluation procedures. The objective is to provide a set of flexible and adaptable guidelines for DL evaluation.

Digital Libraries: A Generic Classification and Evaluation Scheme

Evaluation of digital libraries (DLs) is essential for further development in this area. Whereas previous approaches were restricted to certain facets of the problem, we argue that evaluation of DLs should be based on a broad view of the subject area. For this purpose, we develop a new description scheme using four major dimensions: data/collection, system/technology, users, and usage. For each of these dimensions, we describe the major attributes. Using this scheme, existing DL test beds can be characterised. For this purpose, we have performed a survey by means of a questionnaire, which is now continued by setting up a DL meta-library.

A logging scheme for comparative digital library evaluation

Evaluation of digital libraries assesses their effectiveness, quality and overall impact. To facilitate the comparison of different evaluations and to support the re-use of evaluation data, we are proposing a new logging schema. This schema will allow for logging and sharing of a wide array of data about users, systems and their interactions. We discuss the multi-level logging framework presented in [19] and describe how the community can add to and gain from using the framework. The main focus of this paper is the logging of events within digital libraries on a generalised, conceptual level, as well as the services based on it. These services will allow diverse digital libraries to store their log data in a common repository using a common format. In addition they provide means for analysis and comparison of search history data.

An experimental framework for comparative digital library evaluation: the logging scheme

Evaluation of digital libraries assesses their effectiveness, quality and overall impact. In this paper we present a novel, multi-level logging framework that will provide complete coverage of the different aspects of DL usage for user-system interactions. Based on this framework, we can analyse for various DL stakeholders the logging data according to their specific interests. In addition, analysis tools and a freely accessible log data repository will yield synergies and sustainability in DL evaluation and encourage a community for DL evaluation by providing for discussion on a common ground

AQUA: query visualization for the NCSTRL digital library

AQUA is a prototype implementation of a general user interface paradigm which is capable of modelling iterative query refinement and embedding several common and advanced query techniques. AQUA has been applied to the NCSTRL (Networked Computer Science Technical Reports Library) distributed digital library.

Inter-organizational Interoperability through Integration of Multiagent, Web Service, and Semantic Web Technologies

This paper presents a software architecture for inter-organizational multiagent systems. The architecture integrates Web service technology into multiagent systems to overcome the technical interoperability problem of current multiagent systems in the fast growing service-oriented environments. We integrate Semantic Web technology to make multiagent systems semantically interoperable. We address the problem of interoperability regarding interfaces, messaging protocols, data exchanged, and security whilst considering a dynamic e-business environment. The proposed architecture enables service virtualization, secure service access across organizational boundaries, service-to-agent communication, and OWL reasoning within agents.

Transformation of database populations and operations from the conceptual to the internal level

Abstract In this paper we describe a framework for the automatic transformation of databases from the conceptual to the internal level. Our approach fits in with the ANSI/SPARC three level architecture for database management systems. The framework consists of three components: structures (database schemata), populations (database contents) and operations (database manipulations). On the conceptual level we describe these components for modelling techniques, such as ER, NIAM and the Binary Relationship Model. On the internal level we focus on (nested) relational models. For each component on the conceptual level a separate transformation is provided to the internal level. The components are integrated by a central notion: the predicator. Further integration is obtained by means of basic population rules.

Flexible SLA negotiation using semantic annotations

Moving towards a global market of services requires flexible infrastructures that will deal with the inevitable semantic heterogeneity that occurs during the negotiation that precedes the trading of a service. In order to reach an agreement, the negotiating parties need to understand the concepts describing the Quality of Service (QoS) terms which are part of the Service Level Agreement (SLA). The use of semantic annotations can increase the level of flexibility and automation, allowing the two parties to use their own terminology as long as it is related to the commonly understood conceptual model. This paper discusses how SLA negotiation will benefit from the use of a lightweight backwards compatible semantic annotation mechanism.

StreamOnTheFly: a peer-to-peer network for radio stations and podCasters

The StreamOnTheFly network demonstrates new ways of management and personalisation technologies for audio. The architecture is based on a decentralized network of software components using automatic metadata replication in a peer-to-peer manner. The network also promotes a new common metadata schema and content exchange format. Content reuse and content exchange is made possible by StreamOnTheFly in several use cases.

Visualizing queries in a distributed digital library system

A general user interface paradigm is presented which is capable of modelling iterative query refinement and embedding several common and advanced query techniques. A prototype implementation of this user interface called AQUA has been applied to the NCSTRL (Networked Computer Science Technical Reports Library) distributed digital library.