Nardine Osman

Alternatives to Peer Review: Novel Approaches for Research Evaluation

In this paper we review several novel approaches for research evaluation. We start with a brief overview of the peer review, its controversies, and metrics for assessing efficiency and overall quality of the peer review. We then discuss five approaches, including reputation-based ones, that come out of the research carried out by the LiquidPub project and research groups collaborated with LiquidPub. Those approaches are alternative or complementary to traditional peer review. We discuss pros and cons of the proposed approaches and conclude with a vision for the future of the research evaluation, arguing that no single system can suit all stakeholders in various communities.

Models of Interaction as a Grounding for Peer to Peer Knowledge Sharing

Most current attempts to achieve reliable knowledge sharing on a large scale have relied on pre-engineering of content and supply services. This, like traditional knowledge engineering, does not by itself scale to large, open, peer to peer systems because the cost of being precise about the absolute semantics of services and their knowledge rises rapidly as more services participate. We describe how to break out of this deadlock by focusing on semantics related to interaction and using this to avoid dependency on a priori semantic agreement; instead making semantic commitments incrementally at run time. Our method is based on interaction models that are mobile in the sense that they may be transferred to other components, this being a mechanism for service composition and for coalition formation. By shifting the emphasis to interaction (the details of which may be hidden from users) we can obtain knowledge sharing of sufficient quality for sustainable communities of practice without the barrier of complex meta-data provision prior to community formation.

Run-time model checking of interaction and deontic models for multi-agent systems

This paper is concerned with the problem of obtaining predictable interactions between groups of agents in open environments when individual agents do not expose their BDI logic. The most popular approaches to this in practise have been to model interaction protocols and to model the deontic constraints imposed by individual agents. Both of these approaches are appropriate and necessary but their combination creates the practical problem of ensuring that interaction protocols come into contact with agents that possess compatible deontic constraints. This is essentially an issue of property checking dynamically at run-time. We show how model checking can be applied to this problem.

Trust and matching algorithms for selecting suitable agents

This article addresses the problem of finding suitable agents to collaborate with for a given interaction in distributed open systems, such as multiagent and P2P systems. The agent in question is given the chance to describe its confidence in its own capabilities. However, since agents may be malicious, misinformed, suffer from miscommunication, and so on, one also needs to calculate how much trusted is that agent. This article proposes a novel trust model that calculates the expectation about an agents future performance in a given context by assessing both the agents willingness and capability through the semantic comparison of the current context in question with the agents performance in past similar experiences. The proposed mechanism for assessing trust may be applied to any real world application where past commitments are recorded and observations are made that assess these commitments, and the model can then calculate ones trust in another with respect to a future commitment by assessing the others past performance.

Dynamic Coordination in Fleet Management Systems: Toward Smart Cyber Fleets

Fleet management systems are commonly used to coordinate mobility and delivery services in a broad variety of domains. However, their traditional top-down control architecture becomes a bottleneck in open and dynamic environments, where scalability, proactiveness, and autonomy are becoming key factors for their success. Here, the authors present an abstract event-based architecture for fleet management systems that supports tailoring dynamic control regimes for coordinating fleet vehicles, and illustrate it for the case of medical emergency management. Then, they go one step ahead in the transition toward automatic or driverless fleets, by conceiving fleet management systems in terms of cyber-physical systems, and putting forward the notion of cyber fleets.

Propagation of Opinions in Structural Graphs

Trust and reputation measures are crucial in distributed open systems where agents need to decide whom or what to choose. Existing work has overlooked the impact of an entitys position in its structural graph and its effect on the propagation of trust in such graphs. This paper presents an algorithm for the propagation of reputation in structural graphs. It allows agents to infer their opinion about unfamiliar entities based on their view of related entities. The proposed mechanism focuses on the “part of” relation to illustrate how reputation may flow (or propagate) from one entity to another. The paper bases its reputation measures on opinions, which it defines as probability distributions over an evaluation space, providing a rich representation of opinions.

An interaction-centric approach to support peer coordination in distributed emergency response management

Modern information systems are required to operate in distributed and dynamic environments. In such open settings, coordination technologies play a crucial role in the design of flexible software systems. Research efforts in different areas are converging to devise suitable mechanisms for process and peer coordination: in particular, current results on service-oriented computing and multi-agent systems are being integrated to support dynamic decision-making processes among autonomous components in large, open systems. This paper addresses how agent technologies can be designed, applied, and eventually integrated with standard technologies, in order to build more robust and intelligent systems. The focus of our research is on the engineering, exploitation and evaluation of an agent protocol language in realistic contexts. In particular, a specific executable protocol language is adopted to specify simulated interactions among distributed processes which will be tested in emergency response domain activities (that we will refer to hereafter as e-Response activities), chosen as an example of knowledge-intensive and dynamic application domain where intelligent decision making is crucial. We present a novel approach based on shared protocols models distributed through a peer-to-peer infrastructure and we show how it can be applied in the context of crisis management to support coalition formation and process coordination in open environments. Specifically, a prototype e-Response simulation system - built on a Peer-to-Peer (P2P) infrastructure - has been developed to execute interaction models describing common coordination tasks in the emergency response domain. Preliminary evaluation of the proposed framework demonstrates its capability to support such e-Response tasks.

Dynamic model checking for multi-agent systems

This paper is concerned with the problem of obtaining predictable interactions between groups of agents in open environments when individual agents do not expose their bdi logic. The most popular approaches to this in practise have been to model interaction protocols and to model the deontic constraints imposed by individual agents. Both of these approaches are appropriate and necessary but their combination creates the practical problem of ensuring that interaction protocols are meshed with agents that possess compatible deontic constraints. This is essentially an issue of property checking dynamically at run-time. We show how model checking can be applied to this problem.

Sharing online cultural experiences: an argument-based approach

This paper proposes a system that allows a group of human users to share their cultural experiences online, like buying together a gift from a museum or browsing simultaneously the collection of this museum. We show that such application involves two multiple criteria decision problems for choosing between different alternatives (e.g. possible gifts): one at the level of each user, and one at the level of the group for making joint decisions. The former is made manually by the users via the WeShare interface. This interface displays an image with tags reflecting some features (criteria) of the image. Each user expresses then his opinion by rating the image and each tag. A user may change his choices in light of a report provided by his WeShare agent on the opinion of the group. Joint decisions are made in an automatic way. We provide a negotiation protocol which shows how they are reached. Both types of decisions are based on the notion of argument. Indeed, a tag which is liked by a user constitutes an argument pro the corresponding image whereas a tag which is disliked gives birth to a cons argument. These arguments may have different strengths since a user may express to what extent he likes/dislikes a given tag. Finally, the opinion analysis performed by a WeShare agent consists of aggregating the arguments of the users.

Engineering multiuser museum interactives for shared cultural experiences

Multiuser museum interactives are computer systems installed in museums or galleries which allow several visitors to interact together with digital representations of artefacts and information from the museums collection. In this paper, we describe WeCurate, a socio-technical system that supports co-browsing across multiple devices and enables groups of users to collaboratively curate a collection of images, through negotiation, collective decision making and voting. The engineering of such a system is challenging since it requires to address several problems such as: distributed workflow control, collective decision making and multiuser synchronous interactions. The system uses a peer-to-peer Electronic Institution (EI) to manage and execute a distributed curation workflow and models community interactions into scenes, where users engage in different social activities. Social interactions are enacted by intelligent agents that interface the users participating in the curation workflow with the EI infrastructure. The multiagent system supports collective decision making, representing the actions of the users within the EI, where the agents advocate and support the desires of their users e.g. aggregating opinions for deciding which images are interesting enough to be discussed, and proposing interactions and resolutions between disagreeing group members. Throughout the paper, we describe the enabling technologies of WeCurate, the peer-to-peer EI infrastructure, the agent collective decision making capabilities and the multi-modal interface. We present a system evaluation based on data collected from cultural exhibitions in which WeCurate was used as supporting multiuser interactive. HighlightsWe describe a socio-technical system, supporting co-browsing across multiple devices.The system supports collective decision making based on argumentation and negotiation.Electronic institutions model social interactions, enacted by intelligent agents.We evaluate the social awareness perceived by the users.We evaluate the collective decision models employed by the agents.