Manolis Falelakis

Reasoning for video-mediated group communication

In this paper we present an approach to the reasoning required to support multi-location, multi-camera group-to-group video communication, which we call orchestration. Orchestration is akin to virtual directing: it has to ensure that each location displays the most adequate shots from all the other available sources. Its input is low-level cues extracted automatically from the AV streams. They are processed to detect higher-level events that determine the state of the communication. Directorial decisions are then inferred, reflecting social communication as well as stylistic criteria. Finally, they are transformed into camera and editing commands, directly executable by the AV infrastructure. Here, we present the architecture of the Orchestrator and sketch our rule-based approach to reasoning.

Semantic Integration of Cervical Cancer Data Repositories to Facilitate Multicenter Association Studies: The ASSIST Approach

The current work addresses the unification of Electronic Health Records related to cervical cancer into a single medical knowledge source, in the context of the EU-funded ASSIST research project. The project aims to facilitate the research for cervical precancer and cancer through a system that virtually unifies multiple patient record repositories, physically located in different medical centers/hospitals, thus, increasing flexibility by allowing the formation of study groups “on demand” and by recycling patient records in new studies. To this end, ASSIST uses semantic technologies to translate all medical entities (such as patient examination results, history, habits, genetic profile) and represent them in a common form, encoded in the ASSIST Cervical Cancer Ontology. The current paper presents the knowledge elicitation approach followed, towards the definition and representation of the diseases medical concepts and rules that constitute the basis for the ASSIST Cervical Cancer Ontology. The proposed approach constitutes a paradigm for semantic integration of heterogeneous clinical data that may be applicable to other biomedical application domains.

A Rule-Based Virtual Director Enhancing Group Communication

Audiovisual group communication systems deal with a large number of video streams, and, unlike less advanced videoconferencing systems, require intelligence for selecting adequate views for each of the connected rooms, in order to convey best what is happening in the other locations. Such a decision making component, in our implementation called Orchestration Engine (OE), acts as a Virtual Director. It processes low level events, emitted by content analysis sensors, into editing commands. The OE has two main components: one that semantically lifts low-level events into communication events and one that associates editing decisions to communication contexts. The former has to deal with uncertain and delayed information. The latter subsumes knowledge that reflects both conversation and narrative principles. Both components include contradicting bodies of knowledge. We investigate a rule-based event processing approach and reflect the scalability of our solution regarding competing and contradicting rules.

Improving video-mediated communication with orchestration

Video-mediated communication (VMC) has become a popular communication medium. However, research to date suggests that the inherent constraints of VMC impair effective and efficient communication and task performance. We propose that these negative findings could be attributed to how the technology was used and propose the novel concept of communication orchestration aimed at mitigating some of the signaled limitations. Orchestration is a selection process for displaying information that is deemed relevant for accomplishing an effective and efficient task performance and communicative experience. We report an experiment that confirmed this suggestion. The results indicate that orchestration could be an important novel feature to aid humans when communicating via VMC, but also suggest that there is potential for further improvements in orchestration.

Automatic orchestration of video streams to enhance group communication

Unlike legacy video-conferencing, which connects two nodes each equipped with a camera, recent systems facilitating for video-mediated group communication deal simultaneously with a large number of video streams. This highlights the need for orchestration,, i.e. the intelligent selection of the most adequate camera views to be displayed on each screen. In this paper we present the initial results of a study that evaluates the effects of orchestration on communication within a specific context; that of two remote groups playing a collaborative board game. The results of the experiment indicate that automatic orchestration can provide improvements similar to the ones achieved when live video mixing is performed by human editors.

Dynamic Semantic Identification with Complexity Constraints as a Knapsack Problem

The process of automatic identification of high level semantic entities (e.g., objects, concepts or events) in multimedia documents requires processing by means of algorithms that are used for feature extraction, i.e. low level information needed for the analysis of these documents at a semantic level. This work copes with the high and often prohibitive computational complexity of this procedure. Emphasis is given to a dynamic scheme that allows for efficient distribution of the available computational resources in application. Scenarios that deal with the identification of multiple high level entities with strict simultaneous restrictions, such as real time applications

Complexity Control in Semantic Identification

This paper proposes a methodology for modeling the process of semantic identification and controlling its complexity and accuracy of the results. Each semantic entity is defined in terms of lower level semantic entities and low level features that can be automatically extracted, while different membership degrees are assigned to each one of the entities participating in a definition, depending on their importance for the identification. By selecting only a subset of the features that are used to define a semantic entity both complexity and accuracy of the results are reduced. It is possible, however, to design the identification using the metrics introduced, so that satisfactory results are obtained, while complexity remains below some required limit

Enabling Distributed Theatre Performances through Multi-Camera Telepresence: Capturing System Behaviour in a Script-Based Approach

Audio-visual media has been utilized in theatre in many creative ways, but most of them have been employed to enrich performances on a single stage. Telepresence systems give rise to a new concept: distributed theatre, denoting performances co-acted from two or more different stages, each potentially having its own local audience. To achieve a truly immersive experience, we posit, such systems must satisfy a key requirement: they must be able to dynamically reframe the cameras and mix the content from the various sources on each of the available screens in order to capture the most relevant expressions in each location and appropriately represent them in the other locations. We have developed a script-based approach that allows the production team to express and refine such adaptive behaviour during development and rehearsals, as well as controlling it during the performance. We ran a successful trial of a telepresence system that implements this approach. This paper gives a brief description of the overall approach and of two key software components that control the systems behaviour, with some reflections upon the specific experience of the trial, the overall approach and further research.

Applying semantic technologies in cervical cancer research

Abstract In this paper we present a research system that follows a semantic approach to facilitate medical association studies in the area of cervical cancer. Our system, named ASSIST and developed as an EU research project, assists in cervical cancer research by unifying multiple patient record repositories, physically located in different medical centers or hospitals. Semantic modeling of medical data and rules for inferring domain-specific information allow the system to (i) homogenize the information contained in the isolated repositories by translating it into the terms of a unified semantic representation, (ii) extract diagnostic information not explicitly stored in the individual repositories, and (iii) automate the process of evaluating medical hypotheses by performing case–control association studies, which is the ultimate goal of the system.

Constructing Optimal Fuzzy Metric Trees for Agent Performance Evaluation

The field of multi-agent systems has reached a significant degree of maturity with respect to frameworks, standards and infrastructures. Focus is now shifted to performance evaluation of real-world applications, in order to quantify the practical benefits and drawbacks of agent systems. Our approach extends current work on generic evaluation methodologies for agents by employing fuzzy weighted trees for organizing evaluation-specific concepts/metrics and linguistic terms to intuitively represent and aggregate measurement information.Furthermore, we introduce meta-metrics that measure the validity and complexity of the contribution of each metric in the overall performance evaluation. These are all incorporated for selecting optimal subsets of metrics and designing the evaluation process incompliance with the demands/restrictions of various evaluation setups, thus minimizing intervention by domain experts. The applicability of the proposed methodology is demonstrated through the evaluation of a real-world test case.