Gian Piero Zarri

Conceptual and content-based annotation of (multimedia) documents

This paper focuses on the techniques used in an NKRL environment (NKRL = Narrative Knowledge Representation Language) to deal with a general problem affecting the so-called “semantic/conceptual annotations” techniques. These last, mainly ontology-based, aim at “annotating” multimedia documents by representing, in some way, the “inner meaning/deep content” of these documents. For documents of sufficient size, the content modeling operations are separately executed on ‘significant fragments’ of the documents, e.g., “sentences” for natural language texts or “segments” (minimal units for story advancement) in a video context. The general problem above concerns then the possibility of collecting all the partial conceptual representations into a global one. This integration operation must, moreover, be carried out in such a way that the meaning of the full document could go beyond the simple addition of the ‘meanings’ conveyed by the single fragments. In this context, NKRL makes use of second order knowledge representation structures, “completive construction” and “binding occurrences”, for collecting within the conceptual annotation of a whole “narrative” the basic building blocks corresponding to the representation of its composing elementary events. These solutions, of a quite general nature, are discussed in some depth in this paper. This last includes also a short “state of the art” in the annotation domain and some comparisons with the different methodologies proposed in the past for solving the above ‘integration’ problem.

Ontologies and reasoning techniques for (legal) intelligent information retrieval systems

An application of Narrative Knowledge Representation Language (NKRL) techniques on (declassified) ‘terrorism in Southern Philippines’ documents has been carried out in the context of the IST Parmenides project. This paper describes some aspects of this work: it is our belief, in fact, that the Knowledge Representation techniques and the Intelligent Information Retrieval tools used in this experiment can be of some interest also in an ‘Ontological Modelling of Legal Events and Legal Reasoning’ context.

Generalized World Entities as an Unifying IoT Framework: A Case for the GENIUS Project

After having briefly discussed some possible interpretations of the (still at least partially ambiguous ambiguous) ”IoT” term, this Chapter sums up the aims and the main characteristics of an on-going IoT-inspired project, GENIUS. GENIUS concerns the creation of a flexible, internet-based, IoT cognitive architecture, able to support a wide range of ‘intelligent’ applications focused on the recognition and interaction with the so-called Generalized World Entities (GWEs). The GWE paradigm intends to fill up the present fracture between the detection of entities at the sensor/physical level and their representation/management at the conceptual level. It deals in a unified way with physical objects, humans, robots, media objects and low-level events generated by sensors and with GWEs at higher level of abstraction corresponding to complex, structured events/situations/behaviours implying mutual relationships among GWEs captured at lower conceptual level. GWEs of both classes will be recognised and categorised by using, mainly, a conceptual “representation of the world”, ontology-based, auto-evolving and general enough to take into account both the “static” and “dynamic” characteristics of the GWEs. When all the GWEs (objects, agents, events, complex events, situations, circumstances, behaviours etc.) involved in a given application scenario have been recognised, human-like reasoning procedures in the form of “set of services”, general enough to be used in a vast range of GWE-based applications, can be used to solve real-life problems. Details about the use of the GWE paradigm to set up an “Ambient Assisted Living (AAL)” application for dealing with the “elderly at home problem” are provided in the Chapter.

High-Level Knowledge Representation and Reasoning in a Cognitive IoT/WoT Context

This chapter presents an overview of the Generalized World Entities (GWEs) paradigm, used to add a semantic/conceptual dimension to the ordinary IoT/WoT procedures. Its purpose is to expand the range of entities to be considered when describing a sensor-monitored environment by allowing, in particular, to seamlessly model in a unified way (i.e., within the same representation framework) physical entities like objects, humans, robots, etc. and higher levels of abstraction structures corresponding to general situations/actions/events/behaviours. The unifying factor is provided by the conceptual representation of the world used for modelling the GWEs of both types. This is ontology-based and general enough to take into account both the “static” (background information about, e.g., common notions like robot, person or physical object) and the “dynamic” (foreground information concerning, e.g., a robot or a person moving in real time towards a given object) characteristics of the different entities to deal with. After having presented a short state of the art in the cognitive/semantic IoT/WoT domain, we will specify the notion of GWE by describing its implementation under NKRL (Narrative Knowledge Representation Language) format. NKRL is a high-level modelling language, whose main characteristic concerns the use of two ontologies, an ontology of standard concepts and an ontology of events, this last dealing with the representation of the dynamic and spatio-temporal characterized information denoting behaviours, complex events, situations, circumstances etc. We will show, using several examples, that this dichotomy allows us to effectively model, in a seamlessly way, all the different entities managed by the usual IoT/WoT procedures.

A Semantic-Rich Approach to IoT Using the Generalized World Entities Paradigm

Abstract In this chapter, we present an overview of our proposed Generalized World Entities (GWEs) paradigm, to be used to add a semantic/conceptual dimension to IoT/WoT (Internet of Things/Web of Things) procedures. GWEs offer a unified way to model seamlessly, at the conceptual level, both the digital counterparts of elementary entities, such as physical objects, humans, robots and low-level events, and the semantic representations of higher levels of abstraction. These higher levels correspond to structured situations/behaviors implying mutual relationships among simpler entities. The GWEs approach is both a theoretical and a practical framework that can give a significant contribution to solve some important IoT/WoT problems, such as interoperability and uniform IoT controls. Thanks to a unique ontology-based representation language, the chapter shows the possibility of bridging the semantic gap between the recognition of entities at the sensor level and their representation at the conceptual level.

Functional and semantic roles in a high-level knowledge representation language

We describe in this paper a formalization of the notion of “role” that involves a clear separation between two very different sorts of roles. Semantic roles, like student or customer, are seen as (pre-defined) transitory properties that can be associated with (usually animate) entities. From a formal point of view, they can be represented as standard concepts to be placed into a specific branch of a particular ontology; they formalize the static and classificatory aspects of the notion of role. Functional roles must be used, instead, to model those pervasive and dynamic situations corresponding to events, activities, circumstances etc. that are characterized by spatio-temporal references; see, e.g., “John is now acting as a student”. They denote the specific function with respect to the global meaning of an event/situation/activity... that is performed by the entities involved in this event/situation... and formalize the dynamic and relational aspects of the notion of role. A functional role of the subject/agent/actor/protagonist... type is used to associate “John” with the notion of student or customer (semantic roles) during a specific time interval. Formally, functional roles are expressed as primitive symbols like subject, object, source, beneficiary. Semantic and functional roles interact smoothly when they are used to deal with challenging knowledge representation problems like the so-called “counting problem”, or when we need to set-up powerful inference rules whose atoms can directly denote complex situations. In this paper, the differentiation between semantic and functional roles will be illustrated from an narrative knowledge representation language (NKRL) point of view. NKRL is a high-level conceptual tool used for the computer-usable representation and management of the inner meaning of syntactically complex and semantically rich multimedia information. But, as we will see, the importance of this distinction goes well beyond its usefulness in a specific NKRL context. In particular, the use of functional roles is of paramount importance for the set-up of those evolved n-ary forms of knowledge representation that allow us to get rid from the limitations in expressiveness proper to the standard (binary) solutions.

Modeling and Advanced Exploitation of eChronicle ‘Narrative’ Information

In this paper, we describe NKRL (Narrative Knowledge Representation Language), a conceptual modeling formalism for taking into account the semantic characteristics of this important component of eChronicle information represented by the ‘narrative’ documents. In these documents, the main part of the information consists in the description of the ‘events’ that relate the real or intended behavior of some ‘actors’. Narrative documents of an industrial and economic interest correspond to news stories, corporate documents, normative and legal texts, intelligence messages, medical records, etc. NKRL employs several representational principles and some high-level inference tools.

IoT Semantic Modeling Using the GWE (Generalized World Entities) Paradigm

We present here an overview of the Generalized World Entities (GWEs) paradigm, to be used to add a semantic/conceptual dimension to the IoT/WoT procedures. GWEs offer a unified way to model seamlessly, at conceptual level, both the digital counterparts of elementary entities like physical objects, humans, robots etc. and the semantic representations of higher level of abstractions corresponding to structured situations/behaviors.

Using the Formal Representations of “Elementary Events” to Set Up Computational Models of Full “Narratives”

In this chapter, we describe the conceptual tools that, in an NKRL context (NKRL = Narrative Knowledge Representation Language), allow us to obtain a (computer-usable) description of full “narratives” as logically structured associations of the constituting (and duly formalized) “elementary events.” Dealing with this problem means, in practice, being able to formalize those “connectivity phenomena”—denoted, at “surface level,” by logico-semantic coherence links like causality, goal, co-ordination, subordination, indirect speech, etc.—that assure the conceptual unity of a whole narrative. The second-order, unification based solutions adopted by NKRL in this context, “completive construction” and “binding occurrences,” allow us to take into account the connectivity phenomena by “reifying” the formal representations used to model the constitutive elementary events. These solutions, which are of interest from a general digital humanities point of view, are explained in some depth making use of several illustrating examples.

A Computational Model for Dealing with Narratives

In this paper, we provide some details about NKRL (Narrative Knowledge Representation Language), which is both a conceptual language – based on the use of two different ontologies, a ‘standard’ ontology and an ‘event-oriented’ one – and a querying/inference environment. NKRL has been expressly specified and implemented for an ‘intelligent’ representation and management of narratives and event information.