Rafael Martínez-Tomás

On the correspondence between objects and events for the diagnosis of situations in visual surveillance tasks

A key problem in visual surveillance systems (VSS) is to find an effective procedure for linking the geometric descriptions of a scene at the object level with the corresponding descriptions of the agents intervening in this scene at the activity level. In this work, we explore a constructivist approach based on using the usual Artificial Intelligence (AI) techniques and methods to establish correspondences between the entities and relations of the ontologies in these two levels. The proposal is exemplified using a real interior scenario that uses images from just one fixed camera and where the purpose of the surveillance is to do a preventive diagnosis of the activity of abandoning a potentially dangerous object in a sensitive area. The work stresses: (1) anchoring the object-level labels in the result of analytical processes on blobs, (2) specifying contextual knowledge that has to be injected to link the activities, as described by a human surveillance expert, with the objects, as they are labelled by the same expert from geometric descriptions. The work is set within the context of the 50th anniversary of AI and the leading theories on human visual perception.

A multisensory monitoring and interpretation framework based on the model-view-controller paradigm

This paper proposes a monitoring and interpretation framework inspired in the Model-View-Controller (MVC) paradigm. Indeed, the paper proposes the extension of the traditional MVC paradigm to make it more flexible in incorporating the functionalities of a monitoring and interpretation system. The proposed model is defined as a hybrid distributed system where remote nodes perform lower level processing as well as data acquisition, while a central node is in charge of collecting the information and of its fusion. Firstly, the framework levels as well as their functionalities are described. Then, a fundamental part of the proposed framework, namely the common model, is introduced.

On the effect of feedback in multilevel representation spaces for visual surveillance tasks

In this work we propose a general top-down feedback scheme between adjacent description levels to interpret video sequences. This scheme distinguishes two types of feedback: repair-oriented feedback and focus-oriented feedback. With the first it is possible to improve the systems performance and produce more reliable and consistent information, and with the second it is possible to adjust the computational load to match the aims. Finally, the general feedback scheme is used in different examples for a visual surveillance application which improved the final result of each description level by using the information in the higher adjacent level.

Intelligent multisensory systems in support of information society

Nowadays, our environment is evolving more than ever in terms of connectivity and access to information, among others. People connected through social networks, such as Twitter or Facebook, upload photos, publish news about social events, etc. Many sensors that provide us with information about temperature, pollution, traffic congestion and energy consumption, among others, are being installed everywhere. Even our closest environment has greatly changed so that it now provides more information by means of motion sensors, surveillance cameras, etc. This is one of the results of the so-called information society becoming a reality. However, the availability of such a vast amount and diversity of information can be overwhelming for humans as it has to be processed, fused and analysed for building knowledge that is used to make decisions, recognise situations, etc. It is in this context where intelligent multisensory systems provide support to the information society. These systems are usually defined by two different features: (1) they monitor several types of sensors so that they are able to perceive the same environment from very different perspectives and (2) they interpret the activities that are taking place in the monitored environment by using some kind of artificial intelligence technique, such as the multi-agent systems paradigm. These two features, monitoring and interpretation, have turned these systems into one of the most appropriate solutions for a wide variety of scenarios. For instance, they have proved their usefulness for the monitoring and surveillance of physical environments to recognise situations, activities and interactions among the different participating agents (e.g. humans, cars, aircraft, vessels or ships). This scenario is not restricted only to visual sensors, but also to other sensors capable of complementing and/or confirming the information extracted from video signals. Examples of this scenario can be as different as systems able to detect falls of elderly adults and to put into practice emergency plans, or systems able to support smart grids for the intelligent monitoring of distribution substations to detect events of interest, such as normal voltage values or unbalanced intensity values that can end up blowing fuses and decreasing the quality of service of end consumers. Another interesting scenario is the management of context-aware information in ambient intelligent environments. This involves the gathering, fusion, processing and inference of information in sensor-oriented infrastructures to support ubiquitous applications. For instance, there are systems that provide caregivers with quick and accurate locations of their charges, access to critical treatment, etc. Due to the inherent complexity of intelligent multisensory systems, their development becomes a challenging and demanding activity, mainly because three different research areas have to team up for providing appropriate solutions: sensor networks, artificial intelligence and human– computer interaction. The main objective of this special issue is to summarise recent advances in the area of intelligent multisensory systems. The published papers provide new ideas and problems, clearly indicating the advances made in problem statements, methodologies and applications with respect to the existing results. The special issue also includes papers focusing on advanced methods and presenting considerable novelties in theoretical background and experimental set-up. Some papers introduce applications to newly emerging fields, such as pervasive environments or sensor networks. The advances in the sensor networks research area are critical for the development of intelligent multisensory systems as they determine both their monitoring capabilities and one main cost of the deployment of the systems. For this aim, the development of new kinds of sensors, the development of platforms for their control, as well as the identification of the type of sensors to be used and how they should be distributed over the area to be monitored, are critical factors for these systems. In this special issue, two papers describe relevant advances related to sensor networks. Castillo, Engin, and Feliu Batlle (2014) introduce a two-degree-of-freedom flexible antenna sensor platform designed to physically simulate the ability of a robotic arm, which rapidly reorientates and targets itself towards specific surfaces from different approachable angles. Gascueña, Castillo, Navarro, and Fernández-Caballero (2014) introduce a methodology called INT3-software development process (INT3-SDP), which provides the analysts with the guidelines and models necessary for the description of the environment to be monitored and the sensors to be installed, as well as in the implementation of the software components that perform the INTerpretation of behaviours and situations for an INTelligent INTervention in complex and dynamic environments.

Using semantic technologies and the OSU ontology for modelling context and activities in multi-sensory surveillance systems

Automatic systems that monitor human behaviour for detecting security problems are a challenge today. Previously, our group defined the Horus framework, which is a modular architecture for the integration of multi-sensor monitoring stages. In this work, structure and technologies required for high-level semantic stages of Horus are proposed, and the associated methodological principles established with the aim of recognising specific behaviours and situations. Our methodology distinguishes three semantic levels of events: low level (compromised with sensors), medium level (compromised with context), and high level (target behaviours). The ontology for surveillance and ubiquitous computing has been used to integrate ontologies from specific domains and together with semantic technologies have facilitated the modelling and implementation of scenes and situations by reusing components. A home context and a supermarket context were modelled following this approach, where three suspicious activities were monitored via different virtual sensors. The experiments demonstrate that our proposals facilitate the rapid prototyping of this kind of systems.

The Dissociation between Polarity, Semantic Orientation, and Emotional Tone as an Early Indicator of Cognitive Impairment

The present study aims to identify early cognitive impairment through the efficient use of therapies that can improve the quality of daily life and prevent disease progress. We propose a methodology based on the hypothesis that the dissociation between oral semantic expression and the physical expressions, facial gestures, or emotions transmitted in a persons tone of voice is a possible indicator of cognitive impairment. Experiments were carried out with phrases, analyzing the semantics of the message, and the tone of the voice of patients through unstructured interviews in healthy people and patients at an early Alzheimers stage. The results show that the dissociation in cognitive impairment was an effective indicator, arising from patterns of inconsistency between the analyzed elements. Although the results of our study are encouraging, we believe that further studies are necessary to confirm that this dissociation is a probable indicator of cognitive impairment.

Automatic Drawing Analysis of Figures Included in Neuropsychological Tests for the Assessment and Diagnosis of Mild Cognitive Impairment

This proposal is framed within the group’s general working line of applying artificial intelligence techniques to advance in early mild cognitive impairment diagnosis. If impairment in semantic production was studied in previous works, now we rely on the reduced ability to reproduce or copy simple figures, part of standardized neuropsychological tests designed to assess mild cognitive impairment. Although the long-term goal of this project is to work with all figures from these tests, in this paper we will focus on the automatic analysis of the alternating graphs figure. We develop a quantitative descrition of different features that appear to be very abstract in the test norms and define new features that are not considered so far. Results with just one figure are quite promising (77.7% precision and 77.1 recall).

Smart Spaces and Monitoring Simulation

This piece of work is framed in our group research and is about behaviors identification of the smart spaces monitoring. Particularly, if this monitoring uses cameras, some habitual problems related to lights and shadows in the scene because they make the task of recognition difficult, the identification of objects and even the images captured to analyze the correct functioning of these systems appear. However, the experimentation is difficult in any case because the installation of cameras and other sensors are laborious and it can entail a high cost. Thus, to solve these problems we are developing a simulator in order to create a scene with actors, objects and sensors, which the user requires and can reproduce a defined plot. Thereby we can get two objectives: a) Check the correct location and appropriate characteristics of the equipment installed, virtually, and b) The system can emulate the event generations of the real system and with this it can confirm the utility of the installation. In the global frame of our research, besides, it allows to unhook of the physical level and focus on the behavior interpretation from that virtual smart space. Thus, the problems of generating images and events, which probe the correct functioning of the systems and other possible failures derived directly from the images captured by the devices, are solved.

SBRS: Bridging the Gap between Biomedical Research and Clinical Practice

The field of Biomedical research is currently one with the greatest social impact and publication volume, providing continuous advances and results which should, to a great extent, reach the general clinical practice. Similarly, direct clinical experience may offer experimental results and conclusions which may lead, guide and foster new investigations. However, this interaction between research and clinical practice is yet too far from being optimal. On one side, research results are published without standardization, suffering terminological issues, which prevent its automatic handling and great scale information treatment/management. On the other, for the practitioner, the task of reviewing papers, bibliography, experimental results, etc. in order to keep updated his everyday clinical practice, is very time consuming, causing not to be done continuously.

Bayesian network-based model for the diagnosis of deterioration of semantic content compatible with Alzheimer's disease

Alzheimers Disease (AD) has become a serious public health problem that affects both the patient and his family and social environment, not to mention the high economic cost for families and public administrations. The early detection of AD has become one of the principal focuses of research, and its diagnosis is fundamental when the disease is incipient or even prodromic, because it is at these stages when treatments are more effective. There are numerous research studies to characterise the disease in these stages, and we have used the specific research carried out by Drs. Herminia Peraita and Lina Grasso. The application of Artificial Intelligence techniques, such as Bayesian Networks and Influence Diagrams, may provide a very valuable contribution both to the very research and the application of results. This article justifies using Bayesian Networks and Influence Diagrams to solve this type of problems and because of their great contribution to this application field. The modelling techniques used for constructing the Bayesian Network are mentioned in this article, and a mechanism for automatic learning of the model parameters is established.