Pietro Morerio

The evolution of first person vision methods : a survey

The emergence of new wearable technologies, such as action cameras and smart glasses, has increased the interest of computer vision scientists in the first person perspective. Nowadays, this field is attracting attention and investments of companies aiming to develop commercial devices with first person vision (FPV) recording capabilities. Due to this interest, an increasing demand of methods to process these videos, possibly in real time, is expected. The current approaches present a particular combinations of different image features and quantitative methods to accomplish specific objectives like object detection, activity recognition, user-machine interaction, and so on. This paper summarizes the evolution of the state of the art in FPV video analysis between 1997 and 2014, highlighting, among others, the most commonly used features, methods, challenges, and opportunities within the field.

Bio-inspired relevant interaction modelling in cognitive crowd management

AbstractCognitive algorithms, integrated in intelligent systems, represent an important innovation in designing interactive smart environments. More in details, Cognitive Systems have important applications in anomaly detection and management in advanced video surveillance. These algorithms mainly address the problem of modelling interactions and behaviours among the main entities in a scene. A bio-inspired structure is here proposed, which is able to encode and synthesize signals, not only for the description of single entities behaviours, but also for modelling cause–effect relationships between user actions and changes in environment configurations. Such models are stored within a memory (Autobiographical Memory) during a learning phase. Here the system operates an effective knowledge transfer from a human operator towards an automatic systems called Cognitive Surveillance Node (CSN), which is part of a complex cognitive JDL-based and bio-inspired architecture. After such a knowledge-transfer phase, learned representations can be used, at different levels, either to support human decisions, by detecting anomalous interaction models and thus compensating for human shortcomings, or, in an automatic decision scenario, to identify anomalous patterns and choose the best strategy to preserve stability of the entire system. Results are presented in a video surveillance scenario , where the CSN can observe two interacting entities consisting in a simulated crowd and a human operator. These can interact within a visual 3D simulator, where crowd behaviour is modelled by means of Social Forces. The way anomalies are detected and consequently handled is demonstrated, on synthetic and also on real video sequences, in both the user-support and automatic modes.

People Count Estimation In Small Crowds

This work addresses the problem of people counting in crowded situations, such as urban environments, in computer vision. As crowding density increases in a scene, it might become impossible to count people as single individuals: a global group-based approach is then preferable and in fact often necessary. A simple method for estimating the count of people in such tight crowds is here proposed, relying on accurate camera calibration. A training phase is also needed by the algorithm in order to learn the parameters needed for estimation.

Event Based Switched Dynamic Bayesian Networks for Autonomous Cognitive Crowd Monitoring

Human behavior analysis is one of the most important applications in Intelligent Video Surveillance (IVS) field. In most recent systems addressed by research, automatic support to the human decisions based on object detection, tracking and situation assessment tools is integrated as a part of a complete cognitive artificial process including security maintenance procedures actions that are in the scope of the system. In such cases an IVS needs to represent complex situations that describe alternative possible real time interactions between the dynamic observed situation and operators’ actions. To obtain such knowledge, particular types of Event based Dynamic Bayesian Networks E-DBNs are here proposed that can switch among alternative Bayesian filtering and control lower level modules to capture adaptive reactions of human operators. It is shown that after the off line learning phase Switched E-DBNs can be used to represent and anticipate possible operators’ actions within the IVS. In this sense acquired knowledge can be used for either fully autonomous security preserving systems or for training of new operators. Results are shown by considering a crowd monitoring application in a critical infrastructure. A system is presented where a Cognitive Node (CN) embedding in a structured way Switched E-DBN knowledge can interact with an active visual simulator of crowd situations. It is also shown that outputs from such a simulator can be easily compared with video signals coming from real cameras and processed by typical Bayesian tracking methods.

Towards a unified framework for hand-based methods in First Person Vision

First Person Vision (Egocentric) video analysis stands nowadays as one of the emerging fields in computer vision. The availability of wearable devices recording exactly what the user is looking at is ineluctable and the opportunities and challenges carried by this kind of devices are broad. Particularly, for the first time a device is so intimate with the user to be able to record the movements of his hands, making hand-based applications for First Person Vision one the most explored area in the field. This paper explores the more popular processing steps to develop hand-based applications, and proposes a hierarchical structure that optimally switches between each of the levels to reduce the computational cost of the system and improve its performance.

Early fire and smoke detection based on colour features and motion analysis

This work addresses the issue of fire and smoke detection in a scene within a video surveillance framework. Detection of fire and smoke pixels is at first achieved by means of a motion detection algorithm. In addition, separation of smoke and fire pixels using colour information (within appropriate spaces, specifically chosen in order to enhance specific chromatic features) is performed. In parallel, a pixel selection based on the dynamics of the area is carried out in order to reduce false detection. The output of the three parallel algorithms are eventually fused by means of a MLP.

Filtering SVM frame-by-frame binary classification in a detection framework

Classifying frames, or parts of them, is a common way of carrying out detection tasks in computer vision. However, frame by frame classification suffers from sudden significant variations in image texture, colour and luminosity, resulting in noise in the extracted features and consequently in the decisions taken. Support Vector Machines have been widely validated as powerful tools for frame by frame detection of non-separable datasets, but are extremely sensitive to these variations between adjacent frames, creating as consequence sudden flickering in the classification results. This work proposes a Dynamic Bayesian Network to smooth the classification results of Support Vector Machines (SVM) in detection tasks. The method is evaluated in First Person Vision (FPV) videos, where a SVM is used to decide whether or not the users hands are in his field of view.

A Dynamic Approach and a New Dataset for Hand-detection in First Person Vision

Hand detection and segmentation methods stand as two of the most most prominent objectives in First Person Vision. Their popularity is mainly explained by the importance of a reliable detection and location of the hands to develop human-machine interfaces for emergent wearable cameras. Current developments have been focused on hand segmentation problems, implicitly assuming that hands are always in the field of view of the user. Existing methods are commonly presented with new datasets. However, given their implicit assumption, none of them ensure a proper composition of frames with and without hands, as the hand-detection problem requires. This paper presents a new dataset for hand-detection, carefully designed to guarantee a good balance between positive and negative frames, as well as challenging conditions such as illumination changes, hand occlusions and realistic locations. Additionally, this paper extends a state-of-the-art method using a dynamic filter to improve its detection rate. The improved performance is proposed as a baseline to be used with the dataset.

Distributed cognitive radio architecture with automatic frequency switching

The employment of sophisticated tools for data analysis in distributed or structurally complex systems requires the development of specific architectures and data fusion strategies in order to integrate heterogeneous information coming from the environmental sensors. Recently, intelligent sensor networks have been widely deployed for various purposes concerning both security- and safety-oriented systems. Military and civil applications ranging from border surveillance and public spaces monitoring to ambient intelligence and road safety are examples of such various applications. The architecture presented in this article is based on the Cognitive Node (CN) - a module able to receive data from the sensors, process it in order to find potentially harmful or anomalous events and situations and, in some cases, to interact with the environment itself or contact the human operator. The cognitive model was studied and exploited, focusing on the analysis and decision blocks which represent the crucial phases for assessing potentially unsecure/unsafe events and/or situations. The scalability of the model with regards to different application domains was investigated during the research activity. Proposed results show the capability of the given architecture for analysis and assessment of the occurring interactions, with the goal of maintaining proper security/safety levels in the monitored environment.

Optimizing Superpixel Clustering for Real-Time Egocentric-Vision Applications

In this work, we propose a strategy for optimizing a superpixel algorithm for video signals, in order to get closer to real time performances which are on the one hand needed for egocentric vision applications and on the other must be bearable by wearable technologies. Instead of applying the algorithm frame by frame, we propose a technique inspired to Bayesian filtering and to video coding which allows to re-initialize superpixels using the information from the previous frame. This results in faster convergence and demonstrates how performances improve with respect to the standard application of the algorithm from scratch at each frame.