Nicola Conci

Video-Based Human Behavior Understanding: A Survey

Understanding human behaviors is a challenging problem in computer vision that has recently seen important advances. Human behavior understanding combines image and signal processing, feature extraction, machine learning, and 3-D geometry. Application scenarios range from surveillance to indexing and retrieval, from patient care to industrial safety and sports analysis. Given the broad set of techniques used in video-based behavior understanding and the fast progress in this area, in this paper we organize and survey the corresponding literature, define unambiguous key terms, and discuss links among fundamental building blocks ranging from human detection to action and interaction recognition. The advantages and the drawbacks of the methods are critically discussed, providing a comprehensive coverage of key aspects of video-based human behavior understanding, available datasets for experimentation and comparisons, and important open research issues.

Matador: Mobile task detector for context-aware crowd-sensing campaigns

Ubiquity of internet-connected media- and sensor-equipped portable devices is enabling a new class of applications which exploit the power of crowds to perform sensing tasks in the real world. Such paradigm is referred as crowd-sensing, and lies at the intersection of crowd-sourcing and participatory sensing. This has a wide range of potential applications such as direct involvement of citizens into public decision making. In this work we present Matador, a framework to embed context-awareness in the presentation and execution of crowd-sensing tasks. This allows to present the right tasks, to the right users in the right circumstances, and to preserve normal device functioning. We present the design and prototype implementation of the platform, including an energy-efficient context sampling algorithm. We validate the proposed approach through a numerical study and a small pilot, and demonstrate the ability of the proposed system to efficiently deliver crowd-sensing tasks, while minimizing the consumption of mobile device resources.

Syntactic Matching of Trajectories for Ambient Intelligence Applications

In this paper we propose a novel approach for syntactic description and matching of object trajectories in digital video, suitable for classification and recognition purposes. Trajectories are first segmented by detecting the meaningful discontinuities in time and space, and are successively expressed through an ad-hoc syntax. A suitable metric is then proposed, which allows determining the similarity among trajectories, based on the so-called inexact or approximate matching. The metric mimics the algorithms used in bio-informatics to match DNA sequences, and returns a score, which allows identifying the analogies among different trajectories on both global and local basis. The tool can therefore be adopted for the analysis, classification, and learning of motion patterns, in activity detection or behavioral understanding.

Context-aware mobile crowdsourcing

Ubiquity of internet-connected media-and sensor-equipped portable devices has emerged a range of opportunities for direct involvement of citizens into public decision making, leading to a new participatory format of public administration functioning. Intersecting the power of the crowdsourcing problem-solving paradigm by directly relying on human intelligence, with instantaneity and situation-awareness of mobile technologies, one gets a context-aware crowdsourcing approach for problem-solving in the right circumstances with the right people. In this paper, we present a prototype implementation of a context-aware mobile crowdsourcing system that enables the deployment and execution of crowdsourcing campaigns with users carrying mobile devices. The system is designed to maximize conditions for user participation, while minimizing the usage of energy. The paper describes the system architecture, defines an optimized sampling algorithm, and outlines a preliminary experimentation study carried out.

Object Trajectory Analysis in Video Indexing and Retrieval Applications

The focus of this chapter is to present a survey on the most recent advances in representation and analysis of video object trajectories, with application to indexing and retrieval systems. We will review the main methodologies for the description of motion trajectories, as well as the indexing techniques and similarity metrics used in the retrieval process. Strengths and weaknesses of different solutions will be discussed through a comparative analysis, taking into account performance and implementation issues. In order to provide a deeper insight on the exploitation of these technologies in real world products, a selection of exampleswill be introduced and examined. The set of possible applications is very wide and includes (but it is not limited to) generic browsing of video databases, as well as more specific and context-dependent scenarios such as indexing and retrieval in visual surveillance, traffic monitoring, sport events analysis, video-on-demand, and video broadcasting.

Real time detection of social interactions in surveillance video

In this paper we present a novel method to detect the presence of social interactions occurring in a surveillance scenario. The algorithm we propose complements motion features with proxemics cues, so as to link the human motion with the contextual and environmental information. The extracted features are analyzed through a multi-class SVM. Testing has been carried out distinguishing between casual and intentional interactions, where intentional events are further subdivided into normal and abnormal behaviors. The algorithm is validated on benchmark datasets, as well as on a new dataset specifically designed for interactions analysis.

The S-HOCK dataset: Analyzing crowds at the stadium

The topic of crowd modeling in computer vision usually assumes a single generic typology of crowd, which is very simplistic. In this paper we adopt a taxonomy that is widely accepted in sociology, focusing on a particular category, the spectator crowd, which is formed by people “interested in watching something specific that they came to see” [6]. This can be found at the stadiums, amphitheaters, cinema, etc. In particular, we propose a novel dataset, the Spectators Hockey (S-HOCK), which deals with 4 hockey matches during an international tournament. In the dataset, a massive annotation has been carried out, focusing on the spectators at different levels of details: at a higher level, people have been labeled depending on the team they are supporting and the fact that they know the people close to them; going to the lower levels, standard pose information has been considered (regarding the head, the body) but also fine grained actions such as hands on hips, clapping hands etc. The labeling focused on the game field also, permitting to relate what is going on in the match with the crowd behavior. This brought to more than 100 millions of annotations, useful for standard applications as people counting and head pose estimation but also for novel tasks as spectator categorization. For all of these we provide protocols and baseline results, encouraging further research.

Optimal configuration of PTZ camera networks based on visual quality assessment and coverage maximization

In this paper we present a novel method for video cameras positioning and reconfiguration, to maximize visual coverage in complex indoor environments. Based on a suitable modeling of the camera field-of-view and of the environmental setup, the optimization procedure determines the most appropriate configuration of cameras to satisfy a coverage objective, taking into account a number of parameters on the quality of view at the camera position. This includes the global ground area coverage, the expected geometric distortion, and the entropy of the image. The proposed solution has been validated in different environmental setups, including synthetic settings, taking into account the presence of obstacles and constraints.

Global Coverage Maximization in PTZ-Camera Networks Based on Visual Quality Assessment

In this paper, we propose a novel method to automatically configure a pan-tilt-zoom camera network in order to maximize coverage and visual quality in complex indoor environments. Based on a suitable modeling of cameras and environment, the optimization procedure determines the most appropriate camera position and settings to fulfill a given coverage objective. To achieve this goal, we use a particle swarm optimizer with an appropriate fitness function that takes into account a number of concurrent metrics and constraints. Furthermore, the solution is found working into a simulated environment obtained with the ray-tracing software. The proposed solution has been tested in various synthetic and real environments, taking into account the presence of obstacles and other constraints. We also simulated dynamic situations, such as cameras failures, to test the capability of fast camera network reconfiguration. For the performance evaluation, and in addition to the simulation in virtual scenes, we have also conducted a validation phase in real-world scenarios, where we assessed how the introduction on the optimal configuration improves the solution of some typical computer vision tasks.

Collaborative creativity: The Music Room

In this paper, we reflect on our experience of designing, developing and evaluating interactive spaces for collaborative creativity. In particular, we are interested in designing spaces which allow everybody to compose and play original music. The Music Room is an interactive installation where couples can compose original music by moving in the space. Following the metaphor of love, the music is automatically generated and modulated in terms of pleasantness and intensity, according to the proxemics cues extracted from the visual tracking algorithm. The Music Room was exhibited during the EU Researchers’ Night in Trento, Italy.