Francesco Setti

F-Formation Detection: Individuating Free-Standing Conversational Groups in Images

Detection of groups of interacting people is a very interesting and useful task in many modern technologies, with application fields spanning from video-surveillance to social robotics. In this paper we first furnish a rigorous definition of group considering the background of the social sciences: this allows us to specify many kinds of group, so far neglected in the Computer Vision literature. On top of this taxonomy we present a detailed state of the art on the group detection algorithms. Then, as a main contribution, we present a brand new method for the automatic detection of groups in still images, which is based on a graph-cuts framework for clustering individuals; in particular, we are able to codify in a computational sense the sociological definition of F-formation, that is very useful to encode a group having only proxemic information: position and orientation of people. We call the proposed method Graph-Cuts for F-formation (GCFF). We show how GCFF definitely outperforms all the state of the art methods in terms of different accuracy measures (some of them are brand new), demonstrating also a strong robustness to noise and versatility in recognizing groups of various cardinality.

Multi-scale f-formation discovery for group detection

We present an unsupervised approach for the automatic detection of static interactive groups. The approach builds upon a novel multi-scale Hough voting policy, which incorporates in a flexible way the sociological notion of group as F-formation; the goal is to model at the same time small arrangements of close friends and aggregations of many individuals spread over a large area. Our technique is based on a competition of different voting sessions, each one specialized for a particular group cardinality; all the votes are then evaluated using information theoretic criteria, producing the final set of groups. The proposed technique has been applied on public benchmark sequences and a novel cocktail party dataset, evaluating new group detection metrics and obtaining state-of-the-art performances.

Group detection in still images by F-formation modeling: A comparative study

Automatically detecting groups of conversing people has become a hot challenge, although a formal, widely-accepted definition of them is lacking. This gap can be filled by considering the social psychological notion of an F-formation as a loose geometric arrangement. In the literature, two main approaches followed this line, exploiting Hough voting [1] from one side and Graph Theory [2] on the other. This paper offers a thorough comparison of these two methods, highlighting the strengths and weaknesses of both in different real life scenarios. Our experiments demonstrate a deeper understanding of the problem by identifying the circumstances in which to adopt a particular method. Finally our study outlines what aspects of the problem are important to address for future improvements to this task.

The S-HOCK dataset: Analyzing crowds at the stadium

The topic of crowd modeling in computer vision usually assumes a single generic typology of crowd, which is very simplistic. In this paper we adopt a taxonomy that is widely accepted in sociology, focusing on a particular category, the spectator crowd, which is formed by people “interested in watching something specific that they came to see” [6]. This can be found at the stadiums, amphitheaters, cinema, etc. In particular, we propose a novel dataset, the Spectators Hockey (S-HOCK), which deals with 4 hockey matches during an international tournament. In the dataset, a massive annotation has been carried out, focusing on the spectators at different levels of details: at a higher level, people have been labeled depending on the team they are supporting and the fact that they know the people close to them; going to the lower levels, standard pose information has been considered (regarding the head, the body) but also fine grained actions such as hands on hips, clapping hands etc. The labeling focused on the game field also, permitting to relate what is going on in the match with the crowd behavior. This brought to more than 100 millions of annotations, useful for standard applications as people counting and head pose estimation but also for novel tasks as spectator categorization. For all of these we provide protocols and baseline results, encouraging further research.

A Unified Framework for Uncertainty, Compatibility Analysis, and Data Fusion for Multi-Stereo 3-D Shape Estimation

This paper describes the uncertainty analysis performed for the reconstruction of a 3-D shape. Multiple stereo systems are employed to measure a 3-D surface with superimposed colored markers. The procedure comprised a detailed uncertainty analysis of all measurement phases, and the uncertainties evaluated were employed to perform a compatibility analysis of points acquired by different stereo pairs. The compatible acquired markers were statistically merged in order to obtain the measurement of a 3-D shape and an evaluation of the associated uncertainty. Both the compatibility analysis and the measurement merging are based on the evaluated uncertainty.

Shape measurement system for single point incremental forming (SPIF) manufacts by using trinocular vision and random pattern

Many contemporary works show the interest of the scientific community in measuring the shape of artefacts made by single point incremental forming. In this paper, we will present an algorithm able to detect feature points with a random pattern, check the compatibility of associations exploiting multi-stereo constraints and reject outliers and perform a 3D reconstruction by dense random patterns. The algorithm is suitable for a real-time application, in fact it needs just three images and a synchronous relatively fast processing. The proposed method has been tested on a simple geometry and results have been compared with a coordinate measurement machine acquisition.

Efficient Second Order Multi-Target Tracking with Exclusion Constraints.

Current state of the art multi-target tracking (MTT) exists in an “either/or” situation. Either a greedy approach can be used, that can make use of second-order information which captures object dynamics, such as “objects tend to move in the same direction over adjacent frames”, or one can use global approaches that make use of the information contained in the entire sequence to resolve ambiguous sub-sequences, but are unable to use such second order information. However, the accurate resolution of ambiguous sequences requires both a good model of object dynamics, and global inference. In this work we present a novel approach to MTT that combines the best of both worlds. By formulating the problem of tracking as one of global MAP estimation over a directed acyclic hyper-graph, we are able to both capture long range interactions, and informative second order priors. In practice, our algorithm is extremely effective, with a run time linear in the number of objects to be tracked, possible locations of an object, and the number of frames. We demonstrate the effectiveness of our approach, both on standard MTT data-sets that contain few objects to be tracked, and on point tracking for non-rigid structure from motion, which, with hundreds of points to be tracked simultaneously, strongly benefits from the efficiency of our approach.

Uncertainty analysis for multi-stereo 3d shape estimation

It is described the uncertainty analysis performed for the reconstruction of a 3D shape. Multiple stereo systems are employed to measure a 3D surface with superimposed colored markers. The described procedure comprises a detailed uncertainty analysis of all the measurement phases, and the evaluated uncertainties are employed to perform a compatibility analysis of points acquired by different stereo pairs. The compatible acquired markers are statistically merged in order to obtain a measurement of a 3D shape and an evaluation of the associated uncertainty. Both the compatibility analysis and the measurement merging is based on the evaluated uncertainty.

A method for asteroids 3D surface reconstruction from close approach distances

We present a procedure for asteroids 3D surface reconstruction from images for close approach distances. Different from other 3D reconstruction scenario from spacecraft images, the closer flyby gave the chance to revolve around the asteroid shape and thus acquiring images from different viewpoints with a higher baseline. The chance to have more information of the asteroids surface is however paid by the loss of correspondences between images given the larger baseline. In this paper we present a procedure used to reconstruct the 3D surface of the asteroid 21 Lutetia encountered by Rosetta spacecraft on July the 10th of 2010 at the closest approach distance of 3170 Km. It was possible to reconstruct a wider surface even dealing with strong ratio of missing data in the measurements. Results show the reconstructed 3D surface of the asteroid as a sparse 3D mesh.

Count on Me: Learning to Count on a Single Image

Individuating and locating repetitive patterns in still images is a fundamental task in image processing, typically achieved by means of correlation strategies. In this paper, we provide a solid solution to this task using a differential geometry approach, operating on Lie algebra, and exploiting a mixture of templates. The proposed method asks the user to locate a few instances of the target patterns (seeds) that become visual templates used to explore the image. We propose an iterative algorithm to locate patches similar to the seeds working in three steps: first, clustering the detected patches to generate templates of different classes, then looking for the affine transformations, living on a Lie algebra that best links the templates and the detected patches, and finally detecting new patches with a convolutional strategy. The process ends when no new patches are found. We will show how our method is able to process heterogeneous unstructured images with multiple visual motifs and extremely crowded scenarios with high precision and recall, outperforming all the state-of-the-art methods.