Sebastiano Vascon

A Game-Theoretic Probabilistic Approach for Detecting Conversational Groups

A standing conversational group (also known as F-formation) occurs when two or more people sustain a social interaction, such as chatting at a cocktail party. Detecting such interactions in images or videos is of fundamental importance in many contexts, like surveillance, social signal processing, social robotics or activity classification. This paper presents an approach to this problem by modeling the socio-psychological concept of an F-formation and the biological constraints of social attention. Essentially, an F-formation defines some constraints on how subjects have to be mutually located and oriented while the biological constraints defines the plausible zone in which persons can interact. We develop a game-theoretic framework embedding these constraints, which is supported by a statistical modeling of the uncertainty associated with the position and orientation of people. First, we use a novel representation of the affinity between pairs of people expressed as a distance between distributions over the most plausible oriented region of attention.Additionally, we integrate temporal information over multiple frames to smooth noisy head orientation and pose estimates, solve ambiguous situations and establish a more precise social context. We do this in a principled way by using recent notions from multi-payoff evolutionary game theory. Experiments on several benchmark datasets consistently show the superiority of the proposed approach over state of the art and its robustness under severe noise conditions.

Detecting conversational groups in images and sequences: A robust game-theoretic approach

Abstract Detecting groups is becoming of relevant interest as an important step for scene (and especially activity) understanding. Differently from what is commonly assumed in the computer vision community, different types of groups do exist, and among these, standing conversational groups (a.k.a. F-formations) play an important role. An F-formation is a common type of people aggregation occurring when two or more persons sustain a social interaction, such as a chat at a cocktail party. Indeed, detecting and subsequently classifying such an interaction in images or videos is of considerable importance in many applicative contexts, like surveillance, social signal processing, social robotics or activity classification, to name a few. This paper presents a principled method to approach to this problem grounded upon the socio-psychological concept of an F-formation. More specifically, a game-theoretic framework is proposed, aimed at modeling the spatial structure characterizing F-formations. In other words, since F-formations are subject to geometrical configurations on how humans have to be mutually located and oriented, the proposed solution is able to account for these constraints while also statistically modeling the uncertainty associated with the position and orientation of the engaged persons. Moreover, taking advantage of video data, it is also able to integrate temporal information over multiple frames utilizing the recent notions from multi-payoff evolutionary game theory. The experiments have been performed on several benchmark datasets, consistently showing the superiority of the proposed approach over the state of the art, and its robustness under severe noise conditions.

Detecting emergent leader in a meeting environment using nonverbal visual features only

In this paper, we propose an effective method for emergent leader detection in meeting environments which is based on nonverbal visual features. Identifying emergent leader is an important issue for organizations. It is also a well-investigated topic in social psychology while a relatively new problem in social signal processing (SSP). The effectiveness of nonverbal features have been shown by many previous SSP studies. In general, the nonverbal video-based features were not more effective compared to audio-based features although, their fusion generally improved the overall performance. However, in absence of audio sensors, the accurate detection of social interactions is still crucial. Motivating from that, we propose novel, automatically extracted, nonverbal features to identify the emergent leadership. The extracted nonverbal features were based on automatically estimated visual focus of attention which is based on head pose. The evaluation of the proposed method and the defined features were realized using a new dataset which is firstly introduced in this paper including its design, collection and annotation. The effectiveness of the features and the method were also compared with many state of the art features and methods.

Nanoscale molecular reorganization of the inhibitory postsynaptic density is a determinant of GABAergic synaptic potentiation

Gephyrin is a key scaffold protein mediating the anchoring of GABAA receptors at inhibitory synapses. Here, we exploited superresolution techniques combined with proximity-based clustering analysis and model simulations to investigate the single-molecule gephyrin reorganization during plasticity of inhibitory synapses in mouse hippocampal cultured neurons. This approach revealed that, during the expression of inhibitory LTP, the increase of gephyrin density at postsynaptic sites is associated with the promoted formation of gephyrin nanodomains. We demonstrate that the gephyrin rearrangement in nanodomains stabilizes the amplitude of postsynaptic currents, indicating that, in addition to the number of synaptic GABAA receptors, the nanoscale distribution of GABAA receptors in the postsynaptic area is a crucial determinant for the expression of inhibitory synaptic plasticity. In addition, the methodology implemented here clears the way to the application of the graph-based theory to single-molecule data for the description and quantification of the spatial organization of the synapse at the single-molecule level. SIGNIFICANCE STATEMENT The mechanisms of inhibitory synaptic plasticity are poorly understood, mainly because the size of the synapse is below the diffraction limit, thus reducing the effectiveness of conventional optical and imaging techniques. Here, we exploited superresolution approaches combined with clustering analysis to study at unprecedented resolution the distribution of the inhibitory scaffold protein gephyrin in response to protocols inducing LTP of inhibitory synaptic responses (iLTP). We found that, during the expression of iLTP, the increase of synaptic gephyrin is associated with the fragmentation of gephyrin in subsynaptic nanodomains. We demonstrate that such synaptic gephyrin nanodomains stabilize the amplitude of inhibitory postsynaptic responses, thus identifying the nanoscale gephyrin rearrangement as a key determinant for inhibitory synaptic plasticity.

Automatic White Matter Fiber Clustering Using Dominant Sets

We present an unsupervised approach based on the Dominant Sets framework to automatically segment the white matter fibers into bundles. This framework, rooted in the Game Theory, allows for the automatic determination of the number of clusters from the data itself, without any prior assumption. The clustered bundles are a key information for the generation of unbiased structural connectivity atlases. We have thoroughly validated our algorithm both quantitatively and qualitatively. Indeed, we used biologically plausible synthetic datasets to numerically validate the performance in terms of Precision, Recall and other measures employed in the literature. We also evaluated the algorithm on a real Diffusion Tensor Imaging tractography of a whole mouse brain obtaining promising results. In fact, some of the most prominent brain structures determined by the algorithm correspond to white matter expected anatomy.

Automated multi-subject fiber clustering of mouse brain using dominant sets

Mapping of structural and functional connectivity may provide deeper understanding of brain function and disfunction. Diffusion Magnetic Resonance Imaging (DMRI) is a powerful technique to non-invasively delineate white matter (WM) tracts and to obtain a three-dimensional description of the structural architecture of the brain. However, DMRI tractography methods produce highly multi-dimensional datasets whose interpretation requires advanced analytical tools. Indeed, manual identification of specific neuroanatomical tracts based on prior anatomical knowledge is time-consuming and prone to operator-induced bias. Here we propose an automatic multi-subject fiber clustering method that enables retrieval of group-wise WM fiber bundles. In order to account for variance across subjects, we developed a multi-subject approach based on a method known as Dominant Sets algorithm, via an intra- and cross-subject clustering. The intra-subject step allows us to reduce the complexity of the raw tractography data, thus obtaining homogeneous neuroanatomically-plausible bundles in each diffusion space. The cross-subject step, characterized by a proper space-invariant metric in the original diffusion space, enables the identification of the same WM bundles across multiple subjects without any prior neuroanatomical knowledge. Quantitative analysis was conducted comparing our algorithm with spectral clustering and affinity propagation methods on synthetic dataset. We also performed qualitative analysis on mouse brain tractography retrieving significant WM structures. The approach serves the final goal of detecting WM bundles at a population level, thus paving the way to the study of the WM organization across groups.

Context aware nonnegative matrix factorization clustering

In this article we propose a method to refine the clustering results obtained with the nonnegative matrix factorization (NMF) technique, imposing consistency constraints on the final labeling of the data. The research community focused its effort on the initialization and on the optimization part of this method, without paying attention to the final cluster assignments. We propose a game theoretic framework in which each object to be clustered is represented as a player, which has to choose its cluster membership. The information obtained with NMF is used to initialize the strategy space of the players and a weighted graph is used to model the interactions among the players. These interactions allow the players to choose a cluster which is coherent with the clusters chosen by similar players, a property which is not guaranteed by NMF, since it produces a soft clustering of the data. The results on common benchmarks show that our model is able to improve the performances of many NMF formulations.

Using Dominant Sets for k-NN Prototype Selection

k-Nearest Neighbors is surely one of the most important and widely adopted non-parametric classification methods in pattern recognition. It has evolved in several aspects in the last 50 years, and one of the most known variants consists in the usage of prototypes: a prototype distills a group of similar training points, diminishing drastically the number of comparisons needed for the classification; actually, prototypes are employed in the case the cardinality of the training data is high. In this paper, by using the dominant set clustering framework, we propose four novel strategies for the prototype generation, allowing to produce representative prototypes that mirror the underlying class structure in an expressive and effective way. Our strategy boosts the k-NN classification performance; considering heterogeneous metrics and analyzing 15 diverse datasets, we are among the best 6 prototype-based k-NN approaches, with a computational cost which is strongly inferior to all the competitors. In addition, we show that our proposal beats linear SVM in the case of a pedestrian detection scenario.

On Association Graph Techniques for Hypergraph Matching

Association graph techniques represent a classical approach to tackle the graph matching problem and recently the idea has been generalized to the case of hypergraphs. In this paper, we explore the potential of this approach in conjunction with a class of dynamical systems derived from the Baum-Eagon inequality. In particular, we focus on the pure isomorphism case and show, with extensive experiments on a large synthetic dataset, that despite its simplicity the Baum-Eagon dynamics does an excellent job at finding globally optimal solutions.

Group Detection and Tracking Using Sociological Features

Abstract This chapter describes the most common features and definitions from the sociological science used to detect and track groups of people that are interacting. The necessity of having reliable algorithms to cope with these problems is gaining increasing interest, especially in the fields related to security and video surveillance. Answering the question of “who is present and with whom he/she is interacting in a scene?” is nowadays of utmost importance. Other domains require having good algorithms to face these problems, for example, activity recognition, social robotics, and automatic behavior analysis. The success of detection and tracking algorithms relies on the engineering of the features. In this context, the literature of sociological sciences gives us a set of well-established assumptions and constraints to create more reliable and plausible features and detection algorithms. In this chapter we will describe the existing features of the following two categories: the low-level category used to determine the spatial properties of each person in a scene (person position and head/body orientation), and the high-level category that agglomerates or uses the low-level features to implement sociological and biological definitions (frustum of visual attention). We will see how these features are used by the popular methods of group detection, such as game theory-based and probabilistic approaches. Finally, we will analyze a tracking model that can be integrated with the analyzed features and the described detection methods. The experimental part provides a comprehensive comparison of the performances of different algorithms to detect and track groups on standard and publicly available benchmarks.