Francesca Capozzi

Decoding intentions from movement kinematics

How do we understand the intentions of other people? There has been a longstanding controversy over whether it is possible to understand others’ intentions by simply observing their movements. Here, we show that indeed movement kinematics can form the basis for intention detection. By combining kinematics and psychophysical methods with classification and regression tree (CART) modeling, we found that observers utilized a subset of discriminant kinematic features over the total kinematic pattern in order to detect intention from observation of simple motor acts. Intention discriminability covaried with movement kinematics on a trial-by-trial basis, and was directly related to the expression of discriminative features in the observed movements. These findings demonstrate a definable and measurable relationship between the specific features of observed movements and the ability to discriminate intention, providing quantitative evidence of the significance of movement kinematics for anticipating others’ intentional actions.

Detecting emergent leader in a meeting environment using nonverbal visual features only

In this paper, we propose an effective method for emergent leader detection in meeting environments which is based on nonverbal visual features. Identifying emergent leader is an important issue for organizations. It is also a well-investigated topic in social psychology while a relatively new problem in social signal processing (SSP). The effectiveness of nonverbal features have been shown by many previous SSP studies. In general, the nonverbal video-based features were not more effective compared to audio-based features although, their fusion generally improved the overall performance. However, in absence of audio sensors, the accurate detection of social interactions is still crucial. Motivating from that, we propose novel, automatically extracted, nonverbal features to identify the emergent leadership. The extracted nonverbal features were based on automatically estimated visual focus of attention which is based on head pose. The evaluation of the proposed method and the defined features were realized using a new dataset which is firstly introduced in this paper including its design, collection and annotation. The effectiveness of the features and the method were also compared with many state of the art features and methods.

One is not enough: Group size modulates social gaze-induced object desirability effects

Affective evaluations of objects are influenced by the preferences expressed by other people via their gaze direction, so that objects looked at are liked more than objects looked away from. But when can others’ preferences be trusted? Here, we show that group size influences the extent to which individuals tend to conform to others’ gaze preferences. We adopted the conventional gaze-cuing paradigm and modified the design in such a way that some objects were consistently cued by only one face (single-face condition), whereas other objects were consistently cued by several different faces (multiple-faces condition). While response time measures revealed equal gaze-cuing effects for both conditions, a boost in affective evaluation was observed only for objects looked at by several different faces. Objects looked at by a single face were not rated differently than objects looked away from. These findings suggest that observers make use of group size to evaluate the generalizability of the epistemic information conveyed by others’ gaze: Objects looked at are liked more than objects looked away from, but only when they are looked at by multiple faces.

Altercentric intrusions from multiple perspectives: beyond dyads.

Recent findings suggest that in dyadic contexts observers rapidly and involuntarily process the visual perspective of others and cannot easily resist interference from their viewpoint. To investigate whether spontaneous perspective taking extends beyond dyads, we employed a novel visual perspective task that required participants to select between multiple competing perspectives. Participants were asked to judge their own perspective or the visual perspective of one or two avatars who either looked at the same objects or looked at different objects. Results indicate that when a single avatar was present in the room, participants processed the irrelevant perspective even when it interfered with participants’ explicit judgments about the relevant perspective. A similar interference effect was observed when two avatars looked at the same discs, but not when they looked at different discs. Indeed, when the two avatars looked at different discs, the interference from the irrelevant perspective was significantly reduced. This is the first evidence that the number and orientation of agents modulate spontaneous perspective taking in non-dyadic contexts: observers may efficiently compute another’s perspective, but in presence of more individuals holding discrepant perspectives, they may not spontaneously track multiple viewpoints. These findings are discussed in relation to the hypothesis that perspective calculation occurs in an effortless and automatic manner.

When Far Becomes Near: Perspective Taking Induces Social Remapping of Spatial Relations

On many occasions, people spontaneously or deliberately take the perspective of a person facing them rather than their own perspective. How is this done? Using a spatial perspective task in which participants were asked to identify objects at specific locations, we found that self-perspective judgments were faster for objects presented to the right, rather than the left, and for objects presented closer to the participants’ own bodies. Strikingly, taking the opposing perspective of another person led to a reversal (i.e., remapping) of these effects, with reference to the other person’s position (Experiment 1). A remapping of spatial relations was also observed when an empty chair replaced the other person (Experiment 2), but not when access to the other viewpoint was blocked (Experiment 3). Thus, when the spatial scene allows a physically feasible but opposing point of view, people respond as if their own bodies were in that place. Imagination can thus overcome perception.

Identification of emergent leaders in a meeting scenario using multiple kernel learning

In this paper, an effective framework for detection of emergent leaders in small group is presented. In this scope, the combination of different types of nonverbal visual features; the visual focus of attention, head activity and body activity based features are utilized. Using them together ensued significant results. For the first time, multiple kernel learning (MKL) was applied for the identification of the most and the least emergent leaders. Taking the advantage of MKLs capability to use different kernels which corresponds to different feature subsets having different notions of similarity, significantly improved results compared to the state of the art methods were obtained. Additionally, high correlations between the majority of the features and the social psychology questionnaires which are designed to estimate the leadership or dominance were demonstrated.

Multi-task learning of social psychology assessments and nonverbal features for automatic leadership identification

In social psychology, the leadership investigation is performed using questionnaires which are either i) self-administered or ii) applied to group participants to evaluate other members or iii) filled by external observers. While each of these sources is informative, using them individually might not be as effective as using them jointly. This paper is the first attempt which addresses the automatic identification of leaders in small-group meetings, by learning effective models using nonverbal audio-visual features and the results of social psychology questionnaires that reflect assessments regarding leadership. Learning is based on Multi-Task Learning which is performed without using ground-truth data (GT), but using the results of questionnaires (having substantial agreement with GT), administered to external observers and the participants of the meetings, as tasks. The results show that joint learning results in better performance as compared to single task learning and other baselines.

Staring Reality in the Face: A Comparison of Social Attention across Laboratory and Real World Measures Suggests Little Common Ground

The ability to attend to someone else’s gaze is thought to represent one of the essential building blocks of the human sociocognitive system. This behavior, termed social attention, has traditionally been assessed using laboratory procedures in which participants’ response time and/or accuracy performance indexes attentional function. Recently, a parallel body of emerging research has started to examine social attention during real life social interactions using naturalistic and observational methodologies. The main goal of the present work was to begin connecting these two lines of inquiry. To do so, here we operationalized, indexed, and measured the engagement and shifting components of social attention using covert and overt measures. These measures were obtained during an unconstrained real-world social interaction and during a typical laboratory social cuing task. Our results indicated reliable and overall similar indices of social attention engagement and shifting within each task. However, these measures did not relate across the two tasks. We discuss these results as potentially reflecting the differences in social attention mechanisms, the specificity of the cuing task’s measurement, as well as possible general dissimilarities with respect to context, task goals, and/or social presence. La capacité à porter attention au regard d’une autre personne représenterait l’un des fondements essentiels du système sociocognitif humain. Ce comportement, que l’on désigne attention sociale, a toujours été évalué au moyen de procédures de laboratoire selon lesquelles le temps de réponse et/ou le rendement en termes d’exactitude des participants étaient un indicateur de la fonction attentionnelle. Récemment, un corps parallèle de recherches émergentes a commencé à examiner l’attention sociale au cours d’interactions sociales réelles en utilisant des méthodologies naturalistes et observationnelles. Le but principal des présents travaux était de commencer à connecter ces deux lignes d’enquête. Pour ce faire, dans le cas présent, nous avons opérationnalisé, indexé et mesuré l’engagement et les composants de changement de l’attention sociale au moyen de mesures ouvertes et couvertes. Ces mesures ont été obtenues au cours d’une interaction sociale réelle et exempte de contraintes ainsi qu’au cours d’une tâche typique de repérage visuel social en laboratoire. Nos résultats révélaient des indices fiables et similaires dans l’ensemble au niveau de l’engagement à l’attention sociale et des changements à même chacune des tâches. Ces mesures n’étaient toutefois pas liées sur l’ensemble des deux tâches. Nous discutons de ces résultats du fait qu’ils traduiraient potentiellement les différences en termes de mécanismes d’attention sociale, la spécificité de la mesure de la tâche de repérage, ainsi que des dissemblances générales possibles au niveau du contexte, des objectifs de tâches et/ ou de la présence sociale.

Gaze following in multiagent contexts: Evidence for a quorum-like principle

Research shows that humans spontaneously follow another individual’s gaze. However, little remains known on how they respond when multiple gaze cues diverge across members of a social group. To address this question, we presented participants with displays depicting three (Experiment 1) or five (Experiment 2) agents showing diverging social cues. In a three-person group, one individual looking at the target (33% of the group) was sufficient to elicit gaze-facilitated target responses. With a five-person group, however, three individuals looking at the target (60% of the group) were necessary to produce the same effect. Gaze following in small groups therefore appears to be based on a quorum-like principle, whereby the critical level of social information needed for gaze following is determined by a proportion of consistent social cues scaled as a function of group size. As group size grows, greater agreement is needed to evoke joint attention.

How attention gates social interactions

Social interactions are at the core of social life. However, humans selectively choose their exchange partners and do not engage in all available opportunities for social encounters. In this review, we argue that attentional systems play an important role in guiding the selection of social interactions. Supported by both classic and emerging literature, we identify and characterize the three core processes—perception, interpretation, and evaluation—that interact with attentional systems to modulate selective responses to social environments. Perceptual processes facilitate attentional prioritization of social cues. Interpretative processes link attention with understanding of cues’ social meanings and agents’ mental states. Evaluative processes determine the perceived value of the source of social information. The interplay between attention and these three routes of processing places attention in a powerful role to manage the selection of the vast amount of social information that individuals encounter on a daily basis and, in turn, gate the selection of social interactions.