Diego Tosato

Social interaction discovery by statistical analysis of F-formations.

We present a novel approach for detecting social interactions in a crowded scene by employing solely visual cues. The detection of social interactions in unconstrained scenarios is a valuable and important task, especially for surveillance purposes. Our proposal is inspired by the social signaling literature, and in particular it considers the sociological notion of F-formation. An F-formation is a set of possible configurations in space that people may assume while participating in a social interaction. Our system takes as input the positions of the people in a scene and their (head) orientations; then, employing a voting strategy based on the Hough transform, it recognizes F-formations and the individuals associated with them. Experiments on simulations and real data promote our idea.

Characterizing Humans on Riemannian Manifolds

In surveillance applications, head and body orientation of people is of primary importance for assessing many behavioral traits. Unfortunately, in this context people are often encoded by a few, noisy pixels so that their characterization is difficult. We face this issue, proposing a computational framework which is based on an expressive descriptor, the covariance of features. Covariances have been employed for pedestrian detection purposes, actually a binary classification problem on Riemannian manifolds. In this paper, we show how to extend to the multiclassification case, presenting a novel descriptor, named weighted array of covariances, especially suited for dealing with tiny image representations. The extension requires a novel differential geometry approach in which covariances are projected on a unique tangent space where standard machine learning techniques can be applied. In particular, we adopt the Campbell-Baker-Hausdorff expansion as a means to approximate on the tangent space the genuine (geodesic) distances on the manifold in a very efficient way. We test our methodology on multiple benchmark datasets, and also propose new testing sets, getting convincing results in all the cases.

Social interactions by visual focus of attention in a three-dimensional environment

In human behaviour analysis, the visual focus of attention (VFOA) of a person is a very important cue. VFOA detection is difficult, though, especially in a unconstrained and crowded environment, typical of video surveillance scenarios. In this paper, we estimate the VFOA by defining the Subjective View Frustum, which approximates the visual field of a person in a three-dimensional representation of the scene. This opens up to several intriguing behavioural investigations. In particular, we propose the Inter-Relation Pattern Matrix, which suggests possible social interactions between the people present in a scene. Theoretical justifications and experimental results substantiate the validity and the goodness of the analysis performed.

Part-based human detection on Riemannian manifolds

In this paper we propose a novel part-based framework for pedestrian detection. We model a human as a hierarchy of fixed overlapped parts, each of which described by covariances of features. Each part is modeled by a boosted classifier, learnt using Logitboost on Riemannian manifolds. All the classifiers are then linked to form a high-level classifier, through weighted summation, whose weights are estimated during the learning. The final classifier is simple, light and robust. The experimental results show that we outperform the state-of-the-art human detection performances on the INRIA person dataset.

FPGA-based pedestrian detection using array of covariance features

In this paper we propose a pedestrian detection algorithm and its implementation on a Xilinx Virtex-4 FPGA. The algorithm is a sliding window-based classifier, that exploits a recently designed descriptor, the covariance of features, for characterizing pedestrians in a robust way. In the paper we show how such descriptor, originally suited for maximizing accuracy performances without caring about timings, can be quickly computed in an elegant, parallel way on the FPGA board. A grid of overlapped covariances extracts information from the sliding window, and feeds a linear Support Vector Machine that performs the detection. Experiments are performed on the INRIA pedestrian benchmark; the performances of the FPGA-based detector are discussed in terms of required computational effort and accuracy, showing state-of-the-art detection performances under excellent timings and economic memory usage.

An FPGA-based Classification Architecture on Riemannian Manifolds

In Computer Vision and Pattern Recognition, the object detection problem is a fundamental task, but only a few systems are thought to be realized on an embedded architecture. To this end, we propose an effective, low-latency, affordable classification architecture, especially suited for embedded platforms. In particular, we have designed a novel highly-parallelizable classification framework for an FPGA-based implementation, which is suitable for generic detection problems. The underlying model consists in a weighted sum of boosted binary classifiers, learned on a set of overlapped image patches. Each patch is described by estimating the covariance matrix of a set of features, so forming a very compact and expressive descriptor. Covariances matrices live on Riemannian Manifold, whose topology is particularly simple, so that they can be approximated in the Euclidean Vector Space in a cheap and conservative way. The hardware design has been developed in a parallel fashion and with specific architectural solutions, allowing a fast response without degrading the functional performances. We finally specialize this architecture to the challenging pedestrian detection problem, defining state-of-the art results on the standard INRIA pedestrian benchmark dataset.

A Re-evaluation of Pedestrian Detection on Riemannian Manifolds

Boosting covariance data on Riemannian manifolds has proven to be a convenient strategy in a pedestrian detection context. In this paper we show that the detection performances of the state-of-the-art approach of Tuzel et al. [7] can be greatly improved, from both a computational and a qualitative point of view, by considering practical and theoretical issues, and allowing also the estimation of occlusions in a fine way. The resulting detection system reaches the best performance on the INRIA dataset, setting novel state-of-the art results.

Analyzing Groups: A Social Signaling Perspective

This chapter introduces some basic methods to deal with groups of people in surveillance settings. Recently, modeling groups has become a very active trend for video surveillance researchers. Our solution is proper of the recently forged field of social signaling, since it embeds notions of social psychology into computer vision techniques, offering a novel research perspective for the video surveillance community. In particular, we present methods to discover and track groups of people, and to infer what is the focus of attention of each person, that is, we estimate the portion of a scene that is frequently observed by people. Each method we present is evaluated in an experimental section on real scenario, that gives a clear idea of its performance and potentialities.

Fast FPGA-based architecture for pedestrian detection based on covariance matrices

Pedestrian detection is a crucial task in several video surveillance and automotive scenarios, but only a few detection systems are designed to be realized on an embedded architecture, allowing to increase the processing speed which is one of the key requirements in real applications. In this paper, we propose a novel SoC (System on Chip) architecture for fast pedestrian detection in video. Our implementation is based on a linear SVM (Support Vector Machine) classification framework, learned on a set of overlapped image patches. Each patch is described by a covariance matrix of a set of image features. Exploiting the inner parallelism of the FPGA (Field Programmable Gate Array) boards, we dramatically accelerate the covariance matrices computation that plays a crucial role in the framework. In the experiments, we show the effectiveness and the efficiency of our pedestrian detection system, reaching a detection speed of 132 fps at VGA resolution.