Marco San Biagio

Heterogeneous Auto-similarities of Characteristics (HASC): Exploiting Relational Information for Classification

Capturing the essential characteristics of visual objects by considering how their features are inter-related is a recent philosophy of object classification. In this paper, we embed this principle in a novel image descriptor, dubbed Heterogeneous Auto-Similarities of Characteristics (HASC). HASC is applied to heterogeneous dense features maps, encoding linear relations by co variances and nonlinear associations through information-theoretic measures such as mutual information and entropy. In this way, highly complex structural information can be expressed in a compact, scale invariant and robust manner. The effectiveness of HASC is tested on many diverse detection and classification scenarios, considering objects, textures and pedestrians, on widely known benchmarks (Caltech-101, Brodatz, Daimler Multi-Cue). In all the cases, the results obtained with standard classifiers demonstrate the superiority of HASC with respect to the most adopted local feature descriptors nowadays, such as SIFT, HOG, LBP and feature co variances. In addition, HASC sets the state-of-the-art on the Brodatz texture dataset and the Daimler Multi-Cue pedestrian dataset, without exploiting ad-hoc sophisticated classifiers.

Kernelized covariance for action recognition

In this paper we aim at increasing the descriptive power of the covariance matrix, limited in capturing linear mutual dependencies between variables only. We present a rigorous and principled mathematical pipeline to recover the kernel trick for computing the covariance matrix, enhancing it to model more complex, non-linear relationships conveyed by the raw data. To this end, we propose Kernelized-COV, which generalizes the original covariance representation without compromising the efficiency of the computation. In the experiments, we validate the proposed framework against many previous approaches in the literature, scoring on par or superior with respect to the state of the art on benchmark datasets for 3D action recognition.

Kernel-based classification for brain connectivity graphs on the Riemannian manifold of positive definite matrices

An important task in connectomics studies is the classification of connectivity graphs coming from healthy and pathological subjects. In this paper, we propose a mathematical framework based on Riemannian geometry and kernel methods that can be applied to connectivity matrices for the classification task. We tested our approach using different real datasets of functional and structural connectivity, evaluating different metrics to describe the similarity between graphs. The empirical results obtained clearly show the superior performance of our approach compared with baseline methods, demonstrating the advantages of our manifold framework and its potential for other applications.

Weighted bag of visual words for object recognition

Bag of Visual words (BoV) is one of the most successful strategy for object recognition, used to represent an image as a vector of counts using a learned vocabulary. This strategy assumes that the representation is built using patches that are either densely extracted or sampled from the images using feature detectors. However, the dense strategy captures also the noisy background information, whereas the feature detection strategy can lose important parts of the objects. In this paper we propose a solution in-between these two strategies, by densely extracting patches from the image, and weighting them accordingly to their salience. Intuitively, highly salient patches have an important role in describing an object, while those with low saliency are still taken with low emphasis, instead of discarding them. We embed this idea in the word encoding mechanism adopted in the BoV approaches. The technique is successfully applied to vector quantization and Fisher vector, on Caltech-101 and Caltech-256.

Approximate Log-Hilbert-Schmidt Distances between Covariance Operators for Image Classification

This paper presents a novel framework for visual object recognition using infinite-dimensional covariance operators of input features, in the paradigm of kernel methods on infinite-dimensional Riemannian manifolds. Our formulation provides a rich representation of image features by exploiting their non-linear correlations, using the power of kernel methods and Riemannian geometry. Theoretically, we provide an approximate formulation for the Log-Hilbert-Schmidt distance between covariance operators that is efficient to compute and scalable to large datasets. Empirically, we apply our framework to the task of image classification on eight different, challenging datasets. In almost all cases, the results obtained outperform other state of the art methods, demonstrating the competitiveness and potential of our framework.

ENCODING STRUCTURAL SIMILARITY BY CROSS-COVARIANCE TENSORS FOR IMAGE CLASSIFICATION

In computer vision, an object can be modeled in two main ways: by explicitly measuring its characteristics in terms of feature vectors, and by capturing the relations which link an object with some exemplars, that is, in terms of similarities. In this paper, we propose a new similarity-based descriptor, dubbed structural similarity cross-covariance tensor (SS-CCT), where self-similarities come into play: Here the entity to be measured and the exemplar are regions of the same object, and their similarities are encoded in terms of cross-covariance matrices. These matrices are computed from a set of low-level feature vectors extracted from pairs of regions that cover the entire image. SS-CCT shares some similarities with the widely used covariance matrix descriptor, but extends its power focusing on structural similarities across multiple parts of an image, instead of capturing local similarities in a single region. The effectiveness of SS-CCT is tested on many diverse classification scenarios, considering objects and scenes on widely known benchmarks (Caltech-101, Caltech-256, PASCAL VOC 2007 and SenseCam). In all the cases, the results obtained demonstrate the superiority of our new descriptor against diverse competitors. Furthermore, we also reported an analysis on the reduced computational burden achieved by using and efficient implementation that takes advantage from the integral image representation.

Encoding Classes of Unaligned Objects Using Structural Similarity Cross-Covariance Tensors

Encoding an object essence in terms of self-similarities between its parts is becoming a popular strategy in Computer Vision. In this paper, a new similarity-based descriptor, dubbed Structural Similarity Cross-Covariance Tensor is proposed, aimed to encode relations among different regions of an image in terms of cross-covariance matrices. The latter are calculated between low-level feature vectors extracted from pairs of regions. The new descriptor retains the advantages of the widely used covariance matrix descriptors [1], extending their expressiveness from local similarities inside a region to structural similarities across multiple regions. The new descriptor, applied on top of HOG, is tested on object and scene classification tasks with three datasets. The proposed method always outclasses baseline HOG and yields significant improvement over a recently proposed self-similarity descriptor in the two most challenging datasets.

Exploiting Feature Correlations by Brownian Statistics for People Detection and Recognition

Characterizing an image region by its feature intercorrelations is a modern trend in computer vision. In this paper, we introduce a new image descriptor that can be seen as a natural extension of a standard covariance descriptor with the advantage of capturing nonlinear and nonmonotone dependencies. Inspired from the recent advances in mathematical statistics of Brownian motion, we can express highly complex structural information in a compact and computationally efficient manner. We show that our Brownian covariance descriptor can capture richer image characteristics than the covariance descriptor. Additionally, a detailed analysis of the Brownian manifold reveals that opposite to the classical covariance descriptor, the proposed descriptor lies in a relatively flat manifold, which can be treated as a Euclidean. This brings significant boost in the efficiency of the descriptor. The effectiveness and the generality of our approach is validated on two challenging vision tasks, pedestrian classification, and person reidentification. The experiments are carried out on multiple datasets achieving promising results.

Automatic inspection of aeronautic components

Industrial processes are costly in terms of time, money and customer satisfaction. The global economic pressures have gradually led businesses to improve these processes to become more competitive. As a result, the demand of intelligent visual inspection systems aimed at ensuring the high quality in production lines is increasing. In this paper, we present a computer vision system that, using only images, is able to address two main problems: (i) model checking: automatically check whether a component meets given specifications or rules, (ii) visual inspection: defect inspection on irregular surfaces, in particular, decolourization and scratches detection. In the experimental results, we show the effectiveness of our system and the readiness of such technologies for their integration in industrial processes.

Latent subcategory models for pedestrian detection with partial occlusion handling

Pedestrian detection is one of the most important tasks in Computer Vision, especially in automotive and security applications. One of the most common problems in real scenarios is related to the detection of occluded pedestrians. In this paper, we propose a novel multi-cue pedestrian detection approach able to deal with non homogeneous object samples by learning latent subcategory models trained on both visual and depth-based features. We also propose a novel self-similarity based feature, namely SSTD, to encode the homogeneity in appearance of pedestrians characterized by similar occlusion patterns. Experiments are performed on the Daimler Pedestrian Detection Benchmark Dataset showing the robustness of our approach in actual scenarios.