Andres Fuster-Guillo

A Comparative Study of Registration Methods for RGB-D Video of Static Scenes

The use of RGB-D sensors for mapping and recognition tasks in robotics or, in general, for virtual reconstruction has increased in recent years. The key aspect of these kinds of sensors is that they provide both depth and color information using the same device. In this paper, we present a comparative analysis of the most important methods used in the literature for the registration of subsequent RGB-D video frames in static scenarios. The analysis begins by explaining the characteristics of the registration problem, dividing it into two representative applications: scene modeling and object reconstruction. Then, a detailed experimentation is carried out to determine the behavior of the different methods depending on the application. For both applications, we used standard datasets and a new one built for object reconstruction.

A predictive model for recognizing human behaviour based on trajectory representation

The automatic understanding of the behaviour conducted by humans in scenarios using images as input of the system is a very important and challenging problem involving different areas of computational intelligence. In this paper human activity recognition is studied from a prediction point of view. We propose a model that, in addition to the capabilities of it to predict behaviour from new inputs, it is able to detect behaviour using a portion of the input. Specifically, we propose a prediction activity method based on the Activity Description Vector (ADV) to early detect the behaviour performed by a person in a scene. ADV is used to extract features that are normalized to be the cue of behaviour classifiers. We use complete sequences for training and partial sequences to evaluate the prediction capabilities having a specific observation time of the scene. CAVIAR dataset and different classic classifiers have been used for experimentation in order to evaluate the proposal obtaining great accuracy on the early recognition.

Human behaviour recognition based on trajectory analysis using neural networks

Automated human behaviour analysis has been, and still remains, a challenging problem. It has been dealt from different points of views: from primitive actions to human interaction recognition. This paper is focused on trajectory analysis which allows a simple high level understanding of complex human behaviour. It is proposed a novel representation method of trajectory data, called Activity Description Vector (ADV) based on the number of occurrences of a person is in a specific point of the scenario and the local movements that perform in it. The ADV is calculated for each cell of the scenario in which it is spatially sampled obtaining a cue for different clustering methods. The ADV representation has been tested as the input of several classic classifiers and compared to other approaches using CAVIAR dataset sequences obtaining great accuracy in the recognition of the behaviour of people in a Shopping Centre.

µ-MAR

Many applications including object reconstruction, robot guidance, and scene mapping require the registration of multiple views from a scene to generate a complete geometric and appearance model of it. In real situations, transformations between views are unknown an it is necessary to apply expert inference to estimate them. In the last few years, the emergence of low-cost depth-sensing cameras has strengthened the research on this topic, motivating a plethora of new applications. Although they have enough resolution and accuracy for many applications, some situations may not be solved with general state-of-the-art registration methods due to the Signal-to-Noise ratio (SNR) and the resolution of the data provided. The problem of working with low SNR data, in general terms, may appear in any 3D system, then it is necessary to propose novel solutions in this aspect. In this paper, we propose a method, μ-MAR, able to both coarse and fine register sets of 3D points provided by low-cost depth-sensing cameras, despite it is not restricted to these sensors, into a ∗Corresponding author ∗∗Principal corresponding author Email addresses: msaval@dtic.ua.es (Marcelo Saval-Calvo), jazorin@dtic.ua.es (Jorge Azoŕın-López), fuster@dtic.ua.es (Andrés Fuster-Guilló), hmora@dtic.ua.es (Higinio Mora-Mora) Preprint submitted to Elsevier ar X iv :1 70 8. 01 40 5v 1 [ cs .C V ] 4 A ug 2 01 7 common coordinate system. The method is able to overcome the noisy data problem by means of using a model-based solution of multiplane registration. Specifically, it iteratively registers 3D markers composed by multiple planes extracted from points of multiple views of the scene. As the markers and the object of interest are static in the scenario, the transformations obtained for the markers are applied to the object in order to reconstruct it. Experiments have been performed using synthetic and real data. The synthetic data allows a qualitative and quantitative evaluation by means of visual inspection and Hausdorff distance respectively. The real data experiments show the performance of the proposal using data acquired by a Primesense Carmine RGB-D sensor. The method has been compared to several state-of-the-art methods. The results show the good performance of the μ-MAR to register objects with high accuracy in presence of noisy data outperforming the existing methods.

A Novel Prediction Method for Early Recognition of Global Human Behaviour in Image Sequences

Human behaviour recognition has been, and still remains, a challenging problem that involves different areas of computational intelligence. The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts. In this paper, the problem is studied from a prediction point of view. We propose a novel method able to early detect behaviour using a small portion of the input, in addition to the capabilities of it to predict behaviour from new inputs. Specifically, we propose a predictive method based on a simple representation of trajectories of a person in the scene which allows a high level understanding of the global human behaviour. The representation of the trajectory is used as a descriptor of the activity of the individual. The descriptors are used as a cue of a classification stage for pattern recognition purposes. Classifiers are trained using the trajectory representation of the complete sequence. However, partial sequences are processed to evaluate the early prediction capabilities having a specific observation time of the scene. The experiments have been carried out using the three different dataset of the CAVIAR database taken into account the behaviour of an individual. Additionally, different classic classifiers have been used for experimentation in order to evaluate the robustness of the proposal. Results confirm the high accuracy of the proposal on the early recognition of people behaviours.

Three-dimensional planar model estimation using multi-constraint knowledge based on k-means and RANSAC

Graphical abstractDisplay Omitted HighlightsA novel method for planar model reconstruction from three dimensional point cloud.It allows the accurate reconstruction of planar objects using prior knowledge.It evaluates the clusters of each face and estimates the planes by using constraints.k-means estimates the clusters and a tree search refine them by using constraints.It extends RANSAC introducing a step to evaluate if the inliers comply constraints. Plane model extraction from three-dimensional point clouds is a necessary step in many different applications such as planar object reconstruction, indoor mapping and indoor localization. Different RANdom SAmple Consensus (RANSAC)-based methods have been proposed for this purpose in recent years. In this study, we propose a novel method-based on RANSAC called Multiplane Model Estimation, which can estimate multiple plane models simultaneously from a noisy point cloud using the knowledge extracted from a scene (or an object) in order to reconstruct it accurately. This method comprises two steps: first, it clusters the data into planar faces that preserve some constraints defined by knowledge related to the object (e.g., the angles between faces); and second, the models of the planes are estimated based on these data using a novel multi-constraint RANSAC. We performed experiments in the clustering and RANSAC stages, which showed that the proposed method performed better than state-of-the-art methods.

A Quantitative Comparison of Calibration Methods for RGB-D Sensors Using Different Technologies

RGB-D (Red Green Blue and Depth) sensors are devices that can provide color and depth information from a scene at the same time. Recently, they have been widely used in many solutions due to their commercial growth from the entertainment market to many diverse areas (e.g., robotics, CAD, etc.). In the research community, these devices have had good uptake due to their acceptable level of accuracy for many applications and their low cost, but in some cases, they work at the limit of their sensitivity, near to the minimum feature size that can be perceived. For this reason, calibration processes are critical in order to increase their accuracy and enable them to meet the requirements of such kinds of applications. To the best of our knowledge, there is not a comparative study of calibration algorithms evaluating its results in multiple RGB-D sensors. Specifically, in this paper, a comparison of the three most used calibration methods have been applied to three different RGB-D sensors based on structured light and time-of-flight. The comparison of methods has been carried out by a set of experiments to evaluate the accuracy of depth measurements. Additionally, an object reconstruction application has been used as example of an application for which the sensor works at the limit of its sensitivity. The obtained results of reconstruction have been evaluated through visual inspection and quantitative measurements.

Evaluation of sampling method effects in 3D non-rigid registration

Since the beginning of 3D computer vision problems, the use of techniques to reduce the data to make it treatable preserving the important aspects of the scene has been necessary. Currently, with the new low-cost RGB-D sensors, which provide a stream of color and 3D data of approximately 30 frames per second, this is getting more relevance. Many applications make use of these sensors and need a preprocessing to downsample the data in order to either reduce the processing time or improve the data (e.g., reducing noise or enhancing the important features). In this paper, we present a comparison of different downsampling techniques which are based on different principles. Concretely, five different downsampling methods are included: a bilinear-based method, a normal-based, a color-based, a combination of the normal and color-based samplings, and a growing neural gas (GNG)-based approach. For the comparison, two different models have been used acquired with the Blensor software. Moreover, to evaluate the effect of the downsampling in a real application, a 3D non-rigid registration is performed with the data sampled. From the experimentation we can conclude that depending on the purpose of the application some kernels of the sampling methods can improve drastically the results. Bilinear- and GNG-based methods provide homogeneous point clouds, but color-based and normal-based provide datasets with higher density of points in areas with specific features. In the non-rigid application, if a color-based sampled point cloud is used, it is possible to properly register two datasets for cases where intensity data are relevant in the model and outperform the results if only a homogeneous sampling is used.

Self-Organizing Activity Description Map to represent and classify human behaviour

The automated understanding of people activities from video sequences is an open research topic in which the computer vision and pattern recognition areas have made big efforts in recent years. This paper proposes the Self Organizing Activity Description Map (SOADM). It is a novel neural network based on the self-organizing paradigm to classify high level of semantic understanding from video sequences. The neural network is able to deal with the big gap between human trajectories in a scene and the global behaviour associated to them. Specifically, using simple representations of people trajectories as input, the SOADM is able to both represent and classify human behaviours. Additionally, the map is able to preserve the topological information about the scene. Experiments have been carried out using the Shopping Centre dataset of the CAVIAR database taken into account the global behaviour of an individual. Results confirm the high accuracy of the proposal outperforming previous methods.

Group activity description and recognition based on trajectory analysis and neural networks.

The recognition of group activities using computer vision and pattern recognition methods has been, and still remains, a challenging problem. Most of the research on human behaviour has been focused on recognizing individual issues from actions to behaviours. However, the analysis and recognition of group activities, the relationships of different groups in the scene and the interaction of the individuals in the group is still considered an open problem. This paper proposes a novel representation method to analyse and recognise group activities, called Group Activity Descriptor Vector (GADV). It is calculated from the trajectory described by the group and by the individuals who form it. Specifically, the GADV describes three different components: the trajectory followed by the group, the coherence of the individual trajectories in the group and, finally, the movement relationships among different groups in the scene. The trajectory analysis allows a simple high level understanding of complex groups activities. The GADV representation has been evaluated with different self-organizing neural networks using Behave and Caviar dataset sequences obtaining great accuracy in the recognition of the group activities, outperforming the state of the art methods.