Rafael Muñoz-Salinas

Automatic generation and detection of highly reliable fiducial markers under occlusion

This paper presents a fiducial marker system specially appropriated for camera pose estimation in applications such as augmented reality and robot localization. Three main contributions are presented. First, we propose an algorithm for generating configurable marker dictionaries (in size and number of bits) following a criterion to maximize the inter-marker distance and the number of bit transitions. In the process, we derive the maximum theoretical inter-marker distance that dictionaries of square binary markers can have. Second, a method for automatically detecting the markers and correcting possible errors is proposed. Third, a solution to the occlusion problem in augmented reality applications is shown. To that aim, multiple markers are combined with an occlusion mask calculated by color segmentation. The experiments conducted show that our proposal obtains dictionaries with higher inter-marker distances and lower false negative rates than state-of-the-art systems, and provides an effective solution to the occlusion problem. HighlightsWe propose an algorithm for generating configurable marker dictionaries.We derive the maximum theoretical inter-marker distance.A method for automatically detecting the markers and correcting errors is proposed.A solution to the occlusion problem in augmented reality applications is shown.

People detection and tracking using stereo vision and color

People detection and tracking are important capabilities for applications that desire to achieve a natural human-machine interaction. Although the topic has been extensively explored using a single camera, the availability and low price of new commercial stereo cameras makes them an attractive sensor to develop more sophisticated applications that take advantage of depth information. This work presents a system able to visually detect and track multiple people using a stereo camera placed at an under-head position. This camera position is especially appropriated for human-machine applications that require interacting with people or to analyze human facial gestures. The system models the background as height map that is employed to easily extract foreground objects among which people are found using a face detector. Once a person has been spotted, the system is capable of tracking him while is still looking for more people. Our system tracks people combining color and position information (using the Kalman filter). Tracking based exclusively on position information is unreliable when people establish close interactions. Thus, we also include color information about the people clothes in order to increase the tracking robustness. The system has been extensively tested and the results show that the use of color greatly reduces the errors of the tracking system. Besides, the people detection technique employed, based on combining plan-view map information and a face detector, has proved in our experimentation to avoid false detections in the tests performed. Finally, the low computing time required for the detection and tracking process makes it suitable to be employed in real time applications.

Depth silhouettes for gesture recognition

This paper, evaluates the influence of depth information on the gesture recognition process. We propose depth silhouettes, a natural extension of the binary silhouette concept, as a mechanism to incorporate depth information for gesture recognition. Using depth silhouettes, we define extensions of three classic techniques employed previously for gesture recognition with monocular vision. These include: (a) silhouette compression using PCA and learning with HMM; (b) an exemplar-based gesture recognition using HMM; and (c) temporal templates that in this work are compressed using PCA and learned with SVM. The results obtained show that, independently of the technique employed, the use of depth silhouettes increases the success significantly. Additionally, we show how the best results are obtained through the combined use of PCA and HMM.

Polygonal approximation of digital planar curves through break point suppression

This paper presents a new algorithm that detects a set of dominant points on the boundary of an eight-connected shape to obtain a polygonal approximation of the shape itself. The set of dominant points is obtained from the original break points of the initial boundary, where the integral square is zero. For this goal, most of the original break points are deleted by suppressing those whose perpendicular distance to an approximating straight line is lower than a variable threshold value. The proposed algorithm iteratively deletes redundant break points until the required approximation, which relies on a decrease in the length of the contour and the highest error, is achieved. A comparative experiment with another commonly used algorithm showed that the proposed method produced efficient and effective polygonal approximations for digital planar curves with features of several sizes.

A novel method to look for the hysteresis thresholds for the Canny edge detector

In the last few years, several works have been proposed to solve the problem of determining the hysteresis thresholds in an unsupervised way. In this paper, a novel method to solve this problem is proposed. Given a set of candidates for hysteresis thresholds, the basic idea of the proposed method is to combine gradient information with information obtained when the linking process is applied to all candidates. Using the same dataset and the same evaluation methodology already proposed by other works, the results obtained by our method show a performance better than that of the previous methods. The results obtained by the proposed method have been validated only for the Canny edge detector, but there are no restrictions on applying the proposed method to any other edge detector whose strategy is based on the hysteresis mechanism.

On candidates selection for hysteresis thresholds in edge detection

Manual determination of hysteresis thresholds is time-consuming. Several methods approach the problem of unsupervised determination of edge detector parameters, but they require human intervention to establish the initial range of values in which to detect the best parameter value and the result depends on the range of values initially used. In this paper, a method is proposed to determine candidates to hysteresis thresholds in an unsupervised manner. The method provides a criterion to reduce in a significant way the number of initial values to be considered as threshold candidates. The proposed method can be applied to any feature image provided by an edge detector upon which hysteresis must be implemented.

Generation of fiducial marker dictionaries using Mixed Integer Linear Programming

Square-based fiducial markers are one of the most popular approaches for camera pose estimation due to its fast detection and robustness. In order to maximize their error correction capabilities, it is required to use an inner binary codification with a large inter-marker distance. This paper proposes two Mixed Integer Linear Programming (MILP) approaches to generate configurable square-based fiducial marker dictionaries maximizing their inter-marker distance. The first approach guarantees the optimal solution, however, it can only be applied to relatively small dictionaries and number of bits since the computing times are too long for many situations. The second approach is an alternative formulation to obtain suboptimal dictionaries within restricted time, achieving results that still surpass significantly the current state of the art methods. HighlightsThe paper proposes two methods to obtain fiducial markers based on the MILP paradigm.First model guarantees the optimality in terms of inter-marker distance.Second model generates suboptimal markers within restricted time.The markers generated allow the correction of a higher amount of erroneous bits.

Determining Hysteresis Thresholds for Edge Detection by Combining the Advantages and Disadvantages of Thresholding Methods

Hysteresis is an important technique for edge detection, but the unsupervised determination of its parameters is not an easy problem. In this paper, we propose a method for unsupervised determination of hysteresis thresholds using the advantages and disadvantages of two thresholding methods. The basic idea of our method is to look for the best hysteresis thresholds in a set of candidates. First, the method finds a subset and a overset of the unknown edge points set. Then, it determines the best edge map with the measure ¿2. Compared with a general method to determine the parameters of an edge detector, our method performs well and is less computationally complex. The basic idea of our method can be generalized to other pattern recognition problems.

A Bayesian plan-view map based approach for multiple-person detection and tracking

This work proposes a novel approach for people detection and tracking in colour-with-depth sequences using a particle filtering approach. Detection and tracking are performed in plan-view maps integrating occupancy and height information with a novel plan-view map representation for colour information. Using the three maps, we propose a multiple particle filtering algorithm for people detection and tracking. The observation model proposed integrates information from the three maps so that people with different coloured clothes are not confused even when they interact at close distances. To avoid the coalescence problem when people with similar coloured clothes approach each other, the weight of particles is modified by an interaction factor that combines colour and position information. The algorithm also avoids the coalescence problem in case of total occlusion by means of an occlusion detection and recovering mechanism. Finally, a solution is proposed to improve the exponential complexity of multiple particle filters so that the algorithm proposed has linear complexity. The approach proposed has been tested in several colour-with-depth sequences where people move and interact freely in the environment. In the sequences, people walk at different distances, cross their paths causing frequent occlusions, jump, run and have close interactions such as shaking hands or embracing each other. The experimental results show that our proposal is able to detect and keep track of every person with a low error and deals with partial and total occlusions. Besides, the detection and tracking techniques presented are appropriate for large tracking problems in real-time applications since their complexity is linear, are suitable for parallel processing and allow the integration of information provided by multiple stereo vision sensors.

Adaptive multi-modal stereo people tracking without background modelling

Detecting and tracking persons in the sequences of monocular images are the important and difficult problems in computer vision and have been well studied in these two decades. Recently, the methods based on stereo vision have attracted great attentions since 3D information can be exploited. This paper presents an approach for multiple-people detection and tracking using stereo vision. Tracking is carried out using a multiple particle filtering approach that combines depth, colour and gradient information. We modify the degree of confidence assigned to depth information, according to the amount of it found in the disparity map, using a novel confidence measure. The greater the amount of disparity information found, the higher the degree of confidence assigned to depth information in the final particles weights is. In the worst case (total absence of disparity), the proposed algorithm makes use of the information available (colour and gradient) to track, thus performing as a pure colour-based tracking algorithm. People are detected combining an adaboost classifier with stereo information. In order to test the validity of our proposal, it is evaluated in several sequences of colour and disparity images where people interact in complex situations: walk at different distances, shake hands, cross their paths, jump, run, embrace each other and even swap their positions quickly trying to confuse the system. The experimental results show that the proposal is able to deal with occlusions and to effectively determine both the 3D position of the people being tracked and their 2D head locations in the camera image, and everything is realized in real time. Besides, as the proposed method does not require the use of a background model, it can be considered particularly appropriate for applications that must run on mobile devices.