Miguel García-Silvente

People detection and tracking using stereo vision and color

People detection and tracking are important capabilities for applications that desire to achieve a natural human-machine interaction. Although the topic has been extensively explored using a single camera, the availability and low price of new commercial stereo cameras makes them an attractive sensor to develop more sophisticated applications that take advantage of depth information. This work presents a system able to visually detect and track multiple people using a stereo camera placed at an under-head position. This camera position is especially appropriated for human-machine applications that require interacting with people or to analyze human facial gestures. The system models the background as height map that is employed to easily extract foreground objects among which people are found using a face detector. Once a person has been spotted, the system is capable of tracking him while is still looking for more people. Our system tracks people combining color and position information (using the Kalman filter). Tracking based exclusively on position information is unreliable when people establish close interactions. Thus, we also include color information about the people clothes in order to increase the tracking robustness. The system has been extensively tested and the results show that the use of color greatly reduces the errors of the tracking system. Besides, the people detection technique employed, based on combining plan-view map information and a face detector, has proved in our experimentation to avoid false detections in the tests performed. Finally, the low computing time required for the detection and tracking process makes it suitable to be employed in real time applications.

Boundary simplification using a multiscale dominant-point detection algorithm

Abstract In this paper we intend to characterize boundaries using the Scale-space theory. The aim we try to achieve is the description of a boundary in relation to a subset of points—dominant points—that are obtained from a new multiscale representation of the boundary. Dominant points are characterized by a high curvature value (in the original or smoothed boundary). As a result, the boundary is represented using those points as well as an appropriate interpolation method (the linear one in the simplest case) among them. As the basic tool of our work we will introduce a new multiscale dominant point detection algorithm that detects the points at their natural scales through a reliability condition with respect to the original curve. Because we want to apply the algorithms on complex enough boundaries, we use cartographic boundaries (in which several structures can be obtained at different scales) to evaluate the results.

Adaptive multi-modal stereo people tracking without background modelling

Detecting and tracking persons in the sequences of monocular images are the important and difficult problems in computer vision and have been well studied in these two decades. Recently, the methods based on stereo vision have attracted great attentions since 3D information can be exploited. This paper presents an approach for multiple-people detection and tracking using stereo vision. Tracking is carried out using a multiple particle filtering approach that combines depth, colour and gradient information. We modify the degree of confidence assigned to depth information, according to the amount of it found in the disparity map, using a novel confidence measure. The greater the amount of disparity information found, the higher the degree of confidence assigned to depth information in the final particles weights is. In the worst case (total absence of disparity), the proposed algorithm makes use of the information available (colour and gradient) to track, thus performing as a pure colour-based tracking algorithm. People are detected combining an adaboost classifier with stereo information. In order to test the validity of our proposal, it is evaluated in several sequences of colour and disparity images where people interact in complex situations: walk at different distances, shake hands, cross their paths, jump, run, embrace each other and even swap their positions quickly trying to confuse the system. The experimental results show that the proposal is able to deal with occlusions and to effectively determine both the 3D position of the people being tracked and their 2D head locations in the camera image, and everything is realized in real time. Besides, as the proposed method does not require the use of a background model, it can be considered particularly appropriate for applications that must run on mobile devices.

Detection of doors using a genetic visual fuzzy system for mobile robots

Doors are common objects in indoor environments and their detection can be used in robotic tasks such as map-building, navigation and positioning. This work presents a new approach to door-detection in indoor environments using computer vision. Doors are found in gray-level images by detecting the borders of their architraves. A variation of the Hough Transform is used in order to extract the segments in the image after applying the Canny edge detector. Features like length, direction, or distance between segments are used by a fuzzy system to analyze whether the relationship between them reveals the existence of doors. The system has been designed to detect rectangular doors typical of many indoor environments by the use of expert knowledge. Besides, a tuning mechanism based on a genetic algorithm is proposed to improve the performance of the system according to the particularities of the environment in which it is going to be employed. A large database of images containing doors of our building, seen from different angles and distances, has been created to test the performance of the system before and after the tuning process. The system has shown the ability to detect rectangular doors under heavy perspective deformations and it is fast enough to be used for real-time applications in a mobile robot.

A multiple object tracking approach that combines colour and depth information using a confidence measure

Multiple object tracking is a difficult task, especifically when there is not an explicit model of the object being tracked or when it is not possible to estimate the background of the scene. This paper proposes a novel approach for multiple target tracking. It works without background information and uses an original method that merges colour and depth information. The fusion of both pieces of information is created taking into account a confidence measure about the depth information. The method proposed employs a multiple particle filter approach in which particle weights are modified by an interaction factor in order to avoid the coalescence problem. In addition, the method performs as a pure colour-based technique when no disparity information is available, and takes advantage of depth information to enhance tracking whenever it is possible. Our technique is compared with two pure colour-based tracking approaches (the particle filtering method proposed by Nummiaro et al. [Nummiaro, K., Koller-Meier, E., Van Gool, L., 2003. An adaptive color-based particle filter. Image and Vision Computing, 21, 99-110] and the Kalman/mean-shift tracker [Comaniciu, D., Ramesh, V. 2000. Mean shift and optimal prediction for efficient object tracking. In: IEEE International Conference on Image Processing (ICIP00), vol. 3, pp. 70-73]) and a pure stereo-based approach derived from our problem formulation. The performance of the four algorithms is tested using several colour-with-depth sequences of images showing different coloured targets in complex situations. The results show that our proposal is able to track the targets in case of complex backgrounds and to properly determine the size of their projections in the camera image (while the other methods fail). Besides, the proposed method is fast enough for real-time applications and the use of 3D information helps to track several targets simultaneously without confusing their identities.

Automatic Tuning of a Fuzzy Visual System Using Evolutionary Algorithms: Single-Objective Versus Multiobjective Approaches

One of the main advantages of fuzzy systems is their ability to design comprehensible models of real-world systems, thanks to the use of a fuzzy rule structure easily interpretable by human beings. This is especially useful for the design of fuzzy logic controllers, where the knowledge base can be extracted from expert knowledge. Even more, the availability of a readable structure allows the human expert to customize the fuzzy controller to different environments by manually tuning its components. Nevertheless, this tuning task is usually a time-consuming procedure when done manually, especially when several measures are considered to evaluate the controller performance, and thus the interest in the design of automatic tuning procedures for fuzzy systems has increased along the last few years. In this paper, we tackle the tuning of the fuzzy membership functions of a fuzzy visual system for autonomous robots. This fuzzy visual system is based on a hierarchical structure of three different fuzzy classifiers, whose combined action allows the robot to detect the presence of doors in the images captured by its camera. Although the global knowledge represented in the fuzzy system knowledge base makes it perform properly in the door detection task, its adaptation to the specific conditions of the environment where the robot is operating can significantly improve the classification accuracy. However, the tuning procedure is complex as two different performance indexes are involved in the optimization process (true positive and false positive detections), thus becoming a multiobjective problem. Hence, in order to automatically put the fuzzy system tuning into effect, different single and multiobjective evolutionary algorithms are considered to optimize the two criteria, and their behavior in problem solving is compared.

People Detection and Tracking Through Stereo Vision for Human-Robot Interaction

In this document we present an agent for people detection and tracking through stereo vision. The agent makes use of the active vision to perform the people tracking with a robotic head on which the vision system is installed. Initially, a map of the surrounding environment is created including its motionless characteristics. This map will later on be used to detect objects in motion, and to search people among them by using a face detector. Once a person has been spotted, the agent is capable of tracking them through the robotic head that allows the stereo system to rotate. In order to achieve a robust tracking we have used the Kalman filter. The agent focuses on the person at all times by framing their head and arms on the image. This task could be used by other agents that might need to analyze gestures and expressions of potential application users in order to facilitate the human-robot interaction.

A keypoints-based feature extraction method for iris recognition under variable image quality conditions

Iris recognition is a very reliable biometric modality for human identification. The immutable and unique characteristics of the iris are the foundations for that claim. Currently, research interest in this field points to challenges regarding less-constrained iris recognition systems. In response, we propose a robust keypoints-based feature extraction method for iris recognition under variable image quality conditions. To this end, three detectors have been used to identify distinctive keypoints: Harris-Laplace, Hessian-Laplace, and Fast-Hessian. Once the three sources of keypoints are obtained, they are described in terms of SIFT features. The proposed method combines the three information sources of SIFT features at matching score level. The combination of these sources reinforces the discriminative power of the proposal for recognition on highly or less textured iris images. The fusion is carried out using a proposed weighted sum rule relies on the ranking of three performance measures. The proposed fusion rule computes weights, which represent the reliability degree to which each individual source must contribute in order to determine the more discriminative matching scores. Our experiments rely on iris standard databases which as a whole constitute a challenging and perfect example of variable image quality conditions. According to the results, our proposal is very competitive and outperforms the state-of-the-art algorithms on the topic. In addition, it is demonstrated that the proposed keypoints-based feature extraction method is feasible and that it could be used even in real-time applications if the database is previously processed.

A new fuzzy based algorithm for solving stereo vagueness in detecting and tracking people

This paper describes a system capable of detecting and tracking various people using a new approach based on colour, stereo vision and fuzzy logic. Initially, in the people detection phase, two fuzzy systems are used to filter out false positives of a face detector. Then, in the tracking phase, a new fuzzy logic based particle filter (FLPF) is proposed to fuse stereo and colour information assigning different confidence levels to each of these information sources. Information regarding depth and occlusion is used to create these confidence levels. This way, the system is able to keep track of people, in the reference camera image, even when either stereo information or colour information is confusing or not reliable. To carry out the tracking, the new FLPF is used, so that several particles are generated while several fuzzy systems compute the possibility that some of the generated particles correspond to the new position of people. Our technique outperforms two well known tracking approaches, one based on the method from Nummiaro et al. [1] and other based on the Kalman/meanshift tracker method in Comaniciu and Ramesh [2]. All these approaches were tested using several colour-with-distance sequences simulating real life scenarios. The results show that our system is able to keep track of people in most of the situations where other trackers fail, as well as to determine the size of their projections in the camera image. In addition, the method is fast enough for real time applications.

A multi-agent system architecture for mobile robot navigation based on fuzzy and visual behaviour

A multi-agent system based on behaviour for controlling the navigation task of a mobile robot in office-like environments is presented. The set of agents is structured into a three-layer hybrid architecture. A high level of abstraction plan is created using a topological map of the environment in the Deliberative layer. It is composed by the sequence of rooms and corridors to traverse and doors to cross in order to reach a desired room. The Execution and Monitoring layer translates the plan into a sequence of available skills in order to achieve the desired goal and monitors the execution of the plan. In the Control layer there is a set of agents that implements fuzzy and visual behaviours that run concurrently to guide the robot. Fuzzy behavior manages the vagueness and uncertainty of the range sensor information allowing to navigate safely in the environment. Visual behavior locates a required door to cross and fixate it, indicating the appropriate direction to reach it. Artificial landmarks are placed beside the doors to show its position. The system has been implemented in a Nomad 200 mobile robot and has been validated in numerous experiments in a real office-like environment.