Sergio Taraglio

Sensing the third dimension in stereo vision systems: a cellular neural networks approach

Abstract In this paper the use of the cellular neural network (CNN) paradigm is investigated for the vision-based real-time guidance of robots. This paradigm is employed in recovering information on the tridimensional structure of the environment, through the resolution of the static and the lateral motion stereo vision problems. The proposed approaches exploit the spontaneous internal energy decrease of the CNN, coding the problem in terms of an optimisation task. Results of computer simulations on some test cases for the two different issues are provided. The performance of a hardware implementation of these networks for the tasks presented is outlined.

Cellular neural networks for the stereo matching problem

The applicability of the cellular neural network (CNN) paradigm to the problem of recovering information on the 3D structure of the environment is investigated. The approach proposed is the stereo matching of video images. The starting point of this work is the Zhou-Chellappa neural network implementation (1992) for the same problem. The CNN based system we present here yields the same results as the previous approach, but without the many existing drawbacks.

Cellular neural networks: a genetic algorithm for parameters optimization in artificial vision applications

An optimization method for some of the CNNs parameters, based on evolutionary strategies, is proposed. The new class of feedback template found is more effective in extracting features from the images that an autonomous vehicle acquires, than in the previous CNNs literature.

A dedicated hardware system for CNN stereo vision

The stereo vision algorithm is a very promising technique in autonomous robotics to sense the environment. A successful implementation of this algorithm on Cellular Neural Networks (CNN) has been proposed. In this paper an analogue CNN hardware system able to perform such an algorithm is presented. This new system will be installed directly onto the robot and plays the role of a parallel analogue coprocessor.

Improving a Real-Time Neural-Based Stereo Vision System

Refinements of the energy expression of a real-time neural-based stereo vision system are presented. The neural network optimizes a scalar functional, that represents an area-based stereo matching algorithm. The neural system is reviewed and its performances presented. The proposed improvements are in terms of the exploitation of the image chromatic content and of local pixel information relative to the distance from an image feature. Experimental results showing the performance improvements are presented on synthetic and on real images. The hardware implementation currently in progress will straightforwardly benefit from these improvements.

A practical use of cellular neural networks: the stereo-vision problem as an optimisation

Abstract. A variational way of deriving the relevant parameters of a cellular neural network (CNN) is introduced. The approach exploits the CNN spontaneous internal-energy decrease and is applicable when a given problem can be expressed in terms of an optimisation task. The presented approach is fully mathematical as compared with the typical heuristic search for the correct parameters in the literature on CNNs. This method is practically employed in recovering information on the three-dimensional structure of the environment, through the stereo vision problem. A CNN able to find the conjugate points in a stereogram is fully derived in the proposed framework. Results of computer simulations on several test cases are provided.

A CNN stereo vision hardware system for autonomous robot navigation

The high parallel analogue processing rate makes the cellular neural networks paradigm really useful in such a problems where real-time replies to external stimuli are required. The development of an effective system for autonomous robot navigation can find a valid support from this research. Moreover, the growth of new CNN algorithms can afford the necessary feedback to the hardware developers to improve their realisations. In this paper some measurements of a stereo-vision algorithm on a CNN hardware implementation (the 720DPCNN system) are given.

Evolutionary Approach to Epipolar Geometry Estimation

An image is a two dimensional projection of a three dimensional scene. Hence a degeneration is introduced since no information is retained on the distance of a given point in the space. In order to extract information on the three dimensional contents of a scene from a single image it is necessary to exploit some a priori knowledge either on the features of the scene, i.e. presence/absence of architectural lines, objects sizes, or on the general behaviour of shades, textures, etc. Everything becomes much simpler if more than a single image is available. Whenever more viewpoints and images are available, several geometric relations can be derived among the three dimensional real points and their projections onto the various two dimensional images. These relations can be mathematically described under the assumption of pinhole cameras and furnish constraints among the various image points. If only two images are considered, this research topic is usually referred to as epipolar geometry. Naturally there is no mathematical difference whether the considered images are taken at the same time by two different cameras (the stereoscopic vision problem) or at different times by a single moving camera (optical flow or structure from motion problem). In Robotics both these cases are of great significance. Stereoscopy yields the knowledge of objects and obstacles positions providing a useful key to obtain the safe navigation of a robot in any environment (Zanela & Taraglio, 2002). On the other hand the estimation of the ego-motion, i.e. the measure of camera motion, can be exploited to the end of computing robot odometry and thus spatial position, see e.g. (Caballero et al., 2009). In addition the visual sensing of the environment is becoming ubiquitous out of the ever decreasing costs of both cameras and processors and the cooperative coordination of more cameras can be exploited in many applicative fields such as surveillance or multimedia applications (Arghaian & Cavallaro, 2009). Epipolar geometry is then the geometry of two cameras, i.e. two images, and it is usually represented by a 3 x 3 fundamental matrix, from which it is possible to retrieve all the relevant geometrical information, namely the rigid roto-translation between camera positions. The estimation of the fundamental matrix is based on a set of corresponding features present in both the images of the same scene. Naturally the error in the process is directly linked to the accuracy in the computation of these correspondences. In the following a novel genetic approach to epipolar geometry estimation is presented. This algorithm searches an optimal or sub-optimal solution for the rigid roto-translation between two camera positions in a evolutionary framework. The fitness of the tentative solutions is measured against the full set of correspondences through a function that is able to correctly cope with outliers, i.e. the incorrectly matched points usually due to errors in feature detection and/or in matching. Finally the evolution of the 1

Phase Difference Stereo Disparity Computation on a SIMD Parallel Machine

A parallel version of the phase-based algorithm for disparity estimation in stereo image pairs for the reconstruction of the third dimension is presented. The algorithm is implemented on the Quadrics massively parallel SIMD machine. An analysis of performance as a function of image size and processors number is given. The obtained processing times are compared with two other HW architectures both sequential and parallel.

An efficient implementation of a backpropagation learning algorithm on a Quadrics parallel supercomputer

A parallel implementation of a library to build and train Multi Layer Perceptions via the Back Propagation algorithm is presented. The target machine is the SIMD massively parallel supercomputer Quadrics. Performance measures are provided on three different machines with different number of processors, for two network examples. A sample source code is given.