Tomas Uhlin

Integrating Primary Ocular Processes

Abstract The study of active vision using binocular head-eye systems requires answers to some fundamental questions in control of attention. This paper presents a cooperative solution to resolve the ambiguities generated by the processes engaged in fixation. We suggest an approach based on integration of these processes, resulting in cooperatively extracted unique solutions. The discussion begins by looking at biological vision. Based on this discussion, a model of integration for machine vision is suggested. The implementation of the model on the KTH-head — a head-eye system simulating the essential degrees of freedom in mammalians — is explained, and in this context the primary processes in the head-eye system are briefly described. The major stress is put on the idea that the rivalry processes in vision in general, and the heads behavioural processes in particular, results in a reliable outcome. As an experiment, the ambiguities raised by fixation at repetitive patterns is tested; the cooperative approach proves to handle the problem correctly and find a unique solution for the fixation point dynamically and in real-time.

Towards an active visual observer

We present a binocular active vision system that can attend to and fixate a moving target. Our system has an open and expandable design and it forms the first steps of a long term effort towards developing an active observer using vision to interact with the environment, in particular capable of figure-ground segmentation. We also present partial real-time implementations of this system and show their performance in real-world situations together with motor control. In pursuit we particularly focus on occlusions of other targets, both stationary and moving, and integrate three cues, ego-motion, target motion and target disparity, to obtain an overall robust behavior. An active vision system must be open, expandable, and operate with whatever data are available momentarily. It must also be equipped with means and methods to direct and change its attention. This system is therefore equipped with motion detection for changing attention and pursuit for maintaining attention, both of which run concurrently.<<ETX>>

Active fixation for scene exploration

It is well-known that active selection of fixation points in humans is highly context and task dependent. It is therefore likely that successful computational processes for fixation in active vision should be so too. We are considering active fixation in the context of recognition of man-made objects characterized by their shapes. In this situation the qualitative shape and type of observed junctions play an important role. The fixations are driven by a grouping strategy, which forms sets of connected junctions separated from the surrounding at depth discontinuities. We have furthermore developed a methodology for rapid active detection and classification of junctions by selection of fixation points. The approach is based on direct computations from image data and allows integration of stereo and accommodation cues with luminance information. This work form a part of an effort to perform active recognition of generic objects, in the spirit of Malik and Biederman, but on real imagery rather than on line-drawings.

Closing the loop: detection and pursuit of a moving object by a moving observer

Abstract We present an integrated system which is able to pursue a moving object and maintain the object centered in the image by controlling a robot-head. The system contains three parts: independent motion detection, tracking, and control of a robot-head. This paper focuses on the detection mechanism, and briefly discusses the tracking and control issues. The system runs continuously in time and updates the object localization at a frame-rate of 25 Hz. The moving object can be tracked, although the observer performs an unknown independent motion, involving both translation and rotation. We focus on a simple motion detection algorithm, since computational cost is of major importance for real-time systems with feedback. The algorithm is noniterative and computationally inexpensive. The running time complexity is O ( n ) for an image containing n pixels. The image-processing takes place on a MaxVideo 200 pipeline processor, and the head control algorithm is running on a Transputer T800 network. Some offline experimental results are presented, where comparisons are made between affine and translational image motion models.

Dynamic fixation and active perception

Fixation is the link between the physical environment and the visual observer, both of which can be dynamic. That is, dynamic fixation serves the task of preserving a reference point in the world, despite relative motion. In this respect, fixation is dynamical in two senses: in response to voluntary changes of fixation point or attentive cues-gaze shiftings, and in response to the desire to compensate for the retinal slip-gaze holding.The work presented here, addresses the vergence movement and preservation of binocular fixation during smooth pursuit. This movement is a crucial component of fixation. The two vergence processes, disparity vergence and accommodative vergence, are described; a novel algorithm for robust disparity vergence and an active approach for blur detection and depth from defocus are presented. The main characteristics of the disparity vergence technique are the simplicity of the algorithm, the influence of both left and right images in the course of fixation and the agreement with the fixation model of primates. The major characteristic of the suggested algorithm for blur detection is its active approach which makes it suitable for achieving qualitative and reasonable depth estimations without unrealistic assumptions about the structures in the images.The paper also covers the integration of the two processes disparity vergence and accommodation vergence which are in turn accomplished by an integration of the disparity and blur stimuli. This integration is accounted for in both static and dynamic experiments.

Closing the Loop: Pursuing a Moving Object by a Moving Observer

We present an integrated system, able to pursue a moving object by controlling a robot-head to maintain the moving object centered in the image. This system runs continuously in time and updates the object localization at a frame-rate of 25 Hz. The moving object can be tracked although the observer performs an unknown independent motion, involving both translation and rotation. We focus on a simple algorithm, since computational cost is of major importance for real-time systems with feedback. The algorithm is noniterative and computationally cheap. The running time complexity is \(\mathcal{O}(n)\) for an image containing n pixels.

Active Vision and Seeing Robots

In this paper we argue that the study of seeing systems or robots should be performed from a systems perspective. This implies that the importance of a computational mechanism should be established in view of the tasks that the system can perform using it.

Some problems in active vision

We stress a systems approach for research in active vision. We also argue that design and analysis of seeing agents should be accompanied by experiments, requiring implementations, i.e. a constructive approach. In particular, we discuss two issues that we have worked with: use and integration of multiple cues and attention.

Developing an Active Observer

Seeing robots are usually aimed at functioning in real environments. Hence the figure-ground problem is essential. We argue that for a seeing robot, capable of actively fixating and holding gaze on objects in three dimensions, this problem is manageable, in particular if multiple cues can be used. An important point here is that a seeing robot should be able to utilize what the environment offers, rather than relying on a predetermined set of features.

Fixation by active accommodation

The field of computer vision has long been interested in disparity as the cue for the correspondence between stereo images. The other cue to correspondence, blur, and the fact that vergence is a combination of the two processes, accommodative vergence and disparity vergence, have not been equally appreciated. Following the methodology of active vision that allows the observer to control all his visual parameters, it is quite natural to take advantage of the powerful combination of these two processes. In this article, we try to elucidate such an integration and briefly analyze the cooperation and competition between accommodative vergence and disparity vergence on one hand and disparity and blur stimuli on the other hand. The human fixation mechanism is used as a guide-line and some virtues of this mechanism are used to implement a model for vergence in isolation. Finally, some experimental results are reported.