Fabio Caparrelli

Vision-based closed-loop control of mobile microrobots for microhandling tasks

As part of a European Union ESPRIT funded research project a flexible microrobot system has been developed which can operate under an optical microscope as well as in the chamber of a scanning electron microscope. The system is highly flexible and configurable and uses a wide range of sensors in a closed-loop control strategy. This paper presents an overview of the vision system and its architecture for vision-controlled micro-manipulation. The range of different applications, e.g. assembly of hybrid microsystems, handling of biological cells and manipulation tasks inside an SEM, imposes great demands on the vision system. Fast and reliable object recognition algorithms have been developed and implemented to provide for two modes of operation: automated and semi-automated robot control. The vision system has a modular design, comprising modules for object recognition, tracking and depth estimation. Communication between the vision modules and the control system takes place via a shared memory system embedding an object database. This database holds information about the appearance and the location of all known objects. A depth estimation method based on a modified sheet-of-light triangulation method is also described. Furthermore, the novel approach of electron beam triangulation in the SEM is described.

Computer vision methods for optical microscopes

As the fields of micro- and nano-technology mature, there will be an increased need to build tools that are able to work in these areas. Industry will require solutions for assembling and manipulating components, much as it has done in the macro range. With this need in mind, a new set of challenges requiring novel solutions have to be met. One of them is the ability to provide closed-loop feedback control for manipulators. We foresee that machine vision will play a leading role in this area. This paper introduces a technique for integrating machine vision into the field of micro-technology including two methods, one for tracking and one for depth reconstruction under an optical microscope.

Machine vision methods for autonomous micro‐robotic systems

Purpose – To develop customised machine vision methods for closed‐loop micro‐robotic control systems. The micro‐robots have applications in areas that require micro‐manipulation and micro‐assembly in the micron and sub‐micron range.Design/methodology/approach – Several novel techniques have been developed to perform calibration, object recognition and object tracking in real‐time under a customised high‐magnification camera system. These new methods combine statistical, neural and morphological approaches.Findings – An in‐depth view of the machine vision sub‐system that was designed for the European MiCRoN project (project no. IST‐2001‐33567) is provided. The issue of cooperation arises when several robots with a variety of on‐board tools are placed in the working environment. By combining multiple vision methods, the information obtained can be used effectively to guide the robots in achieving the pre‐planned tasks.Research limitations/implications – Some of these techniques were developed for micro‐visi...

An Analysis of Collective Movement Models for Robotic Swarms

A swarm is defined as a set of two or more independent homogeneous or heterogeneous agents acting in a common environment, in a coherent fashion, and which generates emergent behavior. The creation of artificial swarms or robotic swarms has attracted many researchers in the last two decades. Many studies have been undertaken using practical approaches to swarm construction such as investigating the navigation of the swarm, task allocation and elementary construction. This paper examines aggregations that emerge from three different movement models of relatively simple agents. The agents only differ in their maximum turning angle and their sensing range.

Prototyping tangibles: exploring form and interaction

In order to better explore the opportunities for tangible interaction in new areas such as the home or cultural heritage sites, we used multiple rapidly-developed prototypes that take advantage of existing technology. Physical prototypes allow us to give form to ideas and to evaluate the integration of form and function, two core components of tangible interaction. We discuss potentials and pitfalls when using off-the-shelf digital devices (by embedding a device, cracking it open and building on it, or collating board and parts) through six prototypes developed in two studies. Hacking devices to materialize our ideas proved excellent for fast prototyping. Technology imposed constraints and prompted different design solutions than initially intended offering unexpected ways to engage. On the basis of this experience we outline a process and offer guidelines for the fast prototyping of tangible interactions.

A cognitive architecture for modular and self-reconfigurable robots

The field of reconfigurable swarms of modular robots has achieved a current status of performance that allows applications in diverse fields that are characterized by human support (e.g. exploratory and rescue tasks) or even in human-less environments. The main goal of the EC project REPLICATOR [1] is the development and deployment of a heterogeneous swarm of modular robots that are able to switch autonomously from a swarm of robots, into different organism forms, to reconfigure these forms, and finally to revert to the original swarm mode [2]. To achieve these goals three different types of robot modules have been developed and an extensive suite of embodied distributed cognition methods implemented [3]. Hereby the methodological key aspects address principles of self-organization. In order to tackle our ambitious approach a Grand Challenge has been proposed of autonomous operation of 100 robots for 100 days (100 days, 100 robots). Moreover, a framework coined the SOS-cycle (SOS: Swarm-Organism-Swarm) is developed. It controls the transitions between internal phases that enable the whole system to alternate between different modes mentioned above. This paper describes the vision of the Grand Challenge and the implementation and the results of the different phases of the SOS-cycle.

Recurrent neural robot controllers: feedback mechanisms for identifying environmental motion dynamics

In this paper a series of recurrent controllers for mobile robots have been developed. The system combines the iterative learning capability of neural controllers and the optimisation ability of particle swarms. In particular, three controllers have been developed: an Exo-sensing, an Ego-sensing and a Composite controller which is the hybrid of the latter two. The task for each controller is to learn to follow a moving target and identify its trajectory using only local information. We show how the learned behaviours of each architecture rely on different sensory representations, although good results are obtained in all cases.

Vision based Environment Mapping by Network Connected Multi-robotic System

The conventional environment mapping solutions are computationally very expensive and cannot effectively be used in multi-robotic environment, where small size robots with limited memory and processing resources are used. This study provides an environment mapping solution in which a group of small size robots extract simple distance vector features from the on-board camera images. The robots share these features between them using a wireless communication network setup in infrastructure mode. For mapping the distance vector features on a global map and to show a collective map building operation, the robots needed their accurate location and heading information. The robots location and heading information is computed using two ceiling mounted cameras, which collective localises the robots. Experimental results show that the proposed method provides the required environmental map which can facilitate the robot navigation operation in the environ- ment. It was observed that, using the proposed approach, the near by object boundaries can be mapped with higher accuracy comparatively the far lying objects.

An Efficient Approach to Object Recognition for Mobile Robots

In robotics, the object recognition approaches developed so far have proved very valuable, but their high memory and processing requirements make them suitable only for robots with high processing capability or for offline processing. When it comes to small size robots, these approaches are not effective and light- weight vision processing is adopted which causes a big drop in recognition performance. In this research, a computationally expensive, but efficient appearance-based object recognition approach is considered and tested on a small robotic platform which has limited memory and processing resources. Rather than processing the high resolution images, all the times, to perform recognition, a novel idea of switching between high and low resolutions, based on the “distance to object” is adopted. It is also shown that much of the computation time can be saved by identifying the irrelevant information in the images and avoid processing them with computationally expensive approaches. This helps to bridge the gap between the computationally expensive approaches and embedded platform with limited processing resources.

Implementation of Distributed Mosaic Formation and Object Detection in Modular Robotic Systems

In reconfigurable modular robotics, when robot modules joins to form a robotic organism, they create a dis- tributed processing environment in a unified system. This research builds on the efficient use of these dis- tributed processing resources and presents the manner these resources can be utilised to implement distributed mosaic formation and object detection within the organism. The generation of mosaics provides surrounding awareness to the organism and helps it to localise itself with reference to the objects in the mosaics. Whereas, the detection of objects in the mosaic helps in identifying parts of the mosaic which needed processing.