Bala P. Amavasai

Vision-based closed-loop control of mobile microrobots for microhandling tasks

As part of a European Union ESPRIT funded research project a flexible microrobot system has been developed which can operate under an optical microscope as well as in the chamber of a scanning electron microscope. The system is highly flexible and configurable and uses a wide range of sensors in a closed-loop control strategy. This paper presents an overview of the vision system and its architecture for vision-controlled micro-manipulation. The range of different applications, e.g. assembly of hybrid microsystems, handling of biological cells and manipulation tasks inside an SEM, imposes great demands on the vision system. Fast and reliable object recognition algorithms have been developed and implemented to provide for two modes of operation: automated and semi-automated robot control. The vision system has a modular design, comprising modules for object recognition, tracking and depth estimation. Communication between the vision modules and the control system takes place via a shared memory system embedding an object database. This database holds information about the appearance and the location of all known objects. A depth estimation method based on a modified sheet-of-light triangulation method is also described. Furthermore, the novel approach of electron beam triangulation in the SEM is described.

Computer vision methods for optical microscopes

As the fields of micro- and nano-technology mature, there will be an increased need to build tools that are able to work in these areas. Industry will require solutions for assembling and manipulating components, much as it has done in the macro range. With this need in mind, a new set of challenges requiring novel solutions have to be met. One of them is the ability to provide closed-loop feedback control for manipulators. We foresee that machine vision will play a leading role in this area. This paper introduces a technique for integrating machine vision into the field of micro-technology including two methods, one for tracking and one for depth reconstruction under an optical microscope.

Advanced transmission electron microscope triboprobe with automated closed-loop nanopositioning

Here the design and operation of a novel transmission electron microscope (TEM) triboprobe instrument with real-time vision control for advanced in situ electron microscopy is demonstrated. The NanoLAB triboprobe incorporates a new high stiffness coarse slider design for increased stability and positioning performance. This is linked with an advanced software control system which introduces both new and flexible in situ experimental functional testing modes, plus an automated vision control feedback system. This advancement in instrumentation design unlocks new possibilities of performing a range of new dynamical nanoscale materials tests, including novel friction and fatigue experiments inside the electron microscope.

Machine vision methods for autonomous micro‐robotic systems

Purpose – To develop customised machine vision methods for closed‐loop micro‐robotic control systems. The micro‐robots have applications in areas that require micro‐manipulation and micro‐assembly in the micron and sub‐micron range.Design/methodology/approach – Several novel techniques have been developed to perform calibration, object recognition and object tracking in real‐time under a customised high‐magnification camera system. These new methods combine statistical, neural and morphological approaches.Findings – An in‐depth view of the machine vision sub‐system that was designed for the European MiCRoN project (project no. IST‐2001‐33567) is provided. The issue of cooperation arises when several robots with a variety of on‐board tools are placed in the working environment. By combining multiple vision methods, the information obtained can be used effectively to guide the robots in achieving the pre‐planned tasks.Research limitations/implications – Some of these techniques were developed for micro‐visi...

An Analysis of Collective Movement Models for Robotic Swarms

A swarm is defined as a set of two or more independent homogeneous or heterogeneous agents acting in a common environment, in a coherent fashion, and which generates emergent behavior. The creation of artificial swarms or robotic swarms has attracted many researchers in the last two decades. Many studies have been undertaken using practical approaches to swarm construction such as investigating the navigation of the swarm, task allocation and elementary construction. This paper examines aggregations that emerge from three different movement models of relatively simple agents. The agents only differ in their maximum turning angle and their sensing range.

A machine vision extension for the Ruby programming language

Dynamically typed scripting languages have become popular in recent years. Although interpreted languages allow for substantial reduction of software development time, they are often rejected due to performance concerns. In this paper we present an extension for the programming language Ruby, called HornetsEye, which facilitates the development of real-time machine vision algorithms within Ruby. Apart from providing integration of crucial libraries for input and output, HornetsEye provides fast native implementations (compiled code) for a generic set of array operators. Different array operators were compared with equivalent implementations in C++. Not only was it possible to achieve comparable real-time performance, but also to exceed the efficiency of the C++ implementation in several cases. Implementations of several algorithms were given to demonstrate how the array operators can be used to create concise implementations.

A fully decentralized approach for incremental perception

The area of Swarm Robotics is still in its infancy. Key concepts at the basic level have to be invented and developed in order to achieve the future goal of building large scale physical and controllable autonomous robotic swarms. In this paper we extend the concept of Incremental Perception in swarm robotics into the domain of complete decentralization. Our work is aimed at micro-robotic swarms where the hardware resources available for the robots will be limited. Hence a decentralized system becomes inevitable because it does not require intra-dependence of robot agents, their monitoring system or a communication mechanism for the agents; absence of all these factors results in reduced hardware requirements for the agents. We focus on the co-operative behavior of robots rather than relying on their individual capabilities. We also propose the parameters and functions that are required for a completely decentralized system and show that such a system can be successfully modeled and analyzed.

Steerable Filters Generated with the Hypercomplex Dual-Tree Wavelet Transform

The use of wavelets in the image processing domain is still in its infancy, and largely associated with image compression. With the advent of the dual-tree hypercomplex wavelet transform (D-HWT) and its improved shift invariance and directional selectivity, applications in other areas of image processing are more conceivable. This paper discusses the problems and solutions in developing the DHWT and its inverse. It also offers a practical implementation of the algorithms involved. The aim of this work is to apply the DHWT in machine vision. Tentative work on a possible new way of feature extraction is presented. The paper shows that 2-D hypercomplex basis wavelets can be used to generate steerable filters which allow rotation as well as translation.

Recurrent neural robot controllers: feedback mechanisms for identifying environmental motion dynamics

In this paper a series of recurrent controllers for mobile robots have been developed. The system combines the iterative learning capability of neural controllers and the optimisation ability of particle swarms. In particular, three controllers have been developed: an Exo-sensing, an Ego-sensing and a Composite controller which is the hybrid of the latter two. The task for each controller is to learn to follow a moving target and identify its trajectory using only local information. We show how the learned behaviours of each architecture rely on different sensory representations, although good results are obtained in all cases.

A Dissimilarity Visualisation System for CT: Pilot Study

One of the capabilities of the human vision process when visualising images is the ability to visualise them at different levels of details. A segmentation procedure has been developed to mimic this capability of human vision process. The developed hierarchical clustering based segmentation (HCS) procedure automatically generates a hierarchy of segmented images. The hierarchy represents the continuous merging of similar, spatially adjacent or disjoint, regions as the allowable threshold value of dissimilarity between regions, for merging, is gradually increased. By the very nature of the HCS procedure a large amount of visual information is produced. A graphical user interface (GUI) was designed to present the segmentation output in an informative way for the user to view and interpret. In addition the GUI displays the original image data by optimally mapping the range of data values to the available 256 gray level values. The purpose of this paper is to describe the development of the designed image visualisation system and to demonstrate some of its functionalities