Arul N. Selvan

Vision-based closed-loop control of mobile microrobots for microhandling tasks

As part of a European Union ESPRIT funded research project a flexible microrobot system has been developed which can operate under an optical microscope as well as in the chamber of a scanning electron microscope. The system is highly flexible and configurable and uses a wide range of sensors in a closed-loop control strategy. This paper presents an overview of the vision system and its architecture for vision-controlled micro-manipulation. The range of different applications, e.g. assembly of hybrid microsystems, handling of biological cells and manipulation tasks inside an SEM, imposes great demands on the vision system. Fast and reliable object recognition algorithms have been developed and implemented to provide for two modes of operation: automated and semi-automated robot control. The vision system has a modular design, comprising modules for object recognition, tracking and depth estimation. Communication between the vision modules and the control system takes place via a shared memory system embedding an object database. This database holds information about the appearance and the location of all known objects. A depth estimation method based on a modified sheet-of-light triangulation method is also described. Furthermore, the novel approach of electron beam triangulation in the SEM is described.

Computer vision methods for optical microscopes

As the fields of micro- and nano-technology mature, there will be an increased need to build tools that are able to work in these areas. Industry will require solutions for assembling and manipulating components, much as it has done in the macro range. With this need in mind, a new set of challenges requiring novel solutions have to be met. One of them is the ability to provide closed-loop feedback control for manipulators. We foresee that machine vision will play a leading role in this area. This paper introduces a technique for integrating machine vision into the field of micro-technology including two methods, one for tracking and one for depth reconstruction under an optical microscope.

Machine vision methods for autonomous micro‐robotic systems

Purpose – To develop customised machine vision methods for closed‐loop micro‐robotic control systems. The micro‐robots have applications in areas that require micro‐manipulation and micro‐assembly in the micron and sub‐micron range.Design/methodology/approach – Several novel techniques have been developed to perform calibration, object recognition and object tracking in real‐time under a customised high‐magnification camera system. These new methods combine statistical, neural and morphological approaches.Findings – An in‐depth view of the machine vision sub‐system that was designed for the European MiCRoN project (project no. IST‐2001‐33567) is provided. The issue of cooperation arises when several robots with a variety of on‐board tools are placed in the working environment. By combining multiple vision methods, the information obtained can be used effectively to guide the robots in achieving the pre‐planned tasks.Research limitations/implications – Some of these techniques were developed for micro‐visi...

Communication of medical images to diverse audiences using multimodal imaging

A study has been completed examining design issues concerning the interpretation of and dissemination of multimodal medical imaging data sets to diverse audiences. To create a model data set mouse fibrosarcoma tissue was visualised via magnetic resonance imaging (MRI), Matrix-Assisted Laser Desorption/Ionisation-Mass Spectrometry (MALDI-MSI) and histology. MRI images were acquired using the 0.25T Esaote GScan; MALDI images were acquired using a Q-Star Pulsar I mass spectrometer. Histological staining of the same tissue sections used for MALDI-MSI was then carried out. Areas assigned to hemosiderin deposits due to haemorrhaging could be visualised via MRI. In the MALDI-MSI data obtained the distribution sphingomyelin species could be used to identify regions of viable tumour. Mathematical ‘up sampling’ using hierarchical clustering-based segmentation provided a sophisticated image enhancement tool for both MRI and MALDI-MS and assisted in the correlation of images.

Recurrent neural robot controllers: feedback mechanisms for identifying environmental motion dynamics

In this paper a series of recurrent controllers for mobile robots have been developed. The system combines the iterative learning capability of neural controllers and the optimisation ability of particle swarms. In particular, three controllers have been developed: an Exo-sensing, an Ego-sensing and a Composite controller which is the hybrid of the latter two. The task for each controller is to learn to follow a moving target and identify its trajectory using only local information. We show how the learned behaviours of each architecture rely on different sensory representations, although good results are obtained in all cases.

A Dissimilarity Visualisation System for CT: Pilot Study

One of the capabilities of the human vision process when visualising images is the ability to visualise them at different levels of details. A segmentation procedure has been developed to mimic this capability of human vision process. The developed hierarchical clustering based segmentation (HCS) procedure automatically generates a hierarchy of segmented images. The hierarchy represents the continuous merging of similar, spatially adjacent or disjoint, regions as the allowable threshold value of dissimilarity between regions, for merging, is gradually increased. By the very nature of the HCS procedure a large amount of visual information is produced. A graphical user interface (GUI) was designed to present the segmentation output in an informative way for the user to view and interpret. In addition the GUI displays the original image data by optimally mapping the range of data values to the available 256 gray level values. The purpose of this paper is to describe the development of the designed image visualisation system and to demonstrate some of its functionalities

Infrared Thermal Mapping, Analysis and Interpretation in Biomedicine

Measurement of body temperature is one of the cornerstones of clinical assessment in medicine. Skin, the largest organ of the human body, is essentially a temperature mosaic determined by the rate of blood flow through arterioles and capillaries adjacent to the skin . This makes the conventional methods of ‘spot’ measurement rather limited in providing detailed information of regional skin temperature. Infrared (IR) thermal imaging however has the potential to provide a robust method of surface temperature mapping in disease states where pathology disturbs the ‘normal’ distribution of blood flow to skin. To advance image interpretation from the conventional qualitative narrative to a quantitative and robust system, analytical developments focus on digital images and require computer-aided systems to produce results rapidly and safely. Hierarchical clustering-based segmentation (HCS) provides a generic solution to the complex interpretation of thermal data (pixel by pixel) to produce clusters and boundary regions at levels not discernible by human visual processing. In this chapter, HCS has been used to aid the interpretation of wound images and to identify variations in temperature clusters around and along the surgical wound for their clinical relevance in wound infection .

A perceptual aid to delineating the extent of potential mammographic abnormalities

Being able to accurately determine the extent of a possible malignancy on a mammogram is an important task as this can affect the potential follow up surgical treatment that a woman receives after breast screening. It is known that this can be a difficult task, particularly where the lesion has diffuse abnormalities. A potential computer-aided approach is to employ Hierarchical Clustering-based Segmentation (HCS) and this interactive educational exhibit dynamically demonstrates this technique. HCS is an unsupervised segmentation process that when applied to an image yields a hierarchy of segmentations based on image pixel dissimilarities and so can be used to highlight areas in the mammographic image to aid interpretation.