Mingli Han

Automatic Hysteresis Modeling of Piezoelectric Micromanipulator in Vision-Guided Micromanipulation Systems

Conventional hysteresis modeling of piezoelectric actuators using interferometers or capacitive sensors is often performed off-line. However, the hysteresis of the piezoelectric actuator changes as the load acting on it or the driving frequency of the input signal alters, demanding that the hysteresis of the micromanipulator be modeled on the fly. The employment of interferometers or capacitive sensors is a challenging task in micromanipulation systems due to their special requirements, e.g., the micropipette tip is desired to provide mirror-like reflection of the incoming beam if an interferometer is employed while a capacitive sensor might not be easily placed in the workspace. An automatic Prandtl-Ishlinskii hysteresis modeling method is proposed and implemented using vision-feedback. The method can be conducted on the fly in real time making it suitable for time critical vision-guided micromanipulation, while providing comparable accuracy with that of using interferometers.

Automatic control of mechanical forces acting on cell biomembranes using a vision-guided microrobotic system in computer microscopy.

A prototype for automatic control of mechanical forces acting on cell biomembranes is proposed in this paper. This prototype consists of vision‐guided position control of the holder and micro‐force sensor, automatic mechanical property characterization of cell biomembranes and automatic control of mechanical forces acting on cell biomembranes. A template‐free calibration method and autofocusing of multiple objects are introduced in the vision‐guided position control to minimize external biological contamination and position the cell, holder and micro‐force sensor into the same focal plane, respectively. A third‐order polynomial modified from biomembrane point‐load model describing the relationship between the measured mechanical force and the deformations of biomembranes is proposed. This simplified model is easily identified and inversed to facilitate the automatic control of mechanical forces. Experimental results based on zebrafish embryos demonstrate the feasibility of the proposed prototype.

Automatic Vision Guided Small Cell Injection: Feature Detection, Positioning, Penetration and Injection

Vision guided automatic cell injection system has become increasingly important in the past ten years. The existing cell injection systems mainly deal with large cells with size bigger than 40 micrometers. However, when dealing with small cells, the internal and external disturbances presented in the system may affect the success rate of injections significantly. In this paper, the system setup of vision guided cell injection system is presented. Feature tracking, micropipette positioning, cell membrane penetration and biological material injection processes for small cell with size around 20 micrometers are discussed. Sum of square difference (SSD) tracking algorithm based on 2 dimensional (2D)-to-2D feature correspondences is used to handle disturbances. A modified proportional position controller is adopted to control the micropipette. The importance of cell membrane penetration modeling is emphasized.

Plant Cell Injection Based on Autofocusing Algorithm

Plant cell injection is difficult due to the complicated structure of cells. Possesses great challenges to carry out the injection task automatically because of the large area it covers and the multiple layers of cells. In this paper, an automatic injection system based on the microscopic focus measurement is developed to automate injection process and also to enable the flexible selection of the target cells. A new strategy is presented to overcome the problem of that the micromanipulator and cells cannot be in the same focal plane during the injection process. The depth information which is the key unknown variable needed to accomplish the injection process is estimated by the autofocusing algorithm. Complex biomanipulation task, like physical transfer of material into various locations within the plant cells, is possible and the successful rate is quantified.

Vision based cell strain modeling and control system

Mechanical force and geometric deformation that are applied on the cell will be transduced into biomedical signals which has been known as mechanotransduction. In turn, we expect that a particular cell function could be realized by giving a specific physical stimulus to the cell. In this work, a novel cell membrane strain model for circular cell membrane under indentation is developed. Based on this model, the deformation of the cell membrane is comprehensively measured. A cell strain modeling and control system is proposed to identify the deformability of the cell membrane and control the membrane to deform into the desired profile. The system employs the visual servoing to realize the feedback control on the cell membrane. An automatic cell boundary detection scheme is utilized to effectively optimize the visual module. With the developed control scheme, cells can be stressed into prescribed deformation without the knowledge of cells mechanical properties which enables the biologist to focus on the physical change of the cell and stimulate a desired activity in the cell by implementing the proposed system.

Calibration of piezoelectric actuator-based vision guided cell microinjection system

In this paper, a new calibration method for piezoelectric actuator-based vision guided cell microinjection system is proposed. This method puts all the parameterspsila uncertainties in a lumped rotational matrix instead of classifying the parameters into extrinsic and intrinsic categories. The difference between the measured and nominal output is assumed to be caused by parameterspsila uncertainties. Parameterspsila uncertainties are identified by collecting 3 pairs of input and output differences. Image-based and actuator-based position controllers are designed based on the calibration results. Experiments are made to demonstrate the usefulness of this calibration method.

Autofocusing algorithm comparison in bright field microscopy for automatic vision aided cell micromanipulation

Autofocusing is an essential technique in many machine vision aided microscopy application. This paper presents a comparison study of 6 autofocusing algorithms under bright field illumination: a) Normalized Variance (VAR), b) Tenengrad Gradient (TEN), c) DB06 wavelet filter (DB06), d) Fast Fourier Transform (FFT), e) Standard Deviation (STD) and f) Sum Modulus Difference (SMD). In the study, all the 6 algorithms are integrated with the exhaustive search technique and implemented using LabVIEW on a Pentium 4 desktop computer. A total of 2,204 microscope images of a micropipette tip are acquired at different microscope objective positions controlled by a high precision stepper motor under 2.8X magnification, are used to evaluate the performance of the algorithms in terms of processing speed, accuracy, consistency, sensitivity to image size and sensitivity to movement step resolution. It can be concluded that VAR and STD perform well in all performance measures.

Position Control using 2D-to-2D Feature Correspondences in Vision Guided Cell Micromanipulation

Conventional camera calibration that utilizes the extrinsic and intrinsic parameters of the camera and the objects has certain limitations for micro-level cell operations due to the presence of hardware deviations and external disturbances during the experimental process, thereby invalidating the extrinsic parameters. This invalidation is often neglected in macro-world visual servoing and affects the visual image processing quality, causing deviation from the desired position in micro-level cell operations. To increase the success rate of vision guided biological micromanipulations, a novel algorithm monitoring the changing image pattern of the manipulators including the injection micropipette and cell holder is designed and implemented based on 2 dimensional (2D)-to 2D feature correspondences and can adjust the manipulator and perform position control simultaneously. When any deviation is found, the manipulator is retracted to the initial focusing plane before continuing the operation.

Real-time modeling and control of the circular cell membranes strain

Making changes of cells function by regulating the external mechanical environment is one of the major interests in the mechanobiology field. Based on extensive studies, force at nano-to-micro newton and geometric shape changes at nano-to-micrometer are the physical stimuli that can be sensed by cells. It has been postulated that controllable cell responses can be produced by activating diversity of mechanosesory proteins through physical changes of force or shape. In this paper, a real-time machine vision algorithm is proposed to improve the efficiency and robustness of the cell membranes strain calculation. The proposed adaptive image thresholding method with modified numerical implementation is able to apply on all the images captured in the deforming process to extract the deformed cell boundary in real-time. Based on the proposed method, the cell membrane strain is modeled and controlled to deform the cell into a predefined deformation. It enables biologists to study the biochemical changes within the cell by providing a controllable geometric changes. It also expects that a particular cell status or function could be produced by giving a proper deformation.

Real-time stressing and force sensing on biological cells

Biological cells possess biochemical modules and physical shapes to maintain appropriate biological function. Different types of force and deformation are applied on cells to investigate the response and mechanical properties. In the biophysics field, studies use indentation deformation on cell membranes to examine the elastic-viscoelastic properties of biological cells. Experiments in different predefined profiles and frequencies are required to test the fidelity and predictive capability of cells creep function. The accuracy and the repeatability of the given stimulus are the significant factor in the experiments to obtain reliable measurements, which are very difficult to realize using manual operations. Automatic micromanipulation systems have substantial advantages over the conventional manual operations in aspects of reliability, accuracy and repeatability. In this paper, an automatic micromanipulation system is introduced and a series of experiments are conducted to stress zebrafish embryo in different sinusoidal profiles. The experimental results show that the system is able to stress the biological cell in desired stimulation and give consistent force outputs in realtime, meanwhile mechanical properties of the zebrafish embryo are also analyzed.