Michael A. Greminger

Vision-based force measurement

This paper demonstrates a method to visually measure the force distribution applied to a linearly elastic object using the contour data in an image. The force measurement is accomplished by making use of the result from linear elasticity that the displacement field of the contour of a linearly elastic object is sufficient to completely recover the force distribution applied to the object. This result leads naturally to a deformable template matching approach where the template is deformed according to the governing equations of linear elasticity. An energy minimization method is used to match the template to the contour data in the image. This technique of visually measuring forces we refer to as vision-based force measurement (VBFM). VBFM has the potential to increase the robustness and reliability of micromanipulation and biomanipulation tasks where force sensing is essential for success. The effectiveness of VBFM is demonstrated for both a microcantilever beam and a microgripper. A sensor resolution of less than +/-3 nN for the microcantilever and +/-3 mN for the microgripper was achieved using VBFM. Performance optimizations for the energy minimization problem are also discussed that make this algorithm feasible for real-time applications.

The development of a MEMS gyroscope for absolute angle measurement

Microelectromechanical systems (MEMS) gyroscopes are typically designed to measure angular rate of rotation. A measurement of the angle itself is useful in many applications but cannot be obtained by integrating the angular rate due to the presence of bias errors which cause a drift. This paper presents an innovative design for a vibrating gyroscope that can directly measure both angle and angular rate. The design is based on the principle of measuring the angle of free vibration of a suspended mass with respect to the casing of the gyroscope. Several critical challenges have to be handled before the theoretical sensing concept can be converted into a reliable practical sensor. These include compensating for the presence of dissipative forces, mismatched springs, cross-axis stiffness and transmission of rotary torque. These challenges are addressed by the development of a composite nonlinear feedback control system that compensates for each of the above effects and ensures that the mass continues to behave as a freely vibrating structure. Theoretical analysis and simulation results presented in the paper show that the gyroscope can accurately measure both angle and angular rate for low-bandwidth applications.

Sensing nanonewton level forces by visually tracking structural deformations

When assembling MEMS devices or manipulating biological cells it is often beneficial to have information about the force that is being applied to these objects. This force information is difficult to measure at these scales. We demonstrate a method to reliably measure nanonewton scale forces applied to a micro scale cantilever beam using a computer vision approach. A template matching algorithm is used to estimate the beam deflection to sub-pixel resolution in order to determine the force applied to the beam. The template, in addition to containing information about the geometry of the beam, contains information about the elastic properties of the beam. Minimizing the error between this elastic template and the actual image by means of numerical optimization techniques, we are able to measure forces to within /spl plusmn/3 nN. In addition, we also discuss how this method can be generalized to measure forces in elastic configurations other than a simple cantilever beam using a micro-tweezer as an example. This provides the opportunity for this method to be used with specially designed micromanipulators to provide force as well as vision feedback for micromanipulation tasks.

Modeling elastic objects with neural networks for vision-based force measurement

This paper presents a method to model the deformation of an elastic object with an artificial neural network. The neural network is trained directly from images of the elastic object deforming under known loads. Using this process, models can be created for objects such as biological tissues that cannot be modeled by existing techniques. The neural network elastic model is used in conjunction with a deformable template matching algorithm to perform vision-based force measurement (VBFM). We demonstrate this learning method on objects with both linear and nonlinear elastic properties.

Detection of heteroplasmy in individual mitochondrial particles

AbstractMitochondrial DNA (mtDNA) mutations have been associated with disease and aging. Since each cell has thousands of mtDNA copies, clustered into nucleoids of five to ten mtDNA molecules each, determining the effects of a given mtDNA mutation and their connection with disease phenotype is not straightforward. It has been postulated that heteroplasmy (coexistence of mutated and wild-type DNA) follows simple probability rules dictated by the random distribution of mtDNA molecules at the nucleoid level. This model has been used to explain how mutation levels correlate with the onset of disease phenotype and loss of cellular function. Nonetheless, experimental evidence of heteroplasmy at the nucleoid level is scarce. Here, we report a new method to determine heteroplasmy of individual mitochondrial particles containing one or more nucleoids. The method uses capillary cytometry with laser-induced fluorescence detection to detect individual mitochondrial particles stained with PicoGreen, which makes it possible to quantify the mtDNA copy number of each particle. After detection, one or more particles are collected into polymerase chain reaction (PCR) wells and then subjected to real-time multiplexed PCR amplification. This PCR strategy is suitable to obtain the relative abundance of mutated and wild-type mtDNA. The results obtained here indicate that individual mitochondrial particles and nucleoids contained within these particles are not heteroplasmic. The results presented here suggest that current models of mtDNA segregation and distribution (i.e., heteroplasmic nucleoids) need further consideration. FigureSetup for collection of individual mitochondrial particles into PCR vials after their laser-induced fluorescence detection (image in the background). Laser-induced fluorescence detection of individual mitochondria particles (upper right). Multiplex real time PCR of the collected sample reveals the presence of wild type and deleted mtDNA (i.e., heteroplasmy) when the traces cross the upper and lower thresholds, respectively (lower right).

Deformable object tracking using the boundary element method

This paper presents a method to perform 2D (two-dimensional) deformable object tracking using the boundary element method (BEM). BEM, like the finite element method (FEM), is a technique to model an elastic solid. BEM differs from FEM in that only the contour of an object needs to be meshed for BEM, making this method attractive for computer vision problems. For FEM, the interior of the object must be meshed also. In order to track deformable objects, a deformable template is defined that uses BEM to model displacements. The template is registered to the image by applying a force field that deforms the template to match the image. This force field is found using an energy minimization approach. Even though the deformable template uses a linear elastic model, it can be used to track the deformations of objects with nonlinear material properties or in cases where there are large deformations. We demonstrate the performance of this method on objects with linear and nonlinear elastic properties. In addition, it is discussed how this method can be readily extended to 3D (three-dimensional) deformable object tracking.

Investigating Protein Structure Change in the Zona Pellucida with a Microrobotic System

In this paper we present a microrobotic system that integrates microscope vision and microforce feedback for characterizing biomembrane mechanical properties. We describe robust visual tracking of deformable biomembrane contours using physics-based models. A multi-axis microelectromechanical systems based force sensor is used to determine applied forces on biomembranes and to develop a novel biomembrane mechanical model. By visually extracting biomembrane deformations during loading, geometry changes can be used to estimate applied forces using a biomembrane mechanical model and the determined elastic modulus. Forces on a biomembrane can be visually observed and controlled, thus creating a framework for vision and force assimilated cell manipulation. The experimental results quantitatively describe a stiffness increase seen in the mouse zona pellucida (ZP) after fertilization. Understanding this stiffness increase, referred to as “zona hardening”, helps provide an understanding of ZP protein structure development, i.e., an increase in the number of cross links of protein ZP1 between ZP2 and ZP3 units that is conjectured to be responsible for zona hardening. Furthermore, the system, technique, and model presented in this paper can be applied to investigating mechanical properties of other biomembranes and other cell types, which has the potential to facilitate many biological studies, such as cell injury and recovery where biomembrane mechanical property changes need to be monitored.

A Deformable Object Tracking Algorithm Based on the Boundary Element Method that is Robust to Occlusions and Spurious Edges

Abstract The manipulation of deformable objects is an important problem in robotics and arises in many applications including biomanipulation, microassembly, and robotic surgery. For some applications, the robotic manipulator itself may be deformable. Vision-based deformable object tracking can provide feedback for these applications. Computer vision is a logical sensing choice for tracking deformable objects because the large amount of data that is collected by a vision system allows many points within the deformable object to be tracked simultaneously. This article introduces a template based deformable object tracking algorithm, based on the boundary element method, that is able to track a wide range of deformable objects. The robustness of this algorithm to occlusions and to spurious edges in the source image is also demonstrated. A robust error measure is used to handle the problem of occlusion and an improved edge detector based on the Canny edge operator is used to suppress spurious edges. This article concludes by quantifying the performance increase provided by the robust error measure and the robust edge detector. The performance of the algorithm is also demonstrated through the tracking of a sequence of cardiac MRI images.

Investigating protein structure with a microrobotic system

This paper presents a microrobotic system integrating microscope vision and microforce feedback for characterizing biomembrane mechanical properties. Robust visual tracking of deformable biomembrane contour using physics-based models is described. A multi-axis MEMS-based force sensor is used to determine applied forces on biomembranes and develop a novel biomembrane mechanical model. By visually extracting geometry changes on a biomembrane during loading, geometry changes can be used to estimate applied forces using the biomembrane mechanical model and the determined elastic modulus. Forces on a biomembrane can be visually observed and controlled, thus creating a framework for vision and force assimilated cell manipulation. The experimental results quantitatively describe mouse zona pellucida (ZP) stiffness increase during ZP hardening and provide an understanding of ZP protein structure development, i.e., an increase in the number of cross links of protein ZP1 between ZP2-ZP3 units that is conjectured to be responsible for ZP stiffness increase. Furthermore, the system, technique, and model presented in this paper can be applied to investigating mechanical properties of other biomembranes and other cell types, which has the potential to facilitate many biological studies, such as cell injury and recovery where biomembrane mechanical property changes need to be monitored.

A four degree of freedom MEMS microgripper with novel bi-directional thermal actuators

A four degree of freedom thermally actuated MEMS microgripper is presented in this paper. Each jaw of the microgripper has independent x and y degrees of freedom. These extra degrees of freedom give the gripper added dexterity for manipulating microparts. The motion of each gripper jaw is provided by a two degree of freedom compliant mechanism which is based on a five bar rigid link mechanism. This gripper also introduces a novel thermal microactuator design that is capable of bi-directional actuation giving it a greater range of motion than previous thermal actuator designs. The actuator provides a total range of motion of 12.7 /spl mu/m and a maximum force of 1.9 mN. Also, since this microgripper is based on a compliant mechanism, deformable object tracking can be used to provide force as well as position feedback for the gripper. This combination of an increased number of degrees of freedom and increased sensory feedback provides a level of dexterity that has not been previously available in microassembly.