Zhengtao Zhang

Visual Measurement and Prediction of Ball Trajectory for Table Tennis Robot

A high-speed stereovision system with two smart cameras is presented to track a table tennis ball, which adopts a distributed parallel processing architecture based on a local area network. A set of novel algorithms with little computation and good robustness running in the smart cameras is also proposed to recognize and track the ball in the images. A computer receives the image coordinates of the ball from the cameras via the local area network and computes its 3-D positions in the working frame. Then, the flying trajectory of the ball is estimated and predicted according to the measured positions and the flying and rebound models. The main motion parameters of the ball such as the landing point and striking point are calculated from its predicted trajectory. Experimental results show that the developed image-processing algorithms are robust enough to distinguish the ball from a complex dynamic background. The predicted landing point and striking point of the ball have satisfactory precision.

High Precision Automatic Assembly Based on Microscopic Vision and Force Information

An automatic system is developed to realize high precision assembly of two components in the size of mm level with an interference fit in 3-dimensional (3-D) space with 6-degree-of- freedoms (DOF), which consists of a manipulator, an adjusting platform, a sensing system and a computer. The manipulator is employed to align component B to the component A in position. The adjusting platform aligns the component A to component B in orientations and inserts A into B. The sensing system includes three microscopes and a force sensor. The three microscopes are mounted approximately orthogonal to observe components from different directions in the aligning stage. The force sensor is introduced to detect the contact force in assembly process. In the aligning stage, a pose control method based on image Jacobian matrix is proposed. In the insertion stage, a position control method based on the contact force is proposed. The calibration of image Jacobian matrix is also presented. Experimental results demonstrate the effectiveness of the proposed system and methods.

Aligning micro-gripper to ring object in high precision with microscope vision

In this paper, a platform based on microscope vision is established to align and grip a ring object. An effective alignment algorithm is proposed based on a microscope vision system in order to realize the gripping in high precision. The multiple features of the micro-gripper in the image such as the edge length and the circle center are detected to regulate the microscope vision system. The desired position and orientation of the ring object is determined according to the features of the micro-gripper in the image captured by the horizontal camera. The ring objects pose is adjusted to the desired pose above. Once the ring objects pose coincides with the micro-grippers pose, the micro-gripper is moved to the ring object under the visual guidance of the vertical camera and grips it. Experimental results demonstrate the effectiveness of the alignment algorithm.

Relative Pose Estimation for Alignment of Long Cylindrical Components Based on Microscopic Vision

In this paper, an efficient relative pose estimation method based on multimicroscopic vision is presented for the alignment of long cylindrical components in six degree-of-freedom in the 3-D space. First, the relative pose estimation method measures the relative orientation errors between components in a coarse-to-fine manner. The coarse relative orientation estimation reflects the average orientation difference among multiedges and it promises the relative orientation errors can converge to a limited extent rapidly. The fine relative orientation estimation is to detect whether the components will interfere with each other in the insertion process and it provides higher orientation measurement accuracy under multilateral constraint circumstance. The method to determine the invisible feature lines of long cylindrical components of prism shape with odd edges for relative position error estimation is discussed in detail. Afterward, the relative position between components is measured by side-view cameras using the feature lines. In addition, the method to calibrate the optical axis of microscopic camera is also presented. The relative pose estimation method can reliably estimate the relative pose between long cylindrical components of prism shape with odd edges. Alignment experiments and results demonstrate the effectiveness of the proposed relative pose estimation method.

A Novel and Effective Surface Flaw Inspection Instrument for Large-Aperture Optical Elements

Surface defects on precision optical elements must be carefully inspected since they impact the normal operation of an optical system. It is a challenge to inspect defects of large-aperture optical elements using an imaging system because of efficiency and accuracy. This paper designs a novel and effective inspection instrument with two imaging systems for large-aperture optical elements. They are the dark-field imaging system (DFIS) with a line scan camera in 10-μm resolution and the bright-field (BF) imaging system with a microscopic camera in a 0.85-μm resolution. To keep the clarity of the DFIS in large-scope quickly scanning, an adaptive scan path planning method is proposed. A set of novel algorithms is proposed to process a large number of dark-field images. Due to the limitations of the DFIS in scattering effect, the corresponding BF images are obtained according to the dark-field images. The classification of flaws and their sizes measurement based on BF images are also presented. The experiments show that the instrument can scan an optical element with the size of 810 mm × 460 mm in less than 6 min and the inspection precision can reach 3 μm.

Precision Assembly Among Multiple Thin Objects With Various Fit Types

This paper investigates precision assembly combining several subprocesses on a platform with multiple robot arms. The task manipulates irregular objects and incorporates interference and clearance assemblies. We use a mechanism with six robot arms and three microscopes to fulfill this assignment, and propose a general control strategy to attain high precision and protect objects. System calibration includes the image Jacobian matrix to connect Cartesian motion with the image movement and force transformation matrix to relate between coordinates of force sensor and manipulators. In the control structure, a general strategy is proposed for assembly with multiple manipulators, in which we present an assembly plan module to generate desired states of each object by using an optimization approach, design a method to determine the collision when blocking, and propose a hybrid control scheme for both contact and clearance assembly controllers. Experiments are carried out to demonstrate the validation of the proposed methods.

Design of microassembly system and research on coarse-to-fine alignment strategy in combination with active zooming

In this paper, a microassembly system based on 3-channles microvision system used for assembling 2 microparts named Si arm and Al shell is presented. Firstly, the structure of the system is described in detail. Then, a coarse-to-fine alignment strategy in combination with active zooming algorithm is presented. In the coarse alignment stage, alignment process is conducted with maximum field of view (FOV). In the fine alignment stage, the microscope is of maximum magnification to ensure the highest assembly accuracy. At last, the relative pose of the microparts is estimated in the assembly procedure. The experiment results show that the proposed algorithms can detect the assembly parameters online precisely.

Partially Decoupled Image-Based Visual Servoing Using Different Sensitive Features

A new image-based visual servoing method based on sensitive features is presented to separately realize the position control and orientation control. Line features are used for the orientation control because of their sensitivities to rotational motions. Point features and area size features are employed to realize the position control since area size features are very sensitive to the objects’ depths. The translations resulting from rotational motions are introduced into the position control as the compensation in order to eliminate the influence of the camera’s motions on the point features. The depths for all active features are estimated via interaction matrices, features variations, and the executed camera motions. The proposed method can keep the tracked objects in the camera’s field of view in the visual servoing process. In addition, the determination methods of the interaction matrices for point, line, and area size features are proposed. Comparing to the traditional method, the proposed determination method of the interaction matrix for line is independent from the parameters of the plane containing the line. Experimental results verify the effectiveness of the proposed methods.

Motion-based microscopic camera calibration and application on micro tube-hole insertion

Abstract. An automatic approach based on microscopic visual control is proposed for a microassembly task, which is to insert a 10-μm-diameter glass tube into a 12-μm-diameter hole on a silicon substrate. A three-degree-of-freedom manipulator is used to control the motion of the glass tube. A microscopic camera is mounted on a movable platform toward the hole in the inclined direction ∼30  deg from horizontal plane in order to view the hole and the tube. A calibration method based on active motions is designed to estimate the intrinsic and extrinsic parameters of the microscopic camera, which includes only two steps of motion of the tube’s tip on the focal plane of the microscopic camera. The relative position errors are computed from the image feature errors and the parameters of the microscopic camera. A position-based control strategy is applied to align the tip to the position above the hole, which controls the tip to move to the hole along x, y, and z axes simultaneously. The tip is moved down a specified distance to insert into the hole after autofocus in order to improve the insertion accuracy. The experimental results verify the effectiveness of the proposed method.

A fast and robust circle detection method using perpendicular bisector of chords

In this paper a new method to detect the center of a circle is proposed. This method uses pairs of perpendicular bisectors of each two no-repeat intersecting chords to calculate the intersection which is the center of the circle. It can overcome the drawbacks of deviation caused by edge points nonuniformly distributed by centroid method, and drawbacks of highly computation time and a large amount of storage which is brought by Hough Transform, and drawbacks of much disturbance by random noises brought by least square method. Experimental results show that the perpendicular bisector method can eliminate the disturbance of random noises, and can obtain exact circle center rapidly. It can reach an accuracy of less than one pixel for circle center detection.