Robert B. Kelley

Camera models based on data from two calibration planes

Abstract An image is a projection of a three dimensional space onto a two dimensional space. Camera calibration supplies the model parameter values that are needed to compute the line of sight ray in space that corresponds to a point in the image plane. Three models, which were based on data from two calibration planes, are compared with the classical pin-hole camera model. Tests were performed which determined the accuracy of the four modeling techniques for solid state and vidicon cameras. Different lenses were also tested. The two plane linear spline model had the best accuracy; the two plane quadratic model performed nearly as well and had smaller time and storage requirements. The pin-hole camera model was uniformly outperformed by the two plane models, particularly for the vidicon camera tested.

A Robot System Which Acquires Cylindrical Workpieces from Bins

The feasibility of robots employing vision to acquire randomly oriented cylinders has been demonstrated for the first time. An experimental robot system using vision and a parallel jaw gripper was able to acquire randomly oriented cylindrical workpieces piled in bins. Binary image analysis was adequate to guide the gripper into the multilayered, piles. Complementary information was provided by simple sensors on the gripper. Experiments were performed using titanium cylinders 6 cm × 15 cm diameter and 7.6 cm × 3 cm diameter. Cycle times to acquire a cylinder and deliver it to a receiving chute ranged from 8 to 10 s when a single supply of one-size cylinders was used. By using a dual supply bin configuration with one bin of each size and overlapping arm motion and image analysis tasks, the cycle times for one cylinder from alternate bins ranged from 5.5 to 7.5 s per piece. In the future robots with such capabilities can be applied to enhance automation applications, especially in small batch production.

An Orienting Robot for Feeding Workpieces Stored in Bins

A robot system has been tested which can acquire a class of workpieces that are unoriented in bins and transport them, one at a time, to a destination with the proper orientation. This kind of robot has numerous applications in industries which manufacture discrete-part products in batches. A variety of approaches to the design of an orienting robot have been identified. For acquisition, gripper design and vision algorithms should be considered together. A vacuum gripper which adapts to surface angle and an algorithm which finds the center of smooth regions in the bin image have been shown to form an effective combination. Knowledge of the position and orientation of the workpieces near the top of the bin is not necessary for acquisition. Collisions with workpieces in the bin whose poses are not known can be avoided by having the gripper follow a line of sight approach path. To simplify the computation of workpiece orientation, it was found to be useful to employ a gripper which locked a workpiece in one of a finite number of states and which did not obscure the workpiece. For an orienting robot which is designed to service a variety of different workpieces, using an arm with six degrees of freedom greatly facilitates workpiece transportation.

Three vision algorithms for acquiring workpieces from bins

Automation requires workpieces to be well organized. Current machine feeding technology is dominated by the use of human labor, mechanical feeders, or orientation preservation. All these techniques have problems and limitations that make an alternate solution desirable. Three vision algorithms are presented which enable a computer-controlled robot to acquire a single piece from a bin containing randomly placed identical pieces. The algorithms do not try to solve the problem by identifying the position and orientation of a piece in the bin. Rather, the algorithms recognize where there is a section of any piece which a particular type of gripper would be able to grasp with a reasonable chance of success. Thus the algorithms are holdsite driven and depend on the piece and gripper type. Vacuum cup and parallel-jaw gripper types are treated. The heuristically defined holdsites for vacuum cup grippers are patches of smooth surfaces; for parallel-jaw grippers, holdsites are opposing, linear, or curvilinear parallel edges. The three-dimensional position of holdsites cannot be computed from a single image of the bin. Thus once a hold-sight is identified, the gripper is made to travel along a line-of-sight path. The missing degree of freedom, distance from the holdsite to the camera, is obtained by a contact or noncontact proximity sensor in the gripper. Acquisition requires two other sensors: a grasping sensor to indicate success; and a mechanical overload sensor, to abort the attempt in case of error.

Error Analysis of Surface Normals Determined by Radiometry

Surface normals can be computed from three images of a workpiece taken under three distinct lighting conditions without requiring surface continuity. Radiometric methods are susceptible to systematic errors such as: errors in the measurement of light source orientations; mismatched light source irradiance; detector nonlinearity; the presence of specular reflection or shadows; the spatial and spectral distribution of incident light; surface size, material, and microstructure; and the length and properties of the light source to target path. Typically, a 1° error in surface orientation of a Lambertian workpiece is caused by a 1 percent change in image intensity due to variations in incident light intensity or a 1° change in orientation of a collimated light source. Tests on a white nylon sphere indicate that by using modest error prevention and calibration schemes, surface angles off the camera axis can be computed within 5°, except at edge pixels. Equations for the sensitivity of surface normals to major error sources have been derived. Results of surface normal estimation and edge extraction experiments on various non-Lambertian and textured workpieces are also presented.

General Purpose Hands for Bin-Picking Robots

Bin-picking robots acquire a workpiece from a bin, reorient it, and place it into a machine. However, to achieve adaptability the system must incorporate general purpose hands capable of working with a wide variety of workpieces. To this end, eight guidelines for the design of general purpose hands for bin-picking robots have been identified on the basis of functional requirements. Using the eight guidelines as a framework, three hands were designed. The first design utilized a single vacuum cup which can sense contact, adapt to unknown surface angles, sense grip, and maintain a fixed hand-to-workpiece relationship. The second is a contour adapting vacuum gripper comprised of an array of 20 vacuum-gripping units which are capable of sensing discrete surface contour points while grasping. The third is a parallel jaw arrangement similar to those commonly found in industry, except that it has the capability of sensing overload and the presence of a workpiece within the hand. Each of the hands is described in terms of design features, control systems, and mode of operation. Advantages and disadvantages are indicated.

Image Feature Extraction Using Diameter-Limited Gradient Direction Histograms

Features extracted by operators which examine diameter-limited gradient direction histograms are important because they describe images of industrial workpieces efficiently and have the potential for rapid computation via special purpose hardware. When such operators are passed over an image in raster fashion, features such as the following are detected: a strong peak in the direction histogram indicating the presence of a relatively straight edge, a second strong direction indicating a corner, a wide direction group indicating a curved edge, and a uniform distribution of directions over the histogram indicating small holes. Diameter-limited optimization can be used to substantially reduce the number of pixels which have been given feature labels by such operators without losing descriptive power. Feature labels for direction histograms having a second strong direction, a wide direction group, or a uniform distribution might be retained only if all other pixels within a circular aperture have a lower bin value. Pixels with a strong direction label might be retained only if all other pixels within a circular aperture and along the gradient direction have a smaller bin value. Tracking can then be applied to the remaining strong direction pixels in the direction perpendicular to the gradient to achieve representation of edges by endpoints. Experimental results indicate that descriptions which are compatible with human interpretation can be achieved.

Visually estimating workpiece pose in a robot hand using the feature points method

An automatic method of visually estimating the pose of a workpiece in a robot hand has been developed. This method is not subject to the constraint that the workpiece have one of a finite number of states in the robot hand. Workpiece pose is estimated using the three dimensional locations of workpiece feature points. Workpiece features can be located in space by image feature tracking during a known arm motion and by using trigonometric relations. The correspondence to features on a model of the workpiece can be established by using transformations which verify interfeature distances and angles. The ability to estimate workpiece pose is necessary to enable robots to accomplish many industrial tasks.

Visual Control For Robot Handling Of Unoriented Parts

Robots guided by image analysis have successfully acquired parts from bins, computed part position and orientation, and performed manipulation to bring parts to the same position and orientation. This task is common to many workstations in manufacturing industries since parts are often supplied in bins and must be oriented before insertion into a machine. Systems which are currently technically demonstrated as feasible have profitable applications now, but the number of suitable applications now is small when compared to those which will be appropriate in several years. More research is needed to make such robot systems fast, easily programmed, inexpensive, reliable, and applicable to a wider variety of parts. The roles of vision include directing a hand into a bin to grasp a single piece, computing the orientation of a piece in a robots hand, piece guidance into a fixture, verification of placement in a fixture, and inspection for piece integrity.

Binary And Gray Scale Robot Vision

The use of industrial robots in a non-ideal environment requires sensors. A versatile general-purpose sensor is a machine vision system. Rather than attempting to review the capabilities of commercially available machine vision hardware, this paper examines the application of both binary and gray scale techniques to the sensory tasks associated with robot part acquisition, part inspection, and part reorientation. The utility of hybrid (mixed binary and gray scale) techniques in robot vision is illustrated.