Xiangjun Zou

Robust Grape Cluster Detection in a Vineyard by Combining the AdaBoost Framework and Multiple Color Components

The automatic fruit detection and precision picking in unstructured environments was always a difficult and frontline problem in the harvesting robots field. To realize the accurate identification of grape clusters in a vineyard, an approach for the automatic detection of ripe grape by combining the AdaBoost framework and multiple color components was developed by using a simple vision sensor. This approach mainly included three steps: (1) the dataset of classifier training samples was obtained by capturing the images from grape planting scenes using a color digital camera, extracting the effective color components for grape clusters, and then constructing the corresponding linear classification models using the threshold method; (2) based on these linear models and the dataset, a strong classifier was constructed by using the AdaBoost framework; and (3) all the pixels of the captured images were classified by the strong classifier, the noise was eliminated by the region threshold method and morphological filtering, and the grape clusters were finally marked using the enclosing rectangle method. Nine hundred testing samples were used to verify the constructed strong classifier, and the classification accuracy reached up to 96.56%, higher than other linear classification models. Moreover, 200 images captured under three different illuminations in the vineyard were selected as the testing images on which the proposed approach was applied, and the average detection rate was as high as 93.74%. The experimental results show that the approach can partly restrain the influence of the complex background such as the weather condition, leaves and changing illumination.

A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard

Abstract Reliable and robust vision algorithms to detect the cutting points on peduncles of overlapping grape clusters in the unstructured vineyard are essential for efficient use of a harvesting robot. In this study, we designed an approach to detect these cutting points in three main steps. First, the areas of pixels representing grape clusters in vineyard images were obtained using a segmentation algorithm based on k-means clustering and an effective color component. Next, the edge images of grape clusters were extracted, and then a geometric model was used to obtain the contour intersection points of double overlapping grape clusters. Profile analysis was used to separate the regional pixels of double grape clusters by a line connecting double intersection points. Finally, the region of interest of the peduncle for each grape clusters was determined based on the geometric information of each pixel region, and a computational method was used to determine the appropriate cutting point on the peduncle of each grape cluster by use of a geometric constraint method. Thirty vineyard images that were captured from different perspectives were tested to validate the performance of the presented approach in a complex environment. The average recognition accuracy was 88.33%, and the success rate of visual detection of the cutting point on the peduncle of double overlapping grape clusters was 81.66%. The demonstrated performance of this developed method indicated that it could be used by harvesting robots.

Recognition and Matching of Clustered Mature Litchi Fruits Using Binocular Charge-Coupled Device (CCD) Color Cameras

Recognition and matching of litchi fruits are critical steps for litchi harvesting robots to successfully grasp litchi. However, due to the randomness of litchi growth, such as clustered growth with uncertain number of fruits and random occlusion by leaves, branches and other fruits, the recognition and matching of the fruit become a challenge. Therefore, this study firstly defined mature litchi fruit as three clustered categories. Then an approach for recognition and matching of clustered mature litchi fruit was developed based on litchi color images acquired by binocular charge-coupled device (CCD) color cameras. The approach mainly included three steps: (1) calibration of binocular color cameras and litchi image acquisition; (2) segmentation of litchi fruits using four kinds of supervised classifiers, and recognition of the pre-defined categories of clustered litchi fruit using a pixel threshold method; and (3) matching the recognized clustered fruit using a geometric center-based matching method. The experimental results showed that the proposed recognition method could be robust against the influences of varying illumination and occlusion conditions, and precisely recognize clustered litchi fruit. In the tested 432 clustered litchi fruits, the highest and lowest average recognition rates were 94.17% and 92.00% under sunny back-lighting and partial occlusion, and sunny front-lighting and non-occlusion conditions, respectively. From 50 pairs of tested images, the highest and lowest matching success rates were 97.37% and 91.96% under sunny back-lighting and non-occlusion, and sunny front-lighting and partial occlusion conditions, respectively.