Brian L. Steward

Color image segmentation with genetic algorithm for in-field weed sensing.

This study was undertaken to develop machine vision-based weed detection technology for outdoor natural lighting conditions. Supervised color image segmentation using a binary-coded genetic algorithm (GA) identifying a region in Hue-Saturation-Intensity (HSI) color space (GAHSI) for outdoor field weed sensing was successfully implemented. Images from two extreme intensity lighting conditions, those under sunny and cloudy sky conditions, were mosaicked to explore the possibility of using GAHSI to locate a plant region in color space when these two extremes were presented simultaneously. The GAHSI result provided evidence for the existence and separability of such a region. In the experiment, GAHSI performance was measured by comparing the GAHSI-segmented image with a corresponding hand- segmented reference image. When compared with cluster analysis-based segmentation results, the GAHSI achieved equivalent performance. Keywords. Genetic algorithm, Weed sensing, Color image segmentation, Lighting condition.

CLASSIFICATION OF BROADLEAF AND GRASS WEEDS USING GABOR WAVELETS AND AN ARTIFICIAL NEURAL NETWORK

A texture–based weed classification method was developed. The method consisted of a low–level Gabor wavelets–based feature extraction algorithm and a high–level neural network–based pattern recognition algorithm. This classification method was specifically developed to explore the feasibility of classifying weed images into broadleaf and grass categories for spatially selective weed control. In this research, three species of broadleaf weeds (common cocklebur, velvetleaf, and ivyleaf morning glory) and two grasses (giant foxtail and crabgrass) that are common in Illinois were studied. After processing 40 sample images with 20 samples from each class, the results showed that the method was capable of classifying all the samples correctly with high computational efficiency, demonstrating its potential for practical implementation under real–time constraints.

Automatic Corn Plant Population Measurement Using Machine Vision

From yield monitoring data, it is well known that yield variability exists within a field. Plant population variation is a major cause of this yield variability. Automated corn plant population measurement has potential for assessing in-field variation of plant emergence and also for assessing planter performance. Machine vision algorithms for automated corn plant counting were developed to analyze digital video streams. Video streams were captured along 6.1 m long cornrow sections at early stages of plant growth and various natural daylight conditions. A sequential image correspondence algorithm was used to determine overlapped image portions. Plants were segmented from the background using an ellipsoidal decision surface, and spatial analysis was used to identify individual crop plants. Performance of this automated method was evaluated by comparing its results with manual stand counts. Sixty experimental units were evaluated for counting results with corn population varying from 14 to 48 plants per 6.1 cornrow length. The results showed that in low weed field conditions, the system plants counts well correlated to manual counts (R 2 = 0.90). Standard error of population estimate was 1.8 plants over 34.3 manual plant count that corresponds to 5.4% of average error.

Real-time weed detection in outdoor field conditions

Though most herbicide is applied uniformly in agronomic fields, there is strong evidence that weeds are not distributed uniformly within the crop fields. If an effective weed detection system were developed, both economic and environmental benefits would result from its use for site-specific weed management. Past work in this area has focused mainly on either low spatial resolution photo-detectors or off-line machine vision system. This study was undertaken to develop real-time machine vision weed detection for outdoor lighting conditions. The novel environmentally adaptive segmentation algorithm was developed with the objective of real-time operation on an on-board computer-based system. The EASA used cluster analysis to group pixels of homogeneous color regions of the image together which formed the basis for image segmentation. The performance of several variations of this algorithm was measured by comparing segmented field images produced by the EASA, fixed-color HSI region segmentation, and ISODATA clustering with hand-=segmented reference images. The time cost and questionable accuracy of hand- segmented reference images led to exploration of the use of computer-segmented reference images. Sensitivity and background sensitivity were used as performance measured. Significant differences were found between the means of sensitivity, background sensitivity, and overall performance across segmentation schemes. Similar results were obtained with computer-segmented reference images.

MACHINE-VISION WEED DENSITY ESTIMATION FOR REAL-TIME, OUTDOOR LIGHTING CONDITIONS

A system to estimate the weed density between two rows of soybeans was developed. An environmentally adaptive segmentation algorithm (EASA) was used to segment the plants from the background of the image. The effect of two image data transformations on the segmentation performance of the EASA was investigated, and the RGB-IV1V2 transformation resulted in significantly higher quality segmentation results based on morphological opening and closing pixel loss over the RGB-rgb transformation. An adaptive scanning algorithm (ASA) was developed and used to automatically detect crop inter-row edges and to estimate the number of weeds in the inter-row area. Two sets of images were acquired under sunny and overcast sky conditions. The ASA-detected crop row edge positions were significantly correlated with the manually detected crop row positions, with the distribution skewed towards positions internal to the row. ASA weed density estimates were highly correlated with manual weed counts for both lighting conditions. However, when a limited range of the data was considered, much lower correlations resulted, revealing a loss of spatial color resolution due to the transmission of the video signal. The mean execution time of the ASA was 0.038 s for 0.91 m (3 ft) long inter-row regions showing that the algorithm met the real-time constraints necessary to be used as a sensing system for a variable-rate herbicide applicator.

SHAPE AND SIZE ANALYSIS OF CORN PLANT CANOPIES FOR PLANT POPULATION AND SPACING SENSING

An effective corn plant population and spacing sensing system may provide a key layer of field variability information useful for crop management. An algorithm was developed to count corn plants and to estimate plant location and intra-row spacing in segmented images of 6.1-m (20-ft) long row sections. Images were scanned to detect and determine the boundaries of top projected corn plant canopy objects using a chain code methodology. Plant objects were fused together based on a multi-step process that took into account the spatial structure of the crop row. Position, roundness, and area of plant canopies were used to distinguish between corn plants and weeds. Estimates of plant counts in row sections were compared with manual counts across three growth stages, three populations, and three tillage treatments. Overall, the system estimated the number of plants with an RMSE of 1.49 plants per row section, which corresponds to 6.2% RMSE or 3210 plants/ha (1300 plants/acre). No evidence of significant differences in mean plant spacing estimates was detected although significant, albeit small, increases in spacing variance were detected. These results demonstrate the importance of canopy shape and size analysis in the implementation of a machine vision plant population and intra-row spacing sensing system.

Distance-based control system for machine vision-based selective spraying

For effective operation of a selective sprayer with real–time local weed sensing, herbicides must be delivered accurately to weed targets in the field. With a machine vision–based selective spraying system, acquiring sequential images and switching nozzles on and off at the correct locations are critical. An MS Windows–based imaging system was interfaced with a real–time embedded selective spray controller system to accomplish control tasks based on distance traveled. A machine vision–based sensing system and selective herbicide control system was developed and installed on a sprayer. A finite state machine (FSM) model was employed for controller design, and general design specifications were developed for determining the travel distance between states. The spatial application accuracy of the system was measured in the field using artificial targets. The system operated with an overall hit accuracy of 91% with no statistical evidence of hit accuracy or mean pattern length being dependent on vehicle speed. Significant differences in pattern length variance and mean pattern width were detected across speed levels ranging from 3.2 to 14 km/h. Spray patterns tended to shift relative to the target at higher travel speeds.

Modeling and real-time simulation architectures for virtual prototyping of off-road vehicles

Virtual Reality-based simulation technology has evolved as a useful design and analysis tool at an early stage in the design for evaluating performance of human-operated agricultural and construction machinery. Detecting anomalies in the design prior to building physical prototypes and expensive testing leads to significant cost savings. The efficacy of such simulation technology depends on how realistically the simulation mimics the real-life operation of the machinery. It is therefore necessary to achieve ‘real-time’ dynamic simulation of such machines with operator-in-the-loop functionality. Such simulation often leads to intensive computational burdens. A distributed architecture was developed for off-road vehicle dynamic models and 3D graphics visualization to distribute the overall computational load of the system across multiple computational platforms. Multi-rate model simulation was also used to simulate various system dynamics with different integration time steps, so that the computational power can be distributed more intelligently. This architecture consisted of three major components: a dynamic model simulator, a virtual reality simulator for 3D graphics, and an interface to the controller and input hardware devices. Several off-road vehicle dynamics models were developed with varying degrees of fidelity, as well as automatic guidance controller models and a controller area network interface to embedded controllers and user input devices. The simulation architecture reduced the computational load to an individual machine and increased the real-time simulation capability with complex off-road vehicle system models and controllers. This architecture provides an environment to test virtual prototypes of the vehicle systems in real-time and the opportunity to test the functionality of newly developed controller software and hardware.

TOPOGRAPHIC MAPPING THROUGH MEASUREMENT OF VEHICLE ATTITUDE AND ELEVATION

A self-propelled agricultural sprayer was equipped with four RTK DGPS receivers and an inertial measurement unit (IMU) to measure vehicle attitude and field elevation as the vehicle was driven across a field. Data were collected in a stop-and-go fashion at 3.05 m (10 ft) intervals, as well as in a continuous fashion at three different speed levels on a 2.3 ha field area with varying topography. Pitch and roll offset angles were estimated to within 95% confidence intervals that ranged from 0.01° to 0.10°. Using ordinary kriging, digital elevation models (DEMs) were interpolated using only elevation measurements, as well as a combination of elevation and vehicle attitude measurements. The resulting DEMs were compared with each other to evaluate the effect of including attitude measurement on DEM accuracy. At the widest measurement swath width, the DEMs generated with attitude measurements had RMSE values of 10 to 11 cm, which was significantly lower than the RMSE of 15 cm associated with the DEMs generated without attitude measurements. Vehicle speed affected DEM error, but no discernable trends were detected. These results provide evidence supporting the feasibility of using vehicle-based measurements collected during typical field operation for accurate DEM development.

Automatic corn plant population measurement using machine vision

A machine vision–based corn plant population sensing system was developed to measure early growth stage corn population. Video was acquired from a vehicle–mounted digital video camera at V3 to V4 stages under different daylight conditions. Algorithms were developed to sequence video frames and to segment, singulate, and count corn plants. Vegetation segmentation was accomplished using a truncated ellipsoidal decision surface. Two features were extracted from each pixel row of the segmented images: total number of plant pixels, and their median position. Adjacent rows of the same class were grouped together and iteratively refined for final plant counting. Performance of this system was evaluated by comparing its estimation of plant counts with manual stand counts in 60 experimental units of 6.1 m sections of corn rows. The number of corn plants in these experimental units ranged from 14 to 48, corresponding to a population of 30,000 to 103,000 plants /ha. In low–weed field conditions, the system plant count was well correlated to manual stand count (R2 = 0.90). Standard error of population estimate was 1.8 plants over 33.2 mean manual plant count, or 5.4% coefficient of variation.