Lie Tang

Coverage path planning on three-dimensional terrain for arable farming

Field operations should be done in a manner that minimizes time and travels over the field surface and is coordinated with topographic land features. Automated path planning can help to find the best coverage path so that the field operation costs can be minimized. Intelligent algorithms are desired for both two-dimensional (2D) and three-dimensional (3D) terrain field coverage path planning. The algorithm of generating an optimized full coverage pattern for a given 2D planar field by using boustrophedon paths has been investigated and reported before. However, a great proportion of farms have rolling terrains, which have a considerable influence on the design of coverage paths. Coverage path planning in 3D space has a great potential to further optimize field operations. This work addressed four critical tasks: terrain modeling and representation, coverage cost analysis, terrain decomposition, and the development of optimized path searching algorithm. The developed algorithms and methods have been successfully implemented and tested using 3D terrain maps of farm fields with various topographic features. Each field was decomposed into subregions based on its terrain features. A recommended “seed curve” based on a customized cost function was searched for each subregion, and parallel coverage paths were generated by offsetting the found “seed curve” toward its two sides until the whole region was completely covered. Compared with the 2D planning results, the experimental results of 3D coverage path planning showed its superiority in reducing both headland turning cost and soil erosion cost. On the tested fields, on average the 3D planning algorithm saved 10.3% on headland turning cost, 24.7% on soil erosion cost, 81.2% on skipped area cost, and 22.0% on the weighted sum of these costs, where their corresponding weights were 1, 1, and 0.5, respectively.

REAL-TIME CROP ROW IMAGE RECONSTRUCTION FOR AUTOMATIC EMERGED CORN PLANT SPACING MEASUREMENT

In-field variations in corn plant spacing and population can lead to significant yield differences. To minimize these variations, seeds should be placed at a uniform spacing during planting. Since the ability to achieve this uniformity is directly related to planter performance, intensive field evaluations are vitally important prior to design of new planters and currently the designers have to rely on manually collected data that is very time consuming and subject to human errors. A machine vision-based emerged crop sensing system (ECSS) was developed to automate corn plant spacing measurement at early growth stages for planter design and testing engineers. This article documents the first part of the ECSS development, which was the real-time video frame mosaicking for crop row image reconstruction. Specifically, the mosaicking algorithm was based on a normalized correlation measure and was optimized to reduce the computational time and enhance the frame connection accuracy. This mosaicking algorithm was capable of reconstructing crop row images in real-time while the sampling platform was traveling at a velocity up to 1.21 m s-1 (2.73 mph). The mosaicking accuracy of the ECSS was evaluated over three 40 to 50 m long crop rows. The ECSS achieved a mean distance measurement error ratio of -0.11% with a standard deviation of 0.74%.

Plant Identification in Mosaicked Crop Row Images for Automatic Emerged Corn Plant Spacing Measurement

Image processing algorithms for individual corn plant and plant stem center identification were developed. These algorithms were applied to mosaicked crop row image for automatically measuring corn plant spacing at early growth stages. These algorithms utilized multiple sources of information for corn plant detection and plant center location estimation including plant color, plant morphological features, and the crop row centerline. The algorithm was tested over two 41 m (134.5 ft) long corn rows using video acquired two times in both directions. The system had a mean plant misidentification ratio of 3.7%. When compared with manual plant spacing measurements, the system achieved an overall spacing error (RMSE) of 1.7 cm and an overall R2 of 0.96 between manual plant spacing measurement and the system estimates. The developed image processing algorithms were effective in automated corn plant spacing measurement at early growth stages. Interplant spacing errors were mainly due to crop damage and sampling platform vibration that caused mosaicking errors.

Optimal Path Planning for Arable Farming

With the rapid adoption of auto-steering systems, automated path planning has a great potential to further optimize field operations. Field operations should be done in a manner that minimizes time, travels over the field surface and are coordinated with specific field operations and topological features of arable lands. Intelligent algorithms are called for in this area. By experience, boustrophedon paths are always the best choice for farm field coverage. To determine the full coverage pattern of a given field by using boustrophedon paths, we need to know how to decompose the field into sub-regions and how to determine the travel direction within each sub-region. In this paper, an innovative geometric model was built to represent this coverage path planning problem. Then a path planning algorithm was developed based on this proposed geometric model. The algorithm has been proved to be capable of finding a globally optimal decomposition for a given field and the direction of the boustrophedon paths for each sub-region. The search mechanism of the algorithm is guided by a customized cost function that unifies different cost criteria and a divide-and-conquer strategy is adopted. The complexity of the algorithm is 1 1 ( log( )) m m O n n + + for a field with n edges and m obstacles. Several techniques for reducing the computational time were also developed. Field examples with complexity ranges from a simple convex shape to an irregular polygonal shape that has multiple obstacles with its interior were tested with this algorithm. The results have depicted that the proposed algorithm is effective in producing a globally optimal field decomposition and path direction in each sub-region.

Utilizing repeated gps surveys from field operations for development of agricultural field dems.

Topographic data collected using RTK-DGPS-equipped farm vehicles during field operations could addadditional benefits to the original capital investment in the equipment through the development of high-accuracy field DEMs. Repeated surveys of elevation data from field operations may improve DEM accuracy over time. However, minimizing the amount of data to be processed and stored is also an important goal for practical implementation. A method was developed to utilize repeated GPS surveys acquired during field operations for generating field-level DEMs. Elevation measurement error was corrected through a continuity analysis. Fuzzy logic (FL) and weighted averaging (WA) methods were used to combine new surveys with past elevation estimates without requiring storage and reprocessing of past survey data. After 20 surveys were included, the DEM of the study area generated with FL and WA methods had an average root mean squared error (RMSE) of 0.08 m, which was substantially lower than the RMSE of 0.16 m associated with the DEM developed by averaging all data points in each grid. With minimum control of errors in elevation measurements, the effect of these errors can be reduced with appropriate data processing, including continuity analysis, fuzzy logic, and weighted averaging. Two years of GPS surveys of elevation data from field operations could reduce elevation error by 50% in field DEMs.

Inter-plant Spacing Sensing at Early Growth Stages Using a Time-of-Flight of Light Based 3D Vision Sensor

Uniform plant spacing is always desired for equal distribution of water and nutrients among plants. Researchers in the past have shown that variations in plant spacing result in significant variation in final crop yields. Planter manufacturers and researchers have been working closely to develop computer vision-based automatic interplant spacing sensing systems. Current systems mostly utilize top-view images using a stereo rig, or a video camera. These systems are highly sensitive to color variations introduced by shadow formations and glares and have difficulties when plant canopies start occluding. We developed an interplant spacing sensing system using a time-of-flight (TOF) based 3D vision sensor. The camera was capable of capturing depth and intensity data with one single shot. The depth images captured from the side were stitched together using distance information from a wheel encoder in conjunction with a feature-based image sequencing process. Multiple layers of image data were used for stem location identification. The use of depth images made the plant identification less sensitive to color variations. A covered vehicle was designed to prevent the sunlight from directly shedding on the plants and to reduce the interference from wind, which in turn made the system usable throughout the day. The vertical camera position was easily adjustable to work with different growth stages of the crops. The use of side-view images made the system capable to detecting inclined plants and therefore, boosted the performance of the system in precisely locating the stem centers, and thereby minimized the measurement errors. Based on the initial trials on corn plants of growth stages V3-V6, the system has achieved 100% plant identification accuracy with a RSME of 0.15 cm for inter-plant spacing measurements.

Developing a low-cost 3D plant morphological traits characterization system

Abstract A low-cost three-dimensional (3D) plant reconstruction and morphological traits characterization system was developed. Corn plant seedlings were used as research objects for development and validation of the 3D reconstruction and point cloud data analysis algorithms. In this application, precise alignment of multiple 3D views generated by a 3D time-of-flight (ToF) sensor is critical to the 3D reconstruction of a plant. Previous research indicated that there is strong need for high-throughput, high-accuracy, and low-cost 3D plant reconstruction and trait characterization phenotyping systems. This research produced a 3D reconstruction system for indoor plant phenotyping by innovatively integrating a low-cost 2D camera, a low-cost 3D ToF camera, and a chessboard pattern beacon array to track the position and attitude of the 3D ToF sensor and, thus, accomplished precise 3D point cloud registration over multiple views. Specifically, algorithms for beacon target detection, camera pose tracking, and spatial relationship calibration between 2D and 3D cameras were developed for such a low-cost but high-performance 3D reconstruction solution. A plant analysis algorithm in a 3D space was developed to extract the morphological trait parameters of the plants by analyzing their 3D point cloud data. The phenotypical data obtained by this novel and low-cost 3D reconstruction based phenotyping system were validated by the experimental data generated by instrument and manual measurement. The results demonstrated that the developed phenotyping system has achieved promising measurement accuracy, fast processing speed while offering a low hardware cost, lending itself to a practical means of acquiring detailed 3D morphological traits for automated indoor plant phenotyping.

Optimal Coverage Path Planning on 3D Terrain

Automated path planning is important for the optimization of field operations. Field operations should be done in a manner that minimizes time, travels over the field surface and are coordinated with specific field operations and topographic land features. Intelligent algorithms are desired for both 2D and 3D terrain field coverage path planning. The algorithm of generating an optimized full coverage pattern for a given 2D planar field by using boustrophedon paths has been investigated and reported before. However, in real world, a great proportion of farms have rolling terrains, which have a considerable influence to the design of coverage paths. Coverage path planning in 3D space has a great potential to further optimize field operations. To achieve this goal, following four research tasks are critical: terrain modeling and representation; coverage cost analysis; terrain decomposition; and the development of optimal path searching algorithm. This work addressed these four tasks and the developed algorithms and methods have been successfully implemented and tested using 3D terrain maps of farm fields with various topographic features. Each field was decomposed into sub-regions based on its terrain features. A recommended “seed curve” based on a customized cost function was searched for each sub-region and parallel coverage paths were generated by offsetting the found “seed curve” sideways until the whole region was completely covered. Compared with the 2D planning results, the experimental results of 3D coverage path planning showed its superiority in reducing both headland turning cost and soil erosion cost. On the tested fields, on average the 3D planning algorithm saved 10.3% on headland turning cost, 24.7% on soil erosion cost, 81.2% on skipped area cost, and 22.0% on the weighted sum of these costs, where their corresponding weights were 1, 1, 0.5, respectively.

Field-based robotic phenotyping of sorghum plant architecture using stereo vision: BAO et al.

Funding information National Institute of Food and Agriculture, Grant/Award Number: 2012‐67009‐19713; United States Department of Agriculture Abstract Sorghum (Sorghum bicolor) is known as a major feedstock for biofuel production. To improve its biomass yield through genetic research, manually measuring yield component traits (e.g. plant height, stem diameter, leaf angle, leaf area, leaf number, and panicle size) in the field is the current best practice. However, such laborious and time‐consuming tasks have become a bottleneck limiting experiment scale and data acquisition frequency. This paper presents a high‐throughput field‐based robotic phenotyping system which performed side‐view stereo imaging for dense sorghum plants with a wide range of plant heights throughout the growing season. Our study demonstrated the suitability of stereo vision for field‐based three‐dimensional plant phenotyping when recent advances in stereo matching algorithms were incorporated. A robust data processing pipeline was developed to quantify the variations or morphological traits in plant architecture, which included plot‐based plant height, plot‐based plant width, convex hull volume, plant surface area, and stem diameter (semiautomated). These image‐derived measurements were highly repeatable and showed high correlations with the in‐field manual measurements. Meanwhile, manually collecting the same traits required a large amount of manpower and time compared to the robotic system. The results demonstrated that the proposed system could be a promising tool for large‐scale field‐based high‐throughput plant phenotyping of bioenergy crops.

Crop recognition under weedy conditions based on 3D imaging for robotic weed control

A 3D time-of-flight camera was applied to develop a crop plant recognition system for broccoli and green bean plants under weedy conditions. The developed system overcame the previously unsolved problems caused by occluded canopy and illumination variation. An efficient noise filter was developed to remove the sparse noise points in 3D point cloud space. Both 2D and 3D features including the gradient of amplitude and depth image, surface curvature, amplitude percentile index, normal direction, and neighbor point count in 3D space were extracted and found effective for recognizing these two types of plants. Separate segmentation algorithms were developed for each of the broccoli and green bean plant in accordance with their 3D geometry and 2D amplitude characteristics. Under the experimental condition where the crops were heavily infested by various types of weed plants, detection rates over 88.3% and 91.2% were achieved for broccoli and green bean plant leaves, respectively. Additionally, the crop plants were segmented out with nearly complete shape. Moreover, the algorithms were computationally optimized, resulting in an image processing speed of over 30 frames per second.