Dimitris S. Paraforos

3-D Imaging Systems for Agricultural Applications—A Review

Efficiency increase of resources through automation of agriculture requires more information about the production process, as well as process and machinery status. Sensors are necessary for monitoring the status and condition of production by recognizing the surrounding structures such as objects, field structures, natural or artificial markers, and obstacles. Currently, three dimensional (3-D) sensors are economically affordable and technologically advanced to a great extent, so a breakthrough is already possible if enough research projects are commercialized. The aim of this review paper is to investigate the state-of-the-art of 3-D vision systems in agriculture, and the role and value that only 3-D data can have to provide information about environmental structures based on the recent progress in optical 3-D sensors. The structure of this research consists of an overview of the different optical 3-D vision techniques, based on the basic principles. Afterwards, their application in agriculture are reviewed. The main focus lays on vehicle navigation, and crop and animal husbandry. The depth dimension brought by 3-D sensors provides key information that greatly facilitates the implementation of automation and robotics in agriculture.

3D Maize Plant Reconstruction Based on Georeferenced Overlapping LiDAR Point Clouds

3D crop reconstruction with a high temporal resolution and by the use of non-destructive measuring technologies can support the automation of plant phenotyping processes. Thereby, the availability of such 3D data can give valuable information about the plant development and the interaction of the plant genotype with the environment. This article presents a new methodology for georeferenced 3D reconstruction of maize plant structure. For this purpose a total station, an IMU, and several 2D LiDARs with different orientations were mounted on an autonomous vehicle. By the multistep methodology presented, based on the application of the ICP algorithm for point cloud fusion, it was possible to perform the georeferenced point clouds overlapping. The overlapping point cloud algorithm showed that the aerial points (corresponding mainly to plant parts) were reduced to 1.5%–9% of the total registered data. The remaining were redundant or ground points. Through the inclusion of different LiDAR point of views of the scene, a more realistic representation of the surrounding is obtained by the incorporation of new useful information but also of noise. The use of georeferenced 3D maize plant reconstruction at different growth stages, combined with the total station accuracy could be highly useful when performing precision agriculture at the crop plant level.

Total station data assessment using an industrial robotic arm for dynamic 3D in-field positioning with sub-centimetre accuracy

AB lines, U-Turn, and Pattern-8 experiments were performed.As the speed increased so did the relative XTE.Changing the position of the TS from inline to perpendicular gave a better accuracy.The maximum mean horizontal XTE value was 4.01mm for Pattern-8 experiment.The vertical relative XTE did not exceed 10mm including the outliers. For agricultural tasks related to precision farming, accurate in-field positioning is a necessity. The accuracy of some centimetres that the real time kinematic-global navigation satellite system (RTK-GNSS) can provide is adequate for many applications, such as auto-steering navigation and section control for spraying or fertiliser applications. Nevertheless, the demand for higher in-field accuracy at a mm level is increasing. A device that is gaining a lot of attention in the agricultural sector for its increased accuracy is a robotic total station (TS) that can track a prism mounted on a vehicle. With the aim to be able to use this device under realistic conditions for dynamic 3D in-field positioning at a sub-centimetre level, the accuracy of the TS was assessed utilising an industrial robotic arm. The robotic arm had a repeatability factor of 0.1mm and was placed outdoors under normal environmental conditions for agriculture practice. Straight AB lines but also U-turn and Pattern-8 experiments were performed. The absolute error of the robotic arm had a maximum mean value of 0.33mm for the Pattern-8 experiment, while the highest error, equal to 1.30mm, was detected in the 95th percentile of the same experiment. The horizontal and vertical relative cross-track error (XTE) between the TS and the robotic arm data was calculated for various speeds and for two different positions of the TS. From the results, it was evident that as the speed increased so did the horizontal relative XTE. Furthermore, changing the position of the TS from in line to perpendicular, in respect to the direction of motion, proved to result in a higher accuracy. The maximum mean horizontal relative XTE value of all experiments was 4.01mm for Pattern-8, which also had the maximum value for the 95th percentile, i.e. 12.86mm. The vertical relative XTE for all experiments did not exceed 10mm including the outliers.

3-D reconstruction of maize plants using a time-of-flight camera

Abstract Point cloud rigid registration and stitching for plants with complex architecture is a challenging task, however, it is an important process to take advantage of the full potential of 3-D cameras for plant phenotyping and agricultural automation for characterizing production environments in agriculture. A methodology for three-dimensional (3-D) reconstruction of maize crop rows was proposed in this research, using high resolution 3-D images that were mapped into the colour images using state-of-the art software. The point cloud registration methodology was based on the Iterative Closest Point (ICP) algorithm. The incoming point cloud was previously filtered using the Random Sample Consensus (RANSAC) algorithm, by reducing the number of soil points until a threshold value was reached. This threshold value was calculated based on the approximate number of plant points in a single 3-D image. After registration and stitching of the crop rows, a plant/soil segmentation process was done relying again on the RANSAC algorithm. A quantitative comparison showed that the number of points obtained with a time-of-flight (TOF) camera, compared with the ones from two light detection and ranging (LIDARs) from a previous research, was roughly 23 times larger. Finally, the reconstruction was validated by comparing the seedling positions as ground truth and the point cloud clusters, obtained using the k- mean s clustering, that represent the plant stem positions. The resulted maize positions from the proposed methodology closely agreed with the ground truth with an average mean and standard deviation of 3.4 cm and ±1.3 cm, respectively.

Machine operation profiles generated from ISO 11783 communication data

Abstract An operation profile is a detailed description of machinery use and provides information about the production process (e.g. time and energy requirements). The high complexity of agricultural machine use over time with its various implements and production processes makes it difficult to automate the operation profile generation for agricultural machinery. Today, modern communication interfaces, like the ISO 11783 (ISOBUS), allow a comfortable data acquisition with comprehensive parameter information of the machine and production process status. This paper describes how the generation of operation profiles can be automated and what kind of machine data and status information are required. Experiments were conducted with a tractor-machine combination during field cultivation. The machine was equipped with a CAN data logger to record ISOBUS messages and GNSS position data during normal operation. A software tool was setup and programmed to drive the data logger. Fields were automatically detected by algorithms to avoid the time consuming manual definition of field boundaries. A graphical user interface simplified the setup and use of the software application. Furthermore, the algorithms analyzed the machine communication regarding the actual machine state based on position, machine activity and work state. The KTBL time classification 2013 was used as a basis for working time analysis as it is designed to divide the overall operational time of agricultural machines into clearly defined, qualitatively different partial time fractions. The machine state could be visualized in the software by a map to show the user where problems occurred e.g. in time losses while waiting or machine breakdown. A diagram presented the timespans spent in different time fractions for comparison to other, similar operations. Combined with other infrastructure data, specific operation profiles could be used to gain more detailed knowledge about machine use or to improve future operations.

Automating the process of importing data into an FMIS using information from tractor’s CAN-Bus communication

This paper is focusing on how to eliminate the required time for importing all the necessary data into a farm management information system (FMIS). This process was automated by using ISO 11783 and SAE J1939 communication information from the tractor’s CAN-Bus. Using a data logger and a machine to machine (M2M) gateway inside the tractor’s cabin, CAN-Bus data were recorded and transmitted to the cloud-based server of the FMIS. There, a script was responsible for parsing and aggregating the raw machine data into specific agricultural tasks and then importing them into the FMIS. The operator could choose the type of the performed task by a number of switches connected with the digital inputs of the data logger.

Crop Row Detection in Maize for Developing Navigation Algorithms Under Changing Plant Growth Stages

To develop robust algorithms for agricultural navigation, different growth stages of the plants have to be considered. For fast validation and repeatable testing of algorithms, a dataset was recorded by a 4 wheeled robot, equipped with a frame of different sensors and was guided through maize rows. The robot position was simultaneously tracked by a total station, to get precise reference of the sensor data. The plant position and parameters were measured for comparing the sensor values. A horizontal laser scanner and corresponding total station data was recorded for 7 times over a period of 6 weeks. It was used to check the performance of a common RANSAC row algorithm. Results showed the best heading detection at a mean growth height of 0.268 m.

Determination of stem position and height of reconstructed maize plants using a time-of-flight camera

Abstract Three dimensional (3-D) reconstruction of maize plant morphology by proximal sensing in agriculture brings high definition data that can be used for a number of applications related with precision agriculture and agricultural robotics. However, 3-D reconstruction without methodologies for extracting useful information is a senseless strategy. In this research, a methodology for stem position estimation is presented relying on the merging of four point clouds, using the Iterative Closes Point algorithm, that were generated from different 3-D perspective views. The proposed methodology is based on bivariate point density histograms for detecting the regional maxima and a radius filter based on the closest Euclidean distance. Then, single plant segmentation was performed by projecting a spatial cylindrical boundary around the estimated stem positions on a merged plant and soil point cloud. After performing a local Random Sample Consensus, the segmented plant point cloud was clustered using the Density-based spatial clustering of applications with noise algorithm. Additionally, a height profile was generated by rasterizing the plant and soil point clouds, separately, with different cell widths. The rasterized soil point cloud was meshed, and the rasterized plant points to soil mesh distance was calculated. The resulting plant stem positions were estimated with an average mean error and standard deviation of 24 mm and 14 mm, respectively. Equivalently, the average mean error and standard deviation of the individual plant height estimation was 30 mm and 35 mm, respectively. Finally, the overall plant height profile mean error average was 8.7 mm. Thus it is possible to determine the stem position and plant height of reconstructed maize plants using a low-cost time-of-flight camera.

Clustering of Laser Scanner Perception Points of Maize Plants

The goal of this work was to cluster maize plants perception points under six different growth stages in noisy 3D point clouds with known positions. The 3D point clouds were assembled with a 2D laser scanner mounted at the front of a mobile robot, fusing the data with the precise robot position, gained by a total station and an Inertial Measurement Unit. For clustering the single plants in the resulting point cloud, a graph-cut based algorithm was used. The algorithm results were compared with the corresponding measured values of plant height and stem position. An accuracy for the estimated height of 1.55 cm and the stem position of 2.05 cm was achieved.

Modelling and simulation of a no-till seeder vertical motion dynamics for precise seeding depth

One of the significant obstacles in achieving a reliable seed germination and even plant field emergence in no-till seeding is a variation in the desired seeding depth. This is caused by the inappropriate response of the seeder motion dynamics to harsh soil conditions and to high operating speed. In order to assess the dynamic response of a no-till seeder, a mathematical model, which simulated the vertical motion of a seeding aggregate, was developed. A correlation between the simulated and the measured parameters resulted in a root-mean-squared (RMS) error of 17.2% and 6.4% for impact force and pitch angle, respectively. The simulated impact force frequencies of interests were detected at the critical frequencies of the measured forces with high coherence values.