Heiner Kuhlmann

Low-Cost 3D Systems: Suitable Tools for Plant Phenotyping

Over the last few years, 3D imaging of plant geometry has become of significant importance for phenotyping and plant breeding. Several sensing techniques, like 3D reconstruction from multiple images and laser scanning, are the methods of choice in different research projects. The use of RGBcameras for 3D reconstruction requires a significant amount of post-processing, whereas in this context, laser scanning needs huge investment costs. The aim of the present study is a comparison between two current 3D imaging low-cost systems and a high precision close-up laser scanner as a reference method. As low-cost systems, the David laser scanning system and the Microsoft Kinect Device were used. The 3D measuring accuracy of both low-cost sensors was estimated based on the deviations of test specimens. Parameters extracted from the volumetric shape of sugar beet taproots, the leaves of sugar beets and the shape of wheat ears were evaluated. These parameters are compared regarding accuracy and correlation to reference measurements. The evaluation scenarios were chosen with respect to recorded plant parameters in current phenotyping projects. In the present study, low-cost 3D imaging devices have been shown to be highly reliable for the demands of plant phenotyping, with the potential to be implemented in automated application procedures, while saving acquisition costs. Our study confirms that a carefully selected low-cost sensor is able to replace an expensive laser scanner in many plant phenotyping scenarios.

Surface feature based classification of plant organs from 3D laserscanned point clouds for plant phenotyping.

BackgroundLaserscanning recently has become a powerful and common method for plant parameterization and plant growth observation on nearly every scale range. However, 3D measurements with high accuracy, spatial resolution and speed result in a multitude of points that require processing and analysis. The primary objective of this research has been to establish a reliable and fast technique for high throughput phenotyping using differentiation, segmentation and classification of single plants by a fully automated system. In this report, we introduce a technique for automated classification of point clouds of plants and present the applicability for plant parameterization.ResultsA surface feature histogram based approach from the field of robotics was adapted to close-up laserscans of plants. Local geometric point features describe class characteristics, which were used to distinguish among different plant organs. This approach has been proven and tested on several plant species. Grapevine stems and leaves were classified with an accuracy of up to 98%. The proposed method was successfully transferred to 3D-laserscans of wheat plants for yield estimation. Wheat ears were separated with an accuracy of 96% from other plant organs. Subsequently, the ear volume was calculated and correlated to the ear weight, the kernel weights and the number of kernels. Furthermore the impact of the data resolution was evaluated considering point to point distances between 0.3 and 4.0 mm with respect to the classification accuracy.ConclusionWe introduced an approach using surface feature histograms for automated plant organ parameterization. Highly reliable classification results of about 96% for the separation of grapevine and wheat organs have been obtained. This approach was found to be independent of the point to point distance and applicable to multiple plant species. Its reliability, flexibility and its high order of automation make this method well suited for the demands of high throughput phenotyping.Highlights• Automatic classification of plant organs using geometrical surface information• Transfer of analysis methods for low resolution point clouds to close-up laser measurements of plants• Analysis of 3D-data requirements for automated plant organ classification

Automated Analysis of Barley Organs Using 3D Laser Scanning: An Approach for High Throughput Phenotyping

Due to the rise of laser scanning the 3D geometry of plant architecture is easy to acquire. Nevertheless, an automated interpretation and, finally, the segmentation into functional groups are still difficult to achieve. Two barley plants were scanned in a time course, and the organs were separated by applying a histogram-based classification algorithm. The leaf organs were represented by meshing algorithms, while the stem organs were parameterized by a least-squares cylinder approximation. We introduced surface feature histograms with an accuracy of 96% for the separation of the barley organs, leaf and stem. This enables growth monitoring in a time course for barley plants. Its reliability was demonstrated by a comparison with manually fitted parameters with a correlation R2 = 0.99 for the leaf area and R2 = 0.98 for the cumulated stem height. A proof of concept has been given for its applicability for the detection of water stress in barley, where the extension growth of an irrigated and a non-irrigated plant has been monitored.

Real-Time Single-Frequency GPS/MEMS-IMU Attitude Determination of Lightweight UAVs

In this paper, a newly-developed direct georeferencing system for the guidance, navigation and control of lightweight unmanned aerial vehicles (UAVs), having a weight limit of 5 kg and a size limit of 1.5 m, and for UAV-based surveying and remote sensing applications is presented. The system is intended to provide highly accurate positions and attitudes (better than 5 cm and 0.5∘) in real time, using lightweight components. The main focus of this paper is on the attitude determination with the system. This attitude determination is based on an onboard single-frequency GPS baseline, MEMS (micro-electro-mechanical systems) inertial sensor readings, magnetic field observations and a 3D position measurement. All of this information is integrated in a sixteen-state error space Kalman filter. Special attention in the algorithm development is paid to the carrier phase ambiguity resolution of the single-frequency GPS baseline observations. We aim at a reliable and instantaneous ambiguity resolution, since the system is used in urban areas, where frequent losses of the GPS signal lock occur and the GPS measurement conditions are challenging. Flight tests and a comparison to a navigation-grade inertial navigation system illustrate the performance of the developed system in dynamic situations. Evaluations show that the accuracies of the system are 0.05∘ for the roll and the pitch angle and 0.2∘ for the yaw angle. The ambiguities of the single-frequency GPS baseline can be resolved instantaneously in more than 90% of the cases.

An automated field phenotyping pipeline for application in grapevine research.

Due to its perennial nature and size, the acquisition of phenotypic data in grapevine research is almost exclusively restricted to the field and done by visual estimation. This kind of evaluation procedure is limited by time, cost and the subjectivity of records. As a consequence, objectivity, automation and more precision of phenotypic data evaluation are needed to increase the number of samples, manage grapevine repositories, enable genetic research of new phenotypic traits and, therefore, increase the efficiency in plant research. In the present study, an automated field phenotyping pipeline was setup and applied in a plot of genetic resources. The application of the PHENObot allows image acquisition from at least 250 individual grapevines per hour directly in the field without user interaction. Data management is handled by a database (IMAGEdata). The automatic image analysis tool BIVcolor (Berries in Vineyards-color) permitted the collection of precise phenotypic data of two important fruit traits, berry size and color, within a large set of plants. The application of the PHENObot represents an automated tool for high-throughput sampling of image data in the field. The automated analysis of these images facilitates the generation of objective and precise phenotypic data on a larger scale.

Accuracy Analysis of a Multi-View Stereo Approach for Phenotyping of Tomato Plants at the Organ Level

Accessing a plants 3D geometry has become of significant importance for phenotyping during the last few years. Close-up laser scanning is an established method to acquire 3D plant shapes in real time with high detail, but it is stationary and has high investment costs. 3D reconstruction from images using structure from motion (SfM) and multi-view stereo (MVS) is a flexible cost-effective method, but requires post-processing procedures. The aim of this study is to evaluate the potential measuring accuracy of an SfM- and MVS-based photogrammetric method for the task of organ-level plant phenotyping. For this, reference data are provided by a high-accuracy close-up laser scanner. Using both methods, point clouds of several tomato plants were reconstructed at six following days. The parameters leaf area, main stem height and convex hull of the complete plant were extracted from the 3D point clouds and compared to the reference data regarding accuracy and correlation. These parameters were chosen regarding the demands of current phenotyping scenarios. The study shows that the photogrammetric approach is highly suitable for the presented monitoring scenario, yielding high correlations to the reference measurements. This cost-effective 3D reconstruction method depicts an alternative to an expensive laser scanner in the studied scenarios with potential for automated procedures.

Generation and application of hyperspectral 3D plant models: methods and challenges

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they also have been used for close-range sensing of plant canopies with a highly complex architecture. However, the complex geometry of plants and their interaction with the illumination setting severely affect the spectral information obtained. Furthermore, the spatial component of analysis results gain in importance as higher plants are represented by multiple plant organs as leaves, stems and seed pods. The combination of hyperspectral images and 3D point clouds is a promising approach to face these problems. We present the generation and application of hyperspectral 3D plant models as a new, interesting application field for computer vision with a variety of challenging tasks. We sum up a geometric calibration method for hyperspectral pushbroom cameras using a reference object for the combination of spectral and spatial information. Furthermore, we show exemplarily new calibration and analysis methods enabled by the hyperspectral 3D models in an experiment with sugar beet plants. An improved normalization, a comparison of image and 3D analysis and the density estimation of infected surface points underline some of the new capabilities gained using this new data type. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified and modeled. In future, reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential to improve automated plant phenotyping significantly.

Improved area-based deformation analysis of a radio telescope’s main reflector based on terrestrial laser scanning

Abstract The main reflectors of radio telescopes deform due to gravitation when changing their elevation angle. This can be analyzed by scanning the paraboloid surface with a terrestrial laser scanner and by determining focal length variations and local deformations from best-fit approximations. For the Effelsberg radio telescope, both groups of deformations are estimated from seven points clouds measured at different elevation angles of the telescope: the focal length decreases by 22.7 mm when tilting the telescope from 90 deg to 7.5 deg elevation angle. Variable deformations of ± 2 mm are detected as well at certain areas. Furthermore, a few surface panels seem to be misaligned. Apart from these results, the present study highlights the need for an appropriate measurement concept and for preprocessing stepswhen using laser scanners for area-based deformation analyses. Especially, data reduction, object segmentation and laser scanner calibration are discussed in more detail. An omission of these steps would significantly degrade the deformation analysis and the significance of its results. This holds for all sorts of laser scanner based analyses.

Challenges and Present Fields of Action at Laser Scanner Based Deformation Analyses

Abstract Due to improved laser scanning technology, laser scanner based deformation analyses are presently widespread. These deformation analyses are no longer based on individual points representing the deformation of an object at selected positions. Instead, they are based on a large number of scan points sampling the whole object. This fact either leads to challenges regarding metrological aspects as well as regarding modeling aspects: – Estimating and quantifying spatial correlations between scan points and incorporating them into the deformation analysis – Separating the laser scanners’ internal systematic errors from areal deformations – Minimizing the bias at areal deformation analyses due to a worse network configuration and limited object knowledge – Developing freeform parameterizations to reproduce arbitrary areal deformations of an object by individual parameters – Incorporating an extended uncertainty model considering also model errors due to imperfect knowledge and simplification of the sampled object. – Only when considering all of these aspects, laser scanner based deformation analyses can benefit from the potential of the areal object sampling. This study aims at naming and reasoning these aspects. Furthermore, it introduces first methodologies and approaches for dealing with them.

Limits of Active Laser Triangulation as an Instrument for High Precision Plant Imaging

Laser scanning is a non-invasive method for collecting and parameterizing 3D data of well reflecting objects. These systems have been used for 3D imaging of plant growth and structure analysis. A prerequisite is that the recorded signals originate from the true plant surface. In this paper we studied the effects of species, leaf chlorophyll content and sensor settings on the suitability and accuracy of a commercial 660 nm active laser triangulation scanning device. We found that surface images of Ficus benjamina leaves were inaccurate at low chlorophyll concentrations and a long sensor exposure time. Imaging of the rough waxy leaf surface of leek (Allium porrum) was possible using very low exposure times, whereas at higher exposure times penetration and multiple refraction prevented the correct imaging of the surface. A comparison of scans with varying exposure time enabled the target-oriented analysis to identify chlorotic, necrotic and healthy leaf areas or mildew infestations. We found plant properties and sensor settings to have a strong influence on the accuracy of measurements. These interactions have to be further elucidated before laser imaging of plants is possible with the high accuracy required for e.g., the observation of plant growth or reactions to water stress.