Jan Behmann

Low-Cost 3D Systems: Suitable Tools for Plant Phenotyping

Over the last few years, 3D imaging of plant geometry has become of significant importance for phenotyping and plant breeding. Several sensing techniques, like 3D reconstruction from multiple images and laser scanning, are the methods of choice in different research projects. The use of RGBcameras for 3D reconstruction requires a significant amount of post-processing, whereas in this context, laser scanning needs huge investment costs. The aim of the present study is a comparison between two current 3D imaging low-cost systems and a high precision close-up laser scanner as a reference method. As low-cost systems, the David laser scanning system and the Microsoft Kinect Device were used. The 3D measuring accuracy of both low-cost sensors was estimated based on the deviations of test specimens. Parameters extracted from the volumetric shape of sugar beet taproots, the leaves of sugar beets and the shape of wheat ears were evaluated. These parameters are compared regarding accuracy and correlation to reference measurements. The evaluation scenarios were chosen with respect to recorded plant parameters in current phenotyping projects. In the present study, low-cost 3D imaging devices have been shown to be highly reliable for the demands of plant phenotyping, with the potential to be implemented in automated application procedures, while saving acquisition costs. Our study confirms that a carefully selected low-cost sensor is able to replace an expensive laser scanner in many plant phenotyping scenarios.

Early drought stress detection in cereals: simplex volume maximisation for hyperspectral image analysis

Early water stress recognition is of great relevance in precision plant breeding and production. Hyperspectral imaging sensors can be a valuable tool for early stress detection with high spatio-temporal resolution. They gather large, high dimensional data cubes posing a significant challenge to data analysis. Classical supervised learning algorithms often fail in applied plant sciences due to their need of labelled datasets, which are difficult to obtain. Therefore, new approaches for unsupervised learning of relevant patterns are needed. We apply for the first time a recent matrix factorisation technique, simplex volume maximisation (SiVM), to hyperspectral data. It is an unsupervised classification approach, optimised for fast computation of massive datasets. It allows calculation of how similar each spectrum is to observed typical spectra. This provides the means to express how likely it is that one plant is suffering from stress. The method was tested for drought stress, applied to potted barley plants in a controlled rain-out shelter experiment and to agricultural corn plots subjected to a two factorial field setup altering water and nutrient availability. Both experiments were conducted on the canopy level. SiVM was significantly better than using a combination of established vegetation indices. In the corn plots, SiVM clearly separated the different treatments, even though the effects on leaf and canopy traits were subtle.

A review of advanced machine learning methods for the detection of biotic stress in precision crop protection

Effective crop protection requires early and accurate detection of biotic stress. In recent years, remarkable results have been achieved in the early detection of weeds, plant diseases and insect pests in crops. These achievements are related both to the development of non-invasive, high resolution optical sensors and data analysis methods that are able to cope with the resolution, size and complexity of the signals from these sensors. Several methods of machine learning have been utilized for precision agriculture such as support vector machines and neural networks for classification (supervised learning); k-means and self-organizing maps for clustering (unsupervised learning). These methods are able to calculate both linear and non-linear models, require few statistical assumptions and adapt flexibly to a wide range of data characteristics. Successful applications include the early detection of plant diseases based on spectral features and weed detection based on shape descriptors with supervised or unsupervised learning methods. This review gives a short introduction into machine learning, analyses its potential for precision crop protection and provides an overview of instructive examples from different fields of precision agriculture.

Generation and application of hyperspectral 3D plant models: methods and challenges

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they also have been used for close-range sensing of plant canopies with a highly complex architecture. However, the complex geometry of plants and their interaction with the illumination setting severely affect the spectral information obtained. Furthermore, the spatial component of analysis results gain in importance as higher plants are represented by multiple plant organs as leaves, stems and seed pods. The combination of hyperspectral images and 3D point clouds is a promising approach to face these problems. We present the generation and application of hyperspectral 3D plant models as a new, interesting application field for computer vision with a variety of challenging tasks. We sum up a geometric calibration method for hyperspectral pushbroom cameras using a reference object for the combination of spectral and spatial information. Furthermore, we show exemplarily new calibration and analysis methods enabled by the hyperspectral 3D models in an experiment with sugar beet plants. An improved normalization, a comparison of image and 3D analysis and the density estimation of infected surface points underline some of the new capabilities gained using this new data type. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified and modeled. In future, reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential to improve automated plant phenotyping significantly.

A Multi-Resolution Approach for an Automated Fusion of Different Low-Cost 3D Sensors

The 3D acquisition of object structures has become a common technique in many fields of work, e.g., industrial quality management, cultural heritage or crime scene documentation. The requirements on the measuring devices are versatile, because spacious scenes have to be imaged with a high level of detail for selected objects. Thus, the used measuring systems are expensive and require an experienced operator. With the rise of low-cost 3D imaging systems, their integration into the digital documentation process is possible. However, common low-cost sensors have the limitation of a trade-off between range and accuracy, providing either a low resolution of single objects or a limited imaging field. Therefore, the use of multiple sensors is desirable. We show the combined use of two low-cost sensors, the Microsoft Kinect and the David laserscanning system, to achieve low-resolved scans of the whole scene and a high level of detail for selected objects, respectively. Afterwards, the high-resolved David objects are automatically assigned to their corresponding Kinect object by the use of surface feature histograms and SVM-classification. The corresponding objects are fitted using an ICP-implementation to produce a multi-resolution map. The applicability is shown for a fictional crime scene and the reconstruction of a ballistic trajectory.

Specim IQ: Evaluation of a New, Miniaturized Handheld Hyperspectral Camera and Its Application for Plant Phenotyping and Disease Detection

Hyperspectral imaging sensors are promising tools for monitoring crop plants or vegetation in different environments. Information on physiology, architecture or biochemistry of plants can be assessed non-invasively and on different scales. For instance, hyperspectral sensors are implemented for stress detection in plant phenotyping processes or in precision agriculture. Up to date, a variety of non-imaging and imaging hyperspectral sensors is available. The measuring process and the handling of most of these sensors is rather complex. Thus, during the last years the demand for sensors with easy user operability arose. The present study introduces the novel hyperspectral camera Specim IQ from Specim (Oulu, Finland). The Specim IQ is a handheld push broom system with integrated operating system and controls. Basic data handling and data analysis processes, such as pre-processing and classification routines are implemented within the camera software. This study provides an introduction into the measurement pipeline of the Specim IQ as well as a radiometric performance comparison with a well-established hyperspectral imager. Case studies for the detection of powdery mildew on barley at the canopy scale and the spectral characterization of Arabidopsis thaliana mutants grown under stressed and non-stressed conditions are presented.

Generation and application of hyperspectral 3D plant models

Hyperspectral imaging sensors have been introduced for measuring the health status of plants. Recently, they have been also used for close-range sensing of plant canopies with a more complex architecture. The complex geometry of plants and their interaction with the illumination scenario severely affect the spectral information obtained. The combination of hyperspectral images and 3D point clouds are a promising approach to face this problem. Based on such hyperspectral 3D models the effects of plant geometry and sensor configuration can be quantified an modeled. Reflectance models can be used to remove or weaken the geometry-related effects in hyperspectral images and, therefore, have the potential potential to improve automated phenotyping significantly.

Support Vector machine and duration-aware conditional random field for identification of spatio-temporal activity patterns by combined indoor positioning and heart rate sensors

Tracking the spatio-temporal activity is highly relevant for domains like security, health, and quality management. Since animal welfare became a topic in politics and legislation locomotion patterns of livestock have received increasing interest. In contrast to the monitoring of pedestrians cattle activity tracking poses special challenges to both sensors and data analysis. Interesting states are not directly observable by a single sensor. In addition, sensors must be accepted by cattle and need to be robust enough to cope with a rough environment. In this article, we introduce the novel combination of heart rate and positioning sensors. Attached to neck and chest they are less interfering than accelerometers at the ankles. Exploiting the potential of such combined sensor system that records locomotion and non-spatial information from the heart rate sensor however is challenging. We introduce a novel two level method for the activity tracking focused on the duration and sequence of activity states. We combine Support Vector Machine (SVM) with Conditional Random Field (CRF) and extend Conditional Random fields by an explicit representation of duration. The SVM characterizes local activity states, whereas the CRF addresses sequences of local states to sequences incorporating spatial and non-spatial contextual knowledge. This combination provides a reliable and comprehensive identification of defined activity patterns, as well as their chronology and durations, suitable for the integration in an activity data base. This data base is used to extract physiological parameters and promises insights into internal states such as fitness, well-being and stress. Interestingly we were able to demonstrate a significant correlation between resting pulse rate and the day of pregnancy.

Potential of hyperspectral imaging to detect and identify the impact of chemical warfare compounds on plant tissue

Abstract The OPCW Member states cover 98% of the global population and landmass. Regrettably, unanticipated chemical warfare agent assaults are reported during the last decades. In addition to the frequent threat situation, the sampling of bio-medical samples from these areas is critical and mainly depends on investigation opportunities of victims. Non-contact sensor technologies are desirable to enable a fast and secure estimation of a situation. Plants react on pollution because of their direct interaction with gases and it is assumed that chemical warfare agents influence plants, respectively. This impact can be analyzed for the detection and characterization of chemical warfare assaults. Nowadays technological progress in digital technologies provides new innovations in detectors, data analysis approaches and software availability which could improve the screening, monitoring and analysis of chemical warfare. Within this context hyperspectral imaging (HSI) is a promising method. Different applications from remote to close range sensing in medicine, food production, military, geography and agriculture do exist already. During the last years HSI showed high potential to determine and assess different plant parameters, e.g. abiotic and biotic stresses by recording the spectral reflectance of plants. Within the present manuscript, the basics principle of HSI as an innovative technique, aspects of recording and analyzing HSI data is presented using wild growing apple leaves which are treated with sulfuric acid, fire or heat. Resulting spectral signatures showed significant changes among the treatments. Especially the shortwave infrared was sensitive to changes due to the different treatments. Furthermore, the calculation of common spectral indices revealed differences due to the treatments which are not visible to the human eye. The results support HSI applications for the detection of chemical warfare agents and elucidate the impact of chemical warfare on plants.

Screening of barley resistance against powdery mildew by simultaneous high-throughput enzyme activity signature profiling and multispectral imaging

Molecular marker analysis allow for a rapid and advanced pre-selection and resistance screenings in plant breeding processes. During the phenotyping process, optical sensors have proved their potential to determine and assess the function of the genotype of the breeding material. Thereby, biomarkers for specific disease resistance traits provide valuable information for calibrating optical sensor approaches during early plant-pathogen interactions. In this context, the combination of physiological, metabolic phenotyping and phenomic profiles could establish efficient identification and quantification of relevant genotypes within breeding processes. Experiments were conducted with near-isogenic lines of H. vulgare (susceptible, mildew locus o (mlo) and Mildew locus a (Mla) resistant). Multispectral imaging of barley plants was daily conducted 0–8 days after inoculation (dai) in a high-throughput facility with 10 wavelength bands from 400 to 1,000 nm. In parallel, the temporal dynamics of the activities of invertase isoenzymes, as key sink specific enzymes that irreversibly cleave the transport sugar sucrose into the hexose monomers, were profiled in a semi high-throughput approach. The activities of cell wall, cytosolic and vacuole invertase revealed specific dynamics of the activity signatures for susceptible genotypes and genotypes with mlo and Mla based resistances 0–120 hours after inoculation (hai). These patterns could be used to differentiate between interaction types and revealed an early influence of Blumeria graminis f.sp. hordei (Bgh) conidia on the specific invertase activity already 0.5 hai. During this early powdery mildew pathogenesis, the reflectance intensity increased in the blue bands and at 690 nm. The Mla resistant plants showed an increased reflectance at 680 and 710 nm and a decreased reflectance in the near infrared bands from 3 dai. Applying a Support Vector Machine classification as a supervised machine learning approach, the pixelwise identification and quantification of powdery mildew diseased barley tissue and hypersensitive response spots were established. This enables an automatic identification of the barley-powdery mildew interaction. The study established a proof-of-concept for plant resistance phenotyping with multispectral imaging in high-throughput. The combination of invertase analysis and multispectral imaging showed to be a complementing validation system. This will provide a deeper understanding of optical data and its implementation into disease resistance screening.