Yiyun Chen

Comparison of multivariate methods for estimating soil total nitrogen with visible/near-infrared spectroscopy

AimsThis study aimed to compare stepwise multiple linear regression (SMLR), partial least squares regression (PLSR) and support vector machine regression (SVMR) for estimating soil total nitrogen (TN) contents with laboratory visible/near-infrared reflectance (Vis/NIR) of selected coarse and heterogeneous soils. Moreover, the effects of the first (1st) vs. second (2nd) derivative of spectral reflectance and the importance wavelengths were explored.MethodsThe TN contents and the Vis/NIR were measured in the laboratory. Several methods were employed for Vis/NIR data pre-processing. The SMLR, PLSR and SVMR models were calibrated and validated using independent datasets.ResultsResults showed that the SVMR and the PLSR models had similar performances, and better performances than the SMLR. The spectral bands near 1450, 1850, 2250, 2330 and 2430xa0nm in the PLSR model were important wavelengths. In addition, the 1st derivative was more appropriate than the 2nd derivative for spectral data pre-processing.ConclusionsPLSR was the most suitable method for estimating TN contents in this study. SVMR may be a promising technique, and its potential needs to be further explored. Moreover, the future studies using outdoor and airborne/satellite hyperspectral data for estimating TN content are necessary for testing the findings.

Monitoring arsenic contamination in agricultural soils with reflectance spectroscopy of rice plants.

The objective of this study was to explore the feasibility and to investigate the mechanism for rapidly monitoring arsenic (As) contamination in agricultural soils with the reflectance spectra of rice plants. Several data pretreatment methods were applied to improve the prediction accuracy. The prediction of soil As contents was achieved by partial least-squares regression (PLSR) using laboratory and field spectra of rice plants, as well as linear regression employing normalized difference spectral index (NDSI) calculated from fild spectra. For laboratory spectra, the optimal PLSR model for predicting soil As contents was achieved using Savitzky-Golay smoothing (SG), first derivative and mean center (MC) (root-mean-square error of prediction (RMSEP)=14.7 mg kg(-1); r=0.64; residual predictive deviation (RPD)=1.31). For field spectra, the optimal PLSR model was also achieved using SG, first derivative and MC (RMSEP=13.7 mg kg(-1); r=0.71; RPD=1.43). In addition, the NDSI with 812 and 782 nm obtained a prediction accuracy with r=0.68, RMSEP=13.7 mg kg(-1), and RPD=1.36. These results indicated that it was feasible to monitor the As contamination in agricultural soils using the reflectance spectra of rice plants. The prediction mechanism might be the relationship between the As contents in soils and the chlorophyll-a/-b contents and cell structure in leaves or canopies of rice plants.

Estimation of arsenic in agricultural soils using hyperspectral vegetation indices of rice

This study systematically analyzed the performance of multivariate hyperspectral vegetation indices of rice (Oryza sativa L.) in estimating the arsenic content in agricultural soils. Field canopy reflectance spectra was obtained in the jointing-booting growth stage of rice. Newly developed and published multivariate vegetation indices were initially calculated to estimate soil arsenic content. The well-performing vegetation indices were then selected using successive projections algorithm (SPA), and the SPA selected vegetation indices were adopted to calibrate a multiple linear regression model for estimating soil arsenic content. Results showed that a three-band vegetation index (R716-R568)/(R552-R568) performed best in the newly developed vegetation indices in estimating soil arsenic content. The photochemical reflectance index (PRI) and red edge position (REP) performed well in the published vegetation indices. Moreover, the linear combination of two vegetation indices ((R716-R568)/(R552-R568) and REP) selected using SPA improved the estimation of soil arsenic content. These results indicated that the newly developed three-band vegetation index (R716-R568)/(R552-R568) might be recommended as an indicator for estimating soil arsenic content in the study area. PRI and REP could be used as universal vegetation indices for monitoring soil arsenic contamination.

Improving the prediction of arsenic contents in agricultural soils by combining the reflectance spectroscopy of soils and rice plants

Abstract Visible and near-infrared reflectance spectroscopy provides a beneficial tool for investigating soil heavy metal contamination. This study aimed to investigate mechanisms of soil arsenic prediction using laboratory based soil and leaf spectra, compare the prediction of arsenic content using soil spectra with that using rice plant spectra, and determine whether the combination of both could improve the prediction of soil arsenic content. A total of 100 samples were collected and the reflectance spectra of soils and rice plants were measured using a FieldSpec3 portable spectroradiometer (350–2500xa0nm). After eliminating spectral outliers, the reflectance spectra were divided into calibration ( n xa0=xa062) and validation ( n xa0=xa032) data sets using the Kennard-Stone algorithm. Genetic algorithm (GA) was used to select useful spectral variables for soil arsenic prediction. Thereafter, the GA-selected spectral variables of the soil and leaf spectra were individually and jointly employed to calibrate the partial least squares regression (PLSR) models using the calibration data set. The regression models were validated and compared using independent validation data set. Furthermore, the correlation coefficients of soil arsenic against soil organic matter, leaf arsenic and leaf chlorophyll were calculated, and the important wavelengths for PLSR modeling were extracted. Results showed that arsenic prediction using the leaf spectra (coefficient of determination in validation, R v 2 xa0=xa00.54; root mean square error in validation, RMSE v xa0=xa012.99xa0mgxa0kg −1 ; and residual prediction deviation in validation, RPD v xa0=xa01.35) was slightly better than using the soil spectra ( R v 2 xa0=xa00.42, RMSE v xa0=xa013.35xa0mgxa0kg −1 , and RPD v xa0=xa01.31). However, results also showed that the combinational use of soil and leaf spectra resulted in higher arsenic prediction ( R v 2 xa0=xa00.63, RMSE v xa0=xa011.94xa0mgxa0kg −1 , RPD v xa0=xa01.47) compared with either soil or leaf spectra alone. Soil spectral bands near 480, 600, 670, 810, 1980, 2050 and 2290xa0nm, leaf spectral bands near 700, 890 and 900xa0nm in PLSR models were important wavelengths for soil arsenic prediction. Moreover, soil arsenic showed significantly positive correlations with soil organic matter ( r xa0=xa00.62, p r xa0=xa00.77, p r xa0=xa0−0.67, p RPD of 1.47 was below the recommended RPD of >2 for soil analysis, arsenic prediction in agricultural soils can be improved by combining the leaf and soil spectra.

Estimating Soil Organic Carbon of Cropland Soil at Different Levels of Soil Moisture Using VIS-NIR Spectroscopy

Soil organic carbon (SOC) is an essential property for soil function, fertility and sustainability of agricultural systems. It can be measured with visible and near-infrared reflectance (VIS-NIR) spectroscopy efficiently based on empirical equations and spectra data for air/oven-dried samples. However, the spectral signal is interfered with by soil moisture content (MC) under in situ conditions, which will affect the accuracy of measurements and calibration transfer among different areas. This study aimed to (1) quantify the influences of MC on SOC prediction by VIS-NIR spectroscopy; and (2) explore the potentials of orthogonal signal correction (OSC) and generalized least squares weighting (GLSW) methods in the removal of moisture interference. Ninety-eight samples were collected from the Jianghan plain, China, and eight MCs were obtained for each sample by a rewetting process. The VIS-NIR spectra of the rewetted soil samples were measured in the laboratory. Partial least squares regression (PLSR) was used to develop SOC prediction models. Specifically, three validation strategies, namely moisture level validation, transferability validation and mixed-moisture validation, were designed to test the potentials of OSC and GLSW in removing the MC effect. Results showed that all of the PLSR models generated at different moisture levels (e.g., 50–100, 250–300 g·kg−1) were moderately successful in SOC predictions (r2pre = 0.58–0.85, RPD = 1.55–2.55). These models, however, could not be transferred to soil samples with different moisture levels. OSC and GLSW methods are useful filter transformations improving model transferability. The GLSW-PLSR model (mean of r2pre = 0.77, root mean square error for prediction (RMSEP) = 3.08 g·kg−1, and residual prediction deviations (RPD) = 2.09) outperforms the OSC-PLSR model (mean of r2pre = 0.67, RMSEP = 3.67 g·kg−1, and RPD = 1.76) when the moisture-mixed protocol is used. Results demonstrated the use of OSC and GLSW combined with PLSR models for efficient estimation of SOC using VIS-NIR under different soil MC conditions.

Prediction of total nitrogen in cropland soil at different levels of soil moisture with Vis/NIR spectroscopy

Visible/near-infrared (Vis/NIR) spectroscopy has been proven to be an effective technique for soil total nitrogen (TN) content estimation in the laboratory conditions. However, the transferability of this technique from laboratory study to field application is complicated by soil moisture effects. This study aims to compare the performance of four spectral transformation strategies, namely, Savitzky–Golay (SG) smoothing, SG smoothing followed by first derivative (FD), orthogonal signal correction (OSC), and generalized least squares weighting (GLSW), in the removal of soil moisture effects on TN estimation. The spectral transformations were applied on 8 sets of spectral reflectance measured from 62 soil samples at 8 moisture levels. The air-dried set was used for partial least squares regression (PLSR) calibration, whereas the other seven sets with moisture gradients were used for external validations. Results show that the SG-PLSR model cannot be transferred from the air-dried samples to the samples with moisture gradients. The FD-PLSR model showed fair TN prediction performance, with five out of seven residual prediction deviations (RPD) that are greater than 1.4. Both OSC-PLSR and GLSW-PLSR had good transferability to the moist samples. More specifically, the GLSW-PLSR model (mean of , root mean square error for prediction [RMSEP] = 0.262, and RPD = 1.885) outperformed the OSC-PLSR model (mean of , RMSEP = 0.277, and RPD = 1.780). The results demonstrate the value of OSC and GLSW in eliminating the effects of moisture on TN estimation, and the GLSW-PLSR is recommended for a better Vis/NIR estimation of TN content under different soil moisture conditions.

Improving Spectral Estimation of Soil Organic Carbon Content through Semi-Supervised Regression

Visible and near infrared (VIS-NIR) spectroscopy has been applied to estimate soil organic carbon (SOC) content with many modeling strategies and techniques, in which a crucial and challenging problem is to obtain accurate estimations using a limited number of samples with reference values (labeled samples). To solve such a challenging problem, this study, with Honghu City (Hubei Province, China) as a study area, aimed to apply semi-supervised regression (SSR) to estimate SOC contents from VIS-NIR spectroscopy. A total of 252 soil samples were collected in four field campaigns for laboratory-based SOC content determinations and spectral measurements. Semi-supervised regression with co-training based on least squares support vector machine regression (Co-LSSVMR) was applied for spectral estimations of SOC contents, and it was further compared with LSSVMR. Results showed that Co-LSSVMR could improve the estimations of SOC contents by exploiting samples without reference values (unlabeled samples) when the number of labeled samples was not excessively small and produce better estimations than LSSVMR. Therefore, SSR could reduce the number of labeled samples required in calibration given an accuracy threshold, and it holds advantages in SOC estimations from VIS-NIR spectroscopy with a limited number of labeled samples. Considering the increasing popularity of airborne platforms and sensors, SSR might be a promising modeling technique for SOC estimations from remotely sensed hyperspectral images.

Proximal and remote sensing techniques for mapping of soil contamination with heavy metals

ABSTRACT Heavy metal soil contamination is a severe environmental problem globally, and its mapping is vital for environmental managers and policymakers to determine its distributions and hotspots. This paper reviewed multiple proximal and remote sensing spectroscopy for convenient and inexpensive method of obtaining soil reflectance spectroscopy or environmental covariates, which can be used for mapping heavy metal soil contamination. Furthermore, spatial prediction using proximal remote-sensed data and environmental covariates was discussed. We suggested that mapping of the spatial distributions of metal species may be important due to the different bioavailabilities and toxicities of various species. The assimilation of multiple proximal/remote-sensed sensors may promote the horizontal and vertical mapping of soil heavy metals. Moreover, combining the advantages of satellite and unmanned aerial vehicle-based hyperspectral imaging systems will facilitate the development of a space–aeronautic incorporation hyperspectral observation technology that can monitor soil environment rapidly and accurately at a large scale.

Prediction of Soil Organic Matter by VIS–NIR Spectroscopy Using Normalized Soil Moisture Index as a Proxy of Soil Moisture

Soil organic matter (SOM) is an important parameter of soil fertility, and visible and near-infrared (VIS–NIR) spectroscopy combined with multivariate modeling techniques have provided new possibilities to estimate SOM. However, the spectral signal is strongly influenced by soil moisture (SM) in the field. Interest in using spectral classification to predict soils in the moist conditions to minimize the influence of SM is growing. The objective of this study was to investigate the transferability of two approaches, SM–based cluster method with known SM (classifying the VIS–NIR spectra into different SM clusters to develop models separately), the normalized soil moisture index (NSMI)–based cluster method with unknown SM (utilizing NSMI to indicate the SM and establish models separately), to predict SOM directly in moist soil spectra. One hundred and twenty one soil samples were collected from Central China, and eight SM levels were obtained for each sample through rewetting experiments. Their reflectance spectra and SOM concentrations were measured in the laboratory. Partial least square-support vector machine (PLS-SVM) was employed to construct SOM prediction models. Specifically, prediction models were developed for NSMI–based clusters with unknown SM data. The models were assessed through three statistics in the processes of calibration and validation: the coefficient of determination (R2), root mean square error (RMSE) and the ratio of the performance to deviation (RPD). Results showed that the variable SM led to reduced VIS–NIR reflectance nonlinearly across the entire spectral range. NSMI was an effective spectral index to indicate the SM. Classifying the VIS–NIR spectra into different SM clusters in known SM states could improve the performance of PLS-SVM models to acceptable prediction accuracies (R2cv = 0.69–0.77, RPD = 1.79–2.08). The estimation of SOM, when using the NSMI–based cluster method with unknown SM (RPD = 1.95–2.04), was similar to the use of the SM–based cluster method with known SM (RPD = 1.79–2.08). The predictive results (RPD = 1.87–2.06) demonstrated that the NSMI-–based cluster method has potential for application outside the laboratory for SOM prediction without knowing the SM explicitly, and this method is also easy to carry out and only requires spectral information.

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

Visible and near-infrared (VIS–NIR) spectroscopy has been extensively applied to estimate soil organic matter (SOM) in the laboratory. However, if field/moist VIS–NIR spectra can be directly applied to estimate SOM, then much of the time and labor would be avoided. Spectral derivative plays an important role in eliminating unwanted interference and optimizing the estimation model. Nonetheless, the conventional integer order derivatives (i.e., the first and second derivatives) may neglect some detailed information related to SOM. Besides, the full-spectrum generally contains redundant spectral variables, which would affect the model accuracy. This study aimed to investigate different combinations of fractional order derivative (FOD) and spectral variable selection techniques (i.e., competitive adaptive reweighted sampling (CARS), elastic net (ENET) and genetic algorithm (GA)) to optimize the VIS–NIR spectral model of moist soil. Ninety-one soil samples were collected from Central China, with their SOM contents and reflectance spectra measured. Support vector machine (SVM) was applied to estimate SOM. Results indicated that moist spectra differed greatly from dried ground spectra. With increasing order of derivative, the spectral resolution improved gradually, but the spectral strength decreased simultaneously. FOD could provide a better tool to counterbalance the contradiction between spectral resolution and spectral strength. In full-spectrum SVM models, the most accurate estimation was achieved by SVM model based on 1.5-order derivative spectra, with validation R2 = 0.79 and ratio of the performance to deviation (RPD) = 2.20. Of all models studied (different combinations of FOD and variable selection techniques), the highest validation model accuracy for SOM was achieved when applying 1.5 derivative spectra and GA method (validation R2 = 0.88 and RPD = 2.89). Among the three variable selection techniques, overall, the GA method yielded the optimal predictability. However, due to its long computation time, one alternative was to use CARS method. The results of this study confirm that a suitable combination of FOD and variable selection can effectively improve the model performance of SOM in moist soil.