Filling a key gap: a soil infrared library for central Africa

Visible and near infrared diffuse reflectance (vis-NIR) spectroscopy is a low-cost, efficient and accurate soil analysis technique and is thus becoming increasingly popular. Soil spectral libraries are commonly constructed as the basis for estimating soil texture and properties. In this study, partial least squares regression was used to develop models to predict the soil organic carbon (SOC) content of 35 soil samples from one field using (i) the Danish soil spectral library (2688 samples), (ii) a spiked spectral library (a combination of 30 samples selected from the local area and the spectral library, 2718 samples) and (iii) three sub-sets selected from the spectral library. In an attempt to improve prediction accuracy, sub-sets of the soil spectral library were made using three different sample selection methods: those geographically closest (84 samples), those with the same landscape and parent material (96 samples) and those with the most alike spectra to spectra from the field investigation (100 samples). These sub-sets were used to develop three calibration models and in predictions of SOC content. The results showed that the geographically closest model, which used the fewest number of samples, gave the lowest root mean square error of prediction ( RMSEP) of 0.19% and the highest ratio of performance to deviation ( RPD) of 3.7, followed by the spiked library, same parent material, the spectral library and the most alike spectra. The spiked library model also gave a low RMSEP value of 0.19% and high RPD value of 3.7% and performed markedly better than the model without spiking, despite using 30 samples for library spiking. The accuracy of the model developed using a sub-set from a spectral library was highly dependent on geographical location, soil parent material and landscape.

<p>Soil spectroscopy in the mid-infrared (MIR) allows the fast and cost-effective derivation of multiple physical and chemical soil properties, e.g., soil organic carbon (SOC) and soil texture, from a single reflectance spectrum. The recent development of extensive soil spectral libraries and field-portable handheld FTIR spectrometers have opened up new opportunities for the widespread application of soil reflectance spectroscopy in the geo- and environmental sciences. Compared to laboratory measurements on pre-treated soil material, field recordings of MIR spectra are impacted by in situ environmental conditions that modify and degrade the measured reflectance signal, most prominently variations in soil moisture and particle size across samples. These conditions prevent leveraging available MIR soil spectral libraries to build predictive models of soil properties directly.</p><p>We evaluated the capacity of the External Parameter Orthogonalization (EPO) algorithm to compensate for moisture and particle size-induced effects on MIR reflectance spectra recorded in the field to generate laboratory-equivalent spectra from the in-situ data, which would allow calibrations of predictive soil property models from soil spectral library data to be transferred to field-recorded spectra. An archive of 230 soils collected across five soil regions in Germany covering a broad range of parent materials, soil texture classes and organic carbon contents was used to evaluate the approach. For each soil sample, MIR reflectance spectra had been acquired both in the field, i.e., measured in situ on the soil surface, and in the laboratory on pre-treated (sieved and ground) soil material. Field spectra were corrected for environmental effects by EPO and used to predict SOC and soil texture with predictive models developed on the laboratory spectra.</p><p>Analysis of the EPO-transformed spectra showed that the algorithm could compensate for some of the significant environmental effects present in the field data, e.g., non-linear baseline shifts and large-scale water absorption features, effectively reducing variation across the soil samples that is not linked to the physical and chemical soil properties of interest. EPO-transformation of the spectra further allowed a robust transfer of calibrations developed on laboratory spectra of pre-treated soils to the field spectra. Predictive accuracies for SOC and soil texture were lower than for pure laboratory applications but generally in line with models developed with an extensive regional calibration sample directly on the field MIR spectra.</p><p>The correction of field MIR spectra with the EPO algorithm thus represents a promising approach to integrating existing soil spectral libraries into the development of predictive soil property models for in-situ MIR reflectance spectra as it would allow the development of predictive models without requiring a large number of additional regional calibration samples for field application of MIR soil spectroscopy.</p>

Soil spectroscopy with machine learning (ML) can estimate soil properties. Extensive soil spectral libraries (SSLs) have been developed for this purpose. However, general models built with those SSLs do not generalize well on new ‘unseen’ local data. The main reason is the different characteristics of the observations in the SSL and the local data, which cause their conditional and marginal distributions to differ. This makes the modelling of soil properties with spectra challenging. General models developed using large ‘global’ SSLs offer broad, systematic information on the soil-spectra relationships. However, to accurately generalize in a local situation, they must be adjusted to capture the site-specific characteristics of the local observations. Most current methods for ‘localizing’ spectroscopic modelling report inconsistent results. An understanding of spectroscopic ‘localization’ is lacking, and there is no framework to guide further developments. Here, we review current localization methods and propose their reformulation as a transfer learning (TL) undertaking. We then demonstrate the implementation of instance-based TL with rs-local 2.0 for modelling the soil organic carbon (SOC) content of 12 sites representing fields, farms and regions from 10 countries on the seven continents. The method uses a small number of instances or observations (measured soil property values and corresponding spectra) from the local site to transfer relevant information from a large and diverse global SSL (GSSL 2.0) with more than 50,000 records. We found that with ≤30 local observations, rs-local 2.0 produces more accurate and stable estimates of SOC than modelling with only the local data. Using the information in the GSSL 2.0 and reducing the number of samples for laboratory analysis, the method improves the cost-efficiency and practicality of soil spectroscopy. We interpreted the transfer by analysing the data, models, and soil and environmental relationships of the local and the ‘transferred’ data to gain insight into the approach. Transferring instances from the GSSL 2.0 to the local sites helped to align their conditional and marginal distributions, making the spectra-SOC relationships in the models more robust. Finally, we propose directions for future research. The guiding principle for developing practical and cost-effective spectroscopy should be to think globally but fit locally. By reformulating the localization problem within a TL framework, we hope to have acquainted the soil science community with a set of methodologies that can inspire the development of new, innovative algorithms for soil spectroscopic modelling.

Soil reflectance is a cumulative attribute determined by interactions between light (photons) and the physical, chemical, and biological properties of soil. Soil properties such as organic matter, moisture, mineral oxide contents, soil texture, and surface roughness all influence soil reflectance at unique wavelengths. Many standard sample preparation techniques are designed to alter soil properties, so as to homogenize samples to improve the consistency of reflectance data collected. This study aims to quantify the effects of one standard sample preparation activity, drying, and repeated wetting–drying cycles on visible–near-infrared soil reflectance, by collecting reflectance data from nine samples which were dried then wetted for a total of three times. This study demonstrates the major, permanent effects of a drying and wetting cycle on soil reflectance, and then presents a model which can be used to correct for these effects. These results have direct implications for remote sensing activities, soil libraries, and soil spectral libraries. These results show that without correction, data from soil spectral libraries, spectral data collected from stored samples, and spectral data transferred between studies, have limited utility for characterizing soils as they exist in the field. Soil spectral data collected in the lab requires correction before it may be used to predict soil properties in the field, as standard sample handling procedures change intrinsic soil properties and introduce systematic error into these data.

Infrared spectroscopy in the visible to near‐infrared (vis–NIR) and mid‐infrared (MIR) regions is a well‐established approach for the prediction of soil properties. Different data fusion and training approaches exist, and the optimal procedures are yet undefined and may depend on the heterogeneity present in the set and on the considered scale. The objectives were to test the usefulness of partial least squares regressions (PLSRs) for soil organic carbon (SOC), total carbon (C t ), total nitrogen (N t ) and pH using vis–NIR and MIR spectroscopy for an independent validation after standard calibration (use of a general PLSR model) or using memory‐based learning (MBL) with and without spiking for a national spectral database. Data fusion approaches were simple concatenation of spectra, outer product analysis (OPA) and model averaging. In total, 481 soils from an Austrian forest soil archive were measured in the vis–NIR and MIR regions, and regressions were calculated. Fivefold calibration‐validation approaches were carried out with a region‐related split of spectra to implement independent validations with n ranging from 47 to 99 soils in different folds. MIR predictions were generally superior over vis–NIR predictions. For all properties, optimal predictions were obtained with data fusion, with OPA and spectra concatenation outperforming model averaging. The greatest robustness of performance was found for OPA and MBL with spiking with R 2 ≥ 0.77 (N), 0.85 (SOC), 0.86 (pH) and 0.88 (C t ) in the validations of all folds. Overall, the results indicate that the combination of OPA for vis–NIR and MIR spectra with MBL and spiking has a high potential to accurately estimate properties when using large‐scale soil spectral libraries as reference data. However, the reduction of cost‐effectiveness using two spectrometers needs to be weighed against the potential increase in accuracy compared to a single MIR spectroscopy approach.

<p>The degradation of European soils is a cause for concern. Examples are the reduction of carbon content and soil fertility. The European Commission therefore recommends further research on how to better monitor soils and their changes over time and space. Digital soil mapping (DSM) is already an established method for the use of hyperspectral information from soil samples for quantifying soil properties under laboratory conditions based on soil spectral libraries. At the remote sensing level, imaging spectroscopy has already achieved good results for the prediction of soil properties on a local scale. Major advantages of this method are that topsoil maps can be updated more frequently, spatially more accurately and with less costs.</p><p>In this study we bring together pedometric and remote sensing approaches to achieve the development of soil spectral models applicable to upcoming global hyperspectral imagery, combining DSM methods and data with Earth Observation expertise. In a first step at laboratory level, we used the EU-wide topsoil database LUCAS. We investigated whether using solely spectral data (without any covariates) and selected classification algorithms combined with PLSR could allow and improve the quantification of soil organic carbon (SOC) content. The best results were obtained for the local PLSR approach with RMSE=5.16 g kg-1, RPD=1.74 and R²=0.67. In addition, the local PLSR approach was tested with LUCAS spectral data resampled to EnMAP satellite spectral resolution, resulting in a very similar SOC prediction model accuracy.</p><p>In the next step, the local PLSR approach was applied to airborne HySpex image data and simulated satellite EnMAP data from a test area in north-eastern Germany where local soil data are available for model validation. This area is associated with one LUCAS point. A direct application of the laboratory-based SOC model to the spectra of the airborne image was not possible due to higher variability in the image data caused by different environmental conditions (solar illumination, mixed soil-vegetation pixels, surface state -roughness, wetness-) and sensor performances different from the laboratory data resulting in an overall lower signal-to-noise ratio in the airborne image. Therefore, after reducing the effect of soil moisture, green vegetation cover, residues coverage, we used a two-step approach where (i) wet chemistry SOC analyses for a set of soil samples from the test area were replaced by the local PLSR approach using the LUCAS database. Then (ii) an airborne model was calibrated using the SOC content from (i) and the corresponding image spectra to calibrate an airborne PLSR. Preliminary results show a good airborne model accuracy for HySpex imagery with RMSE=3.33 g kg-1, RPD=1.59, R²=0.63 and slightly lower but still good accuracy for simulated EnMAP imagery with RMSE=3.72 g kg-1, RPD=1.45, R²=0.55. Both models were then applied to the images to produce SOC maps for bare soils and validated with existing data and previous SOC mapping works in the area based on local datasets. This approach demonstrates the possibility to replace wet chemistry by the local PLSR approach based on large scale soil spectral libraries for SOC mapping.</p>

The influence of vegetation cover on the spectroscopic estimation of soil properties

Soil surveys and mapping using conventional methods are time consuming and costly, particularly for large areas, while the demand for detailed soil data continues to grow for applications in land management and precision agriculture. In this study, a hybrid Random Forest Kriging (RFK) model combining the machine learning capabilities of Random Forest (RF) with the spatial prediction strength of the Kriging geostatistical model was employed alongside three other machine learning algorithms: Artificial Neural Networks (ANN), Gradient Boosting Machines (GBM), and K-Nearest Neighbor (KNN). These models were utilized to predict the spatial distribution of five soil physicochemical properties: potential of Hydrogen (pH), Electrical Conductivity (EC), Calcium Carbonate (CaCO₃), Bulk Density (BD), and Available Water Capacity (AWC) in a region located in Central Anatolia, Turkey. The physicochemical properties of 119 soil samples collected from the study area were analyzed. One of the objectives of this study was to evaluate the effectiveness of low-scale terrain variables as predictors for soil property estimation. For this purpose, three auxiliary terrain variables digital elevation model (DEM), slope, and topographic roughness index (TRI) were derived from remote sensing data and analyzed using GIS techniques. The predicted maps were evaluated utilizing three statistical metrics: Concordance Correlation Coefficient (CCC), root mean square error (RMSE), and mean absolute error (MAE). The results showed that the hybrid RFK model outperformed other machine learning models in predicting soil properties, achieving the best CCC values and the lowest RMSE and MAE, while the performance of the other models remained relatively similar and weak. Furthermore, the findings indicated that the hybrid model's ability to delineate the spatial patterns of soil properties was primarily influenced by elevation and slope, despite the moderate effectiveness of low-scale terrain variables as auxiliary inputs in digital soil mapping.

Mapping soil properties, using geostatistical methods in support of precision agriculture and related activities, requires a large number of samples. To reduce soil sampling and measurement time and cost, a combination of field proximal soil sensors was used to predict and map laboratory-measured soil properties in a 3.4-ha pasture field in southeastern Brazil. Sensor soil properties were measured in situ on a 10 × 10-m dense grid (377 samples) using apparent electrical conductivity meters, apparent magnetic susceptibility meter, gamma-ray spectrometer, water content reflectometer, cone penetrometer, and portable X-ray fluorescence spectrometer (pXRF). Soil samples were collected on a 20 × 20-m thin grid (105 samples) and analyzed in the laboratory for organic C, sum of bases, cation exchange capacity, clay content, soil volumetric moisture, and bulk density. Another 25 samples collected throughout the area were also analyzed for the same soil properties and used for independent validation of models and maps. To test whether the combination of sensors enhances soil property predictions, stepwise multiple linear regression (MLR) models of the laboratory soil properties were derived using individual sensor covariate data versus combined sensor data—except for the pXRF data, which were evaluated separately. Then, to test whether a denser grid sample boosted by sensor-based soil property predictions enhances soil property maps, ordinary kriging of the laboratory-measured soil properties from the thin grid was compared to ordinary kriging of the sensor-based predictions from the dense grid, and ordinary cokriging of the laboratory properties aided by sensor covariate data. The combination of multiple soil sensors improved the MLR predictions for all soil properties relative to single sensors. The pXRF data produced the best MLR predictions for organic C content, clay content, and bulk density, standing out as the best single sensor for soil property prediction, whereas the other sensors combined outperformed the pXRF sensor for the sum of bases, cation exchange capacity, and soil volumetric moisture, based on independent validation. Ordinary kriging of sensor-based predictions outperformed the other interpolation approaches for all soil properties, except organic C content, based on validation results. Thus, combining soil sensors, and using sensor-based soil property predictions to increase the sample size and spatial coverage, leads to more detailed and accurate soil property maps.

As classical soil analysis is time-consuming and expensive, there is a growing demand for visible, near-infrared, and short-wave infrared (Vis-NIR-SWIR, wavelength 350–2500 nm) spectroscopy to predict soil properties. The objectives of this study were to investigate the effects of soil bunds on key soil properties and to develop regression models based on the Vis-NIR-SWIR spectral reflectance of soils in Aba Gerima, Ethiopia. Soil samples were collected from the 0–30 cm soil layer in 48 experimental teff (Eragrostis tef) plots and analysed for soil texture, pH, organic carbon (OC), total nitrogen (TN), available phosphorus (av. P), and potassium (av. K). We measured reflectance from air-dried, ground, and sieved soils with a FieldSpec 4 Spectroradiometer. We used regression models to identify and predict soil properties, as assessed by the coefficient of determination (R2), root mean square error (RMSE), bias, and ratio of performance to deviation (RPD). The results showed high variability (CV ≥ 35%) and substantial variation (P < 0.05 to P < 0.001) in soil texture, OC, and av. P in the catchment. Soil reflectance was lower from bunded plots. The pre-processing techniques, including multiplicative scatter correction, median filter, and Gaussian filter for OC, clay, and sand, respectively were used to transform the soil reflectance. Statistical results were: R2 = 0.71, RPD = 8.13 and bias = 0.12 for OC; R2 = 0.93, RPD = 2.21, bias = 0.94 for clay; and R2 = 0.85 with RPD = 7.54 and bias = 0.0.31 for sand with validation dataset. However, care is essential before applying the models to other regions. In conclusion, the findings of this study suggest spectroradiometry can supplement classical soil analysis. However, more research is needed to increase the prediction performance of Vis-NIR-SWIR reflectance spectroscopy to advance soil management interventions.

To spike or to localize? Strategies to improve the prediction of local soil properties using regional spectral library

&amp;lt;p&amp;gt;Spectral measurements are increasingly used to predict soil properties. Libraries of soil spectra are built and statistical models are used to relate the spectra to wet chemistry measurements. These relationships can then be used to predict the properties of new samples. An important&amp;amp;#160; consideration is the uncertainty associated with the prediction. Often to reduce this error calibration is done at field level. This is time and resource intensive, however, and there is scope to use existing spectral libraries. Our aim was to quantify the uncertainty in the prediction of soil properties from spectral measurements using a local library and compare this to predictions made using a regional library. &amp;amp;#160;&amp;amp;#160;&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;To investigate this, we considered two case study fields in the Cambridgeshire fens (UK) that were planted with lettuce. These fields contain complex soils which are a combination of peat with underlying alluvial and marine silts that became elevated features in the landscape due to peat oxidation and shrinkage. These elevated features are captured by a 2 m x 2 m LiDAR raster used in our study (UK Environment Agency). We took a total 467 soil samples across the fields and made spectral measurements (near- and mid-infrared). A subset of the soil samples underwent wet chemistry analysis for available pH, P, K, total N and soil particle size fraction. For the regional library we use soil the National Soil Inventory spectral database and its respective wet chemistry reference values.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;We used partial least squares to regress the soil spectra for the local and regional spectral libraries against the wet chemistry reference values. These two models were then used to predict the soil properties for both fields. We then mapped the variation in each soil property and the associated uncertainty by kriging. &amp;amp;#160;The variation in some of the soil variables was clearly affected by elevation and there were signs of spatial trend and so we used universal kriging to map the soil properties. To reduce bias, we used residual maximum likelihood estimation (REML) to estimate the variogram by fitting a linear mixed model with the trend accounted for as fixed effects. &amp;amp;#160;We compared these different maps to assess how the calibration regression from local and regional spectral libraries translates itself in uncertainty of kriged maps for five different soil properties within each field.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;amp;#160;&amp;lt;/p&amp;gt;

Modeling techniques can be powerful predictors of soil salinity across various scales, ranging from local landscapes to global territories. This study was aimed to examine the accuracy of soil salinity prediction model integrating ANNs (artificial neural networks) and topographic factors with different cell sizes. For this purpose, soil salinity was determined at 103 points in the east of Mashhad, Razavi Khorasan, Iran. The region was categorized into two distinct parts: study area (1) (with a steep topography) and study area (2) (with a flat topography). To explore the impact of terrain on salinity prediction accuracy, ANNs were trained using topographical factors as inputs across a range of cell sizes (30, 50, 90, 200, and 500 m). The model’s effectiveness was evaluated based on their Root Mean Square Error (RMSE) and coefficient of determination (R2). Results indicated variability in model performance, with RMSE ranging from 0.324 to 0.461 and R2 from 0.159 to 0.281 across the spectrum of cell sizes. Deeper analysis on different topographical influences showed that for the study area (1), a cell size of 30 m yielded the most accurate predictions (RMSE = 0.234 dS/m and R2 = 0.515), whereas for the study area (2), a cell size of 50 m was optimal (RMSE = 0.658 dS/m and R2 = 0.597). In general, the findings concluded that smaller cell sizes can enhance prediction accuracy in areas with complex and varied topography, while larger cell sizes can be more effective in flat areas. This study demonstrates the significance of incorporating terrain attributes and their optimal resolutions for accurate soil salinity prediction. Our findings underscore the importance of tailoring the resolution of input data to match the specific topographic features of the area, challenging the conventional notion that higher input resolution invariably yields better results in soil properties prediction. These insights provide valuable guidance for effective soil management and agricultural practices, as well as contribute to more informed decision-making in land management and environmental conservation.

Abstract. Conventional laboratory analysis of soil properties is often expensive and requires much time if various soil properties are to be measured. Visual and near-infrared (vis–NIR) spectroscopy offers a complementary and cost-efficient way to gain a wide variety of soil information at high spatial and temporal resolutions. Yet, applying vis–NIR spectroscopy requires confidence in the prediction accuracy of the infrared models. In this study, we used soil data from six agricultural fields in eastern Switzerland and calibrated (i) field-specific (local) models and (ii) general models (combining all fields) for soil organic carbon (SOC), permanganate oxidizable carbon (POXC), total nitrogen (N), total carbon (C) and pH using partial least-squares regression. The 30 local models showed a ratio of performance to deviation (RPD) between 1.14 and 5.27, and the root mean square errors (RMSE) were between 1.07 and 2.43 g kg−1 for SOC, between 0.03 and 0.07 g kg−1 for POXC, between 0.09 and 0.14 g kg−1 for total N, between 1.29 and 2.63 g kg−1 for total C, and between 0.04 and 0.19 for pH. Two fields with high carbonate content and poor correlation between the target properties were responsible for six local models with a low performance (RPD &lt; 2). Analysis of variable importance in projection, as well as of correlations between spectral variables and target soil properties, confirmed that high carbonate content masked absorption features for SOC. Field sites with low carbonate content can be combined with general models with only a limited loss in prediction accuracy compared to the field-specific models. On the other hand, for fields with high carbonate contents, the prediction accuracy substantially decreased in general models. Whether the combination of soils with high carbonate contents in one prediction model leads to satisfying prediction accuracies needs further investigation.

Tropical regions such as Java, Indonesia, still lack publication of soil water retention (SWR) information, particularly at upper Citarum watershed. The SWR is one of the critical elements in water storage and movement in the soil and very important to solve ecological and environmental problems. However, getting the access requires a lot of laboratory meas-urement that is time-consuming and expensive. Therefore, utilizing pedotransfer functions (PTFs) to estimate the water in the soil is needed. This study aims to define soil properties related to the SWR and to evaluate the performance of existing PTFs in predicting SWR. The study was carried out at agroforestry land system soil at upper Citarum watershed, Indonesia. Ten point and two continuous existing PTFs developed for tropical regions were applied in this study. Pearson's correlation (r), mean error (ME), root mean square error (RMSE), and modelling efficiency (EF) were used for evaluation. Cation exchange capacity (CEC), organic carbon (OC), bulk density (BD), and clay were considered as potential soil properties for soil water retention prediction. The performance of PTFs by MINASNY, HARTEMINK [2011] at matric potential of –10 kPaand BOTULA [2013] at matric potential of –33 kPa and –1500 kPa were recommended for point PTFs, while PTFs by HODNETT, TOMASELLA [2002] was for continuous PTFs in predicting SWR. The accuracy of the point PTFs is almost better than the continuous PTFs in predicting SWR in agroforestry land system soil at upper Citarum watershed, Indonesia.