Crop type mapping using LiDAR, Sentinel-2 and aerial imagery with machine learning algorithms

Crop type identification and mapping have been a classic issue essential for agriculture characterisation and food security as well as crop health monitoring. Remote sensing was proven to be an efficient tool to define and study agriculture. The complementarity between optical and radar data can hugely improve crop type identification and mapping. This document presents an overview of more than 40 recent works between 2013 and 2019, related to crop type mapping using satellite imagery, and points the general techniques used. Different classification algorithms was studied and compared, as well as the comparison of the inputs, optimal bands and polarization for radar data, and process giving the best results. It is often concluded that fusion improves results when compared with single data using. Nevertheless, it had been challenging to combine datasets from sensors that had different physical concepts, and progress in this domain require the development of systematic procedures.

Mapping and monitoring the distribution of croplands and crop types support policymakers and international organizations by reducing the risks to food security, notably from climate change and, for that purpose, remote sensing is routinely used. However, identifying specific crop types, cropland, and cropping patterns using space-based observations is challenging because different crop types and cropping patterns have similarity spectral signatures. This study applied a methodology to identify cropland and specific crop types, including tobacco, wheat, barley, and gram, as well as the following cropping patterns: wheat-tobacco, wheat-gram, wheat-barley, and wheat-maize, which are common in Gujranwala District, Pakistan, the study region. The methodology consists of combining optical remote sensing images from Sentinel-2 and Landsat-8 with Machine Learning (ML) methods, namely a Decision Tree Classifier (DTC) and a Random Forest (RF) algorithm. The best time-periods for differentiating cropland from other land cover types were identified, and then Sentinel-2 and Landsat 8 NDVI-based time-series were linked to phenological parameters to determine the different crop types and cropping patterns over the study region using their temporal indices and ML algorithms. The methodology was subsequently evaluated using Landsat images, crop statistical data for 2020 and 2021, and field data on cropping patterns. The results highlight the high level of accuracy of the methodological approach presented using Sentinel-2 and Landsat-8 images, together with ML techniques, for mapping not only the distribution of cropland, but also crop types and cropping patterns when validated at the county level. These results reveal that this methodology has benefits for monitoring and evaluating food security in Pakistan, adding to the evidence base of other studies on the use of remote sensing to identify crop types and cropping patterns in other countries.

MODIS phenology-derived, multi-year distribution of conterminous U.S. crop types

The invasion of Ukraine by Russian forces was expected to have global impact on food trade and security, since Ukraine is a breadbasket cereals and oil seeds producer. The NASA Harvest « Rapid Agricultural Assessment for Policy Support » (RAAPS) team was triggered early in the conflict to provide answers to the following questions : (i) How much winter cereals, winter oil seeds and summer crops were planted in Ukraine during the 2021-2022 cropping season?(ii) What proportion of those crops fell under the Russian occupied area? (iii) How much cropland was left unplanted in 2022 due to the war?As insights had to be produced within season, the NASA Harvest RAAPS team produced the first ever, Ukraine scale in-season crop type map based on Planet Labs 3 meter spatial -, 4 bands spectral -, and daily  temporal – resolution data.  Since   no   labeled   datasets   were   available   early   enough   in-season  for applying supervised machine learning techniques, cropland was progressively mapped   into   four   classes   (winter   cereals,   rapeseed,   summer   crops   and barren/non cultivated plots), using semi-supervised clustering techniques and heuristical thresholdings. Expert domain  knowledge  allowed to cope  with missing ground truth training data. First, active cropland was separated into winter crops and potential summer crops. K-means clustering of April and May Planet images, followed by visual cluster assignment, allowed to efficiently separate green crops (winter crops) from barren soils (potential summer crops). Then, another K-means clustering allowed to split winter crops into winter cereals and rapeseed as of end of May, based  on the intense yellow flowering signal of the latter. Finally a set of NDVI based heuristics was applied on potential summer crops in order to assess if green-up happened or not. Crops which   did   not   green   up   as   of   the   11th   of   July   2022   were   considered barren/non-planted. Road side ground surveyed crop type information collected in free Ukraine has been provided by Kussul & al. (2022) in August 2022. Validation against this data provided an overall accuracy of 94 % and a mean F1-score of 91 % for winter cereals, rapeseed and summer crops. No unplanted fields  were collected as part of the ground campaign. Several assessments of proportional area per crop type and occupation status were performed throughout the growing season, as occupation boundaries kept moving. As of the 11th of July 2022, 23.03 % of Ukraines cropland was occupied. 55.29 % of all detected barren fields were located within occupied territories, mainly scattered around the front line. 33.9 % of all winter crops were under occupied territory when harvest ready (mid July). This crop type map was used for computing harvested area, estimating yield and   for   production computation. Following NASA EarthObservatory articles were published,   providing   information   to   the   public   and   private   sector :   (i) https://earthobservatory.nasa.gov/images/150025/measuring-wars-effect-on-a-global-breadbasket    (ii) https://earthobservatory.nasa.gov/images/150590/larger-wheat-harvest-in-ukraine-than-expected 

Crop type mapping provides essential information to control and make decisions related to agricultural practices and their regulations. To map crop types accurately, it is important to capture their phenological stages and fine spatial details, especially in a temporally and spatially heterogeneous landscape. The data availability of new generation multispectral sensors of Landsat-8 (L8) and Sentinel-2 (S2) satellites offers unprecedented options for such applications. Given this, our study aims to display how the synergistic use of these optical sensors can efficiently support crop type mapping research while integrating an object-based image analysis (OBIA). Through the applied methods, we used the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) and the flexible spatiotemporal data-fusion model (FSDAF) to blend L8 and S2 data and obtain reliable normalized difference vegetation index (NDVI) datasets with fine spatial and temporal resolution. Then the crop phenological information was extracted using a Savitzky–Golay filter and fused NDVI time series. Finally, a model combining phenological metrics and fused reconstructed NDVI as classification features was developed using a random forest (RF) classifier/OBIA approach. The results show that the FSDAF method creates more accurate fused NDVI and keeps more spatial details than ESTARFM. The FSDAF model was then used to create fused, high-resolution time-series products that were able to extract crop phenology in single-crop fields while providing a very detailed pattern relative to that from individual sensor time-series data. Moreover, combined L8 and S2 data by FSDAF produced highly significant overall classification accuracies (90.03% for pixel-based RF to 93.12% OBIA RF), outperforming individual sensor use (82.57% for L8-only; 88.45% for S2-only). Our proposed workflow highlights the advantage of spatiotemporal fusing and OBIA environment in spatiotemporally heterogeneous areas and fragmented landscapes, which represents a promising step toward generating fast, accurate, and ready-to-use agricultural data products.

High temporal resolution of synthetic aperture radar (SAR) imagery (e.g., Sentinel-1 (S1) imagery) creates new possibilities for monitoring green vegetation in urban areas and generating land-cover classification (LCC) maps. This research evaluates how different pre-processing steps of SAR imagery affect classification accuracy. Machine learning (ML) methods were applied in three different study areas: random forest (RF), support vector machine (SVM), and extreme gradient boosting (XGB). Since the presence of the speckle noise in radar imagery is inevitable, different adaptive filters were examined. Using the backscattering values of the S1 imagery, the SVM classifier achieved a mean overall accuracy (OA) of 63.14%, and a Kappa coefficient (Kappa) of 0.50. Using the SVM classifier with a Lee filter with a window size of 5×5 (Lee5) for speckle reduction, mean values of 73.86% and 0.64 for OA and Kappa were achieved, respectively. An additional increase in the LCC was obtained with texture features calculated from a grey-level co-occurrence matrix (GLCM). The highest classification accuracy obtained for the extracted GLCM texture features using the SVM classifier, and Lee5 filter was 78.32% and 0.69 for the mean OA and Kappa values, respectively. This study improved LCC with an evaluation of various radiometric and texture features and confirmed the ability to apply an SVM classifier. For the supervised classification, the SVM method outperformed the RF and XGB methods, although the highest computational time was needed for the SVM, whereas XGB performed the fastest. These results suggest pre-processing steps of the SAR imagery for green infrastructure mapping in urban areas. Future research should address the use of multitemporal SAR data along with the pre-processing steps and ML algorithms described in this research.

Crop type mapping by using transfer learning

Glacial lakes are increasing in number and size worldwide, posing growing risks for outburst floods. Norway’s last glacial lake inventory used semi-automatic mapping on Sentinel-2 imagery from 2018–19. In this study, we test a more automated and reproducible workflow for updating glacial lake extents in Norway using Sentinel-2 and Sentinel-1 satellite imagery and a Random Forest classifier. Here, glacial lakes are defined as water bodies within 200 m of glaciers larger than 0.1 km 2 with a minimum lake size of 400 m 2 . A 10th-percentile Sentinel-2 summer composite from 2023–24 mitigated snow and cloud cover, while Sentinel-1 ascending-descending difference composites reduced shadow misclassification without relying on DEMs. Validation across six glacier regions shows high detection reliability (F1-score: 0.81) as well as high delineation accuracy (median deviation <6.5 m). However, manual correction remains necessary, especially in steep terrain. We identified 1382 glacial lakes in 2023–24, covering 124 km 2 —a substantial increase relative to 2018–19. Excluding regulated lakes and adjusting for methodological differences, we estimate a 9–22% lake area increase over the past five years, mainly driven by glacier retreat. The workflow is efficient and reproducible, but regional threshold adaptation and retraining are required for transfer to other regions.

The creation of crop type maps from satellite data has proven challenging and is often impeded by a lack of accurate in situ data. Street-level imagery represents a new potential source of in situ data that may aid crop type mapping, but it requires automated algorithms to recognize the features of interest. This paper aims to demonstrate a method for crop type (i.e., maize, wheat and others) recognition from street-level imagery based on a convolutional neural network using a bottom-up approach. We trained the model with a highly accurate dataset of crowdsourced labelled street-level imagery using the Picture Pile application. The classification results achieved an AUC of 0.87 for wheat, 0.85 for maize and 0.73 for others. Given that wheat and maize are two of the most common food crops grown globally, combined with an ever-increasing amount of available street-level imagery, this approach could help address the need for improved global crop type monitoring. Challenges remain in addressing the noise aspect of street-level imagery (i.e., buildings, hedgerows, automobiles, etc.) and uncertainties due to differences in the time of day and location. Such an approach could also be applied to developing other in situ data sets from street-level imagery, e.g., for land use mapping or socioeconomic indicators.

Crop type information is essential for many practical applications, yet its mapping is often constrained by inherent characteristics of most farming areas, such as fragmentation and small farm plots, changes in crop morphology across the season, and cloud cover. This study investigated whether these limitations could be overcome by using time-series of Synthetic Aperture Radar (SAR) and crop phenological information combined with different Dynamic Time Warping implementation strategies. Focusing on a fragmented landscape with small farm plots in the Netherlands, we used Sentinel-1 dual polarimetry (VV + VH) and TerraSAR-X single polarimetry (HH) images and the Dutch Basic Registration of Crop Plots (BRP) dataset for training and validation. Image pre-processing was followed by the generation of radar vegetation indices and polarimetric decomposition. Crop-specific responses to incident radar signal were analyzed, as well as the accuracy of the crop classification by Time-Weighted Dynamic Time Warping (twDTW) using either backscatter bands only or backscatter bands in combination with derived indices and polarimetric decomposition features. In addition, two further modified Dynamic Time Warping strategies, namely Variable Time Weight Dynamic Time Warping (vtwDTW) and Angular Metric for Shape Similarity (AMSS), were tested for their performance. Furthermore, we investigated the accuracy performance of a decision level fusion of TerraSAR-X and Sentinel-1 classification outputs. Results show that even in a fragmented landscape with relatively small plots (around 0.08 ha), crop types can be successfully mapped by using decision level fusion of the twDTW results of both sensors, reaching an accuracy of 77.1%. When using twDTW on Sentinel-1 (VV + VH) only, including Ratio, MRVI, and DPSVI indices, overall accuracy reached 69.5%; without those indices, accuracy was slightly lower (67.5%). Merging six different grain crops with similar leaf geometry into winter grains and summer grains improved classification accuracy to 80.6%. Our findings demonstrate that twDTW on SAR imagery allows to map crop types in fragmented landscapes with relatively small farm plots, offering potential for crop type mapping in areas with smallholder farming.

With the latest development and increasing availability of high spatial resolution sensors, earth observation technology offers a viable solution for crop identification and management. There is a strong need to produce accurate, reliable and up to date crop type maps for sustainable agriculture monitoring and management. In this study, RapidEye, the first high-resolution multi-spectral satellite system that operationally provides a Red-edge channel, was used to test the potential of the data for crop type mapping. This study was investigated at a selected region mostly covered with agricultural fields locates in the low lands of Menemen (İzmir) Plain, TURKEY. The potential of the three classification algorithms such as Maximum Likelihood Classification, Support Vector Machine and Object Based Image Analysis is tested. Accuracy assessment of land cover maps has been performed through error matrix and kappa indexes. The results highlighted that all selected classifiers as highly useful (over 90%) in mapping of crop types in the study region however the object-based approach slightly outperforming the Support Vector Machine classification by both overall accuracy and Kappa statistics. The success of selected methods also underlines the potential of RapidEye data for mapping crop types in Aegean region.

Timely and accurate estimates of sugarcane yield provide valuable information for food management, bio-energy production, (inter)national trade, industry planning and government policy. Remote sensing and machine learning approaches can improve sugarcane yield estimation. Previous attempts have however often suffered from too few training samples due to the fact that field data collection is expensive and time-consuming. Our study demonstrates that unmanned aerial vehicle (UAV) data can be used to generate field-level yield data using only a limited number of field measurements. Plant height obtained from RGB UAV-images was used to train a model to derive intra-field yield maps based on 41 field sample plots spread over 20 sugarcane fields in the Udon Thani Province, Thailand. The yield maps were subsequently used as reference data to train another model to estimate yield from multi-spectral Sentinel-2 (S2) imagery. The integrated UAV yield and S2 data was found efficient with RMSE of 6.88 t/ha (per 10 m × 10 m pixel), for average yields of about 58 t/ha. The expansion of the sugarcane yield mapping across the entire region of 11,730 km2 was in line with the official statistical yield data and highlighted the high spatial variability of yields, both between and within fields. The presented method is a cost-effective and high-quality yield mapping approach which provides useful information for sustainable sugarcane yield management and decision-making.

Aim of study: The main objective is to determine the best machine-learning algorithm to classify the stand types of Monteverde forests combining LiDAR, orthophotography, and Sentinel-2 data, thus providing an easy and cheap method to classify Monteverde stand types.Area of study: 1500 ha forest in Monteverde, North Tenerife, Canary Islands.Material and methods: RF, SVML, SVMR and ANN algorithms are used to classify the three Monteverde stand types. Before training the model, feature selection of LiDAR, orthophotography, and Sentinel-2 data through VSURF was carried out. Comparison of its accuracy was performed.Main results: Five LiDAR variables were found to be the most efficient for classifying each object, while only one Sentinel-2 index and one Sentinel-2 band was valuable. Additionally, standard deviation and mean of the Red orthophotography colour band, and ratio between Red and Green bands were also found to be suitable. SVML is confirmed as the most accurate algorithm (0.904, 0.041 SD) while ANN showed the lowest value of 0.891 (0.073 SD). SVMR and RF obtain 0.902 (0.060 SD) and 0.904 (0.056 SD) respectively. SVML was found to be the best method given its low standard deviation.Research highlights: The similar high accuracy values among models confirm the importance of taking into account diverse machine-learning methods for stand types classification purposes and different explanatory variables. Although differences between errors may not seem relevant at a first glance, due to the limited size of the study area with only three plus two categories, such differences could be highly important when working at large scales with more stand types.ADDITIONAL KEY WORDSRF algorithm, SVML algorithm, SVMR algorithm, ANN algorithm, LiDAR, orthophotography, Sentinel-2ABBREVIATIONS USEDANN, artificial neural networks algorithm; Band04, Sentinel-2 band 04 image data; BR, brezal; DTHM, digital tree height model; DTHM-2016, digital tree height model based on 2016 LiDAR data; DTM, digital terrain model; DTM-2016, digital terrain model based on 2016 LiDAR data; FBA, fayal-brezal-acebiñal; FCC, canopy cover; HEIGHT-2009, maximum height based on 2009 LiDAR data; HGR, height growth based on 2009 and 2016 LiDAR data; LA, laurisilva; NDVI705, Sentinel-2 index image data; NMF, non-Monteverde forest; NMG, non-Monteverde ground; P95-2016, height percentile 95 based on 2016 LiDAR data; RATIO R/G, ratio between Red and Green bands orthophotograph data; RED, Red band orthophotograph data; Red-SD, standard deviation of the Red band orthophotograph data; RF, random forest algorithm; SVM, support vector machine algorithm; SVML, linear support vector machine algorithm; SVMR, radial support vector machine algorithm; VSURF, variable selection using random forest.

Crop classification makes it possible to obtain the spatial distribution and planting area information of crops. Meanwhile, a multi-year crop mapping helps to study the dynamics of agricultural fields and may serve as the basis of crop structure adjustment and optimization to assist government decision-making. This study aims to explore the feasibility of Sentinel-2 imagery data for crop type mapping and to evaluate the effectiveness of Support Vector Machine (SVM) in discriminating various crop types using a small training dataset. In doing so, the classical SVM classification method was applied to distinguish major crop types in Plymouth County, Iowa, USA, based on Sentinel-2 time-series Normalized Difference Vegetation Index data in 2016. The results show that the SVM model can effectively classify three major crop types (corn, soybean and alfalfa) in Plymouth with an overall accuracy of >67% and a Kappa coefficient of 0.52, which evaluated the potential of utilizing SVM to obtain the spatial distribution information of crops based on the new Sentinel-2 imagery. Besides, this exploration of new data sources for crop classification can give some guidance and the method can be applied to other regions.

Crop type mapping is relevant to a wide range of food security applications. Supervised classification methods commonly generate these data from satellite image time-series. Yet, their successful implementation is hindered by the lack of training samples. Solutions like transfer learning, development of temporal-spectral signatures of the target classes, re-utilization of existing inventories, or crowdsourcing initiatives are commonly applied to generate samples for thematically coarser classifications. These methods are rarely used for generating crop types samples. In this study, we leverage the phenology information of existing data inventories using Time-Weighted Dynamic Time Warping (TWDTW) to address the problem of automatic crop sample generation in two target areas. Resulting labeled samples are refined using proximity measures obtained from Random Forests (RF). Sentinel-2 time-series are used to obtain phenology information from two study areas. The proposed methodology achieved promising results for classes with a reduced inter-classes similarity such as sugar beets (user’s accuracy, UA, of 98% and producer’s accuracy, PA, of 100%) or grains (UA of 98% and PA of 90%). The crops with a high inter-classes similarity yielded less satisfactory results. Potatoes, for example, obtained a high PA of 95%, but a UA of only 36% because of the spectral-temporal similarity with maize. The methodology works well for areas with balanced crop samples. Yet, it favors prevalent classes in areas with imbalanced crops at the expense of a low accuracy for the minority crops. Despite these shortcomings, the proposed methodology offers a viable option to generate crop samples in regions with few ground labels.