Crop type mapping using optical and radar images: a review

LiDAR data are becoming increasingly available, which has opened up many new applications. One such application is crop type mapping. Accurate crop type maps are critical for monitoring water use, estimating harvests and in precision agriculture. The traditional approach to obtaining maps of cultivated fields is by manually digitizing the fields from satellite or aerial imagery and then assigning crop type labels to each field – often informed by data collected during ground and aerial surveys. However, manual digitizing and labeling is time-consuming, expensive and subject to human error. Automated remote sensing methods is a cost-effective alternative, with machine learning gaining popularity for classifying crop types. This study evaluated the use of LiDAR data, Sentinel-2 imagery, aerial imagery and machine learning for differentiating five crop types in an intensively cultivated area. Different combinations of the three datasets were evaluated along with ten machine learning. The classification results were interpreted by comparing overall accuracies, kappa, standard deviation and f-score. It was found that LiDAR data successfully differentiated between different crop types, with XGBoost providing the highest overall accuracy of 87.8%. Furthermore, the crop type maps produced using the LiDAR data were in general agreement with those obtained by using Sentinel-2 data, with LiDAR obtaining a mean overall accuracy of 84.3% and Sentinel-2 a mean overall accuracy of 83.6%. However, the combination of all three datasets proved to be the most effective at differentiating between the crop types, with RF providing the highest overall accuracy of 94.4%. These findings provide a foundation for selecting the appropriate combination of remotely sensed data sources and machine learning algorithms for operational crop type mapping.

With the latest development and increasing availability of high spatial resolution sensors, earth observation technology offers a viable solution for crop identification and management. There is a strong need to produce accurate, reliable and up to date crop type maps for sustainable agriculture monitoring and management. In this study, RapidEye, the first high-resolution multi-spectral satellite system that operationally provides a Red-edge channel, was used to test the potential of the data for crop type mapping. This study was investigated at a selected region mostly covered with agricultural fields locates in the low lands of Menemen (İzmir) Plain, TURKEY. The potential of the three classification algorithms such as Maximum Likelihood Classification, Support Vector Machine and Object Based Image Analysis is tested. Accuracy assessment of land cover maps has been performed through error matrix and kappa indexes. The results highlighted that all selected classifiers as highly useful (over 90%) in mapping of crop types in the study region however the object-based approach slightly outperforming the Support Vector Machine classification by both overall accuracy and Kappa statistics. The success of selected methods also underlines the potential of RapidEye data for mapping crop types in Aegean region.

MODIS phenology-derived, multi-year distribution of conterminous U.S. crop types

Crop type mapping by using transfer learning

In the field of smart farming, automated crop type mapping is a challenging task to guarantee fast and automatic management of the agricultural sector. With the emergence of advanced technologies such as artificial intelligence and geospatial technologies, new concepts were developed to provide realistic solutions to precision agriculture. The present study aims to present a machine learning-based model for automated crop-type mapping with high accuracy. The proposed model is based on the use of both optical and radar satellite images for the classification of crop types with machine learning-based algorithms. Random Forest and Support Vector Machine, were employed to classify the time series of vegetation indices. Several indices extracted from both optical and radar data were calculated. Harmonical modelization was also applied to optical indices, and decomposed into harmonic terms to calculate the fitted values of the time series. The proposed model was implemented using the geospatial processing services of Google Earth Engine and tested with a case study with about 147 satellite images. The results show the annual variability of crops and allowed performing classifications and crop type mapping with accuracy that exceeds the performances of the other existing models.

The invasion of Ukraine by Russian forces was expected to have global impact on food trade and security, since Ukraine is a breadbasket cereals and oil seeds producer. The NASA Harvest « Rapid Agricultural Assessment for Policy Support » (RAAPS) team was triggered early in the conflict to provide answers to the following questions : (i) How much winter cereals, winter oil seeds and summer crops were planted in Ukraine during the 2021-2022 cropping season?(ii) What proportion of those crops fell under the Russian occupied area? (iii) How much cropland was left unplanted in 2022 due to the war?As insights had to be produced within season, the NASA Harvest RAAPS team produced the first ever, Ukraine scale in-season crop type map based on Planet Labs 3 meter spatial -, 4 bands spectral -, and daily  temporal – resolution data.  Since   no   labeled   datasets   were   available   early   enough   in-season  for applying supervised machine learning techniques, cropland was progressively mapped   into   four   classes   (winter   cereals,   rapeseed,   summer   crops   and barren/non cultivated plots), using semi-supervised clustering techniques and heuristical thresholdings. Expert domain  knowledge  allowed to cope  with missing ground truth training data. First, active cropland was separated into winter crops and potential summer crops. K-means clustering of April and May Planet images, followed by visual cluster assignment, allowed to efficiently separate green crops (winter crops) from barren soils (potential summer crops). Then, another K-means clustering allowed to split winter crops into winter cereals and rapeseed as of end of May, based  on the intense yellow flowering signal of the latter. Finally a set of NDVI based heuristics was applied on potential summer crops in order to assess if green-up happened or not. Crops which   did   not   green   up   as   of   the   11th   of   July   2022   were   considered barren/non-planted. Road side ground surveyed crop type information collected in free Ukraine has been provided by Kussul & al. (2022) in August 2022. Validation against this data provided an overall accuracy of 94 % and a mean F1-score of 91 % for winter cereals, rapeseed and summer crops. No unplanted fields  were collected as part of the ground campaign. Several assessments of proportional area per crop type and occupation status were performed throughout the growing season, as occupation boundaries kept moving. As of the 11th of July 2022, 23.03 % of Ukraines cropland was occupied. 55.29 % of all detected barren fields were located within occupied territories, mainly scattered around the front line. 33.9 % of all winter crops were under occupied territory when harvest ready (mid July). This crop type map was used for computing harvested area, estimating yield and   for   production computation. Following NASA EarthObservatory articles were published,   providing   information   to   the   public   and   private   sector :   (i) https://earthobservatory.nasa.gov/images/150025/measuring-wars-effect-on-a-global-breadbasket    (ii) https://earthobservatory.nasa.gov/images/150590/larger-wheat-harvest-in-ukraine-than-expected 

Mapping and monitoring the distribution of croplands and crop types support policymakers and international organizations by reducing the risks to food security, notably from climate change and, for that purpose, remote sensing is routinely used. However, identifying specific crop types, cropland, and cropping patterns using space-based observations is challenging because different crop types and cropping patterns have similarity spectral signatures. This study applied a methodology to identify cropland and specific crop types, including tobacco, wheat, barley, and gram, as well as the following cropping patterns: wheat-tobacco, wheat-gram, wheat-barley, and wheat-maize, which are common in Gujranwala District, Pakistan, the study region. The methodology consists of combining optical remote sensing images from Sentinel-2 and Landsat-8 with Machine Learning (ML) methods, namely a Decision Tree Classifier (DTC) and a Random Forest (RF) algorithm. The best time-periods for differentiating cropland from other land cover types were identified, and then Sentinel-2 and Landsat 8 NDVI-based time-series were linked to phenological parameters to determine the different crop types and cropping patterns over the study region using their temporal indices and ML algorithms. The methodology was subsequently evaluated using Landsat images, crop statistical data for 2020 and 2021, and field data on cropping patterns. The results highlight the high level of accuracy of the methodological approach presented using Sentinel-2 and Landsat-8 images, together with ML techniques, for mapping not only the distribution of cropland, but also crop types and cropping patterns when validated at the county level. These results reveal that this methodology has benefits for monitoring and evaluating food security in Pakistan, adding to the evidence base of other studies on the use of remote sensing to identify crop types and cropping patterns in other countries.

The aim of this paper is to map agricultural crops by classifying satellite image time series. Domain experts in agriculture work with crop type labels that are organised in a hierarchical tree structure, where coarse classes (like orchards) are subdivided into finer ones (like apples, pears, vines, etc.). We develop a crop classification method that exploits this expert knowledge and significantly improves the mapping of rare crop types. The three-level label hierarchy is encoded in a convolutional, recurrent neural network (convRNN), such that for each pixel the model predicts three labels at different level of granularity. This end-to-end trainable, hierarchical network architecture allows the model to learn joint feature representations of rare classes (e.g., apples, pears) at a coarser level (e.g., orchard), thereby boosting classification performance at the fine-grained level. Additionally, labelling at different granularity also makes it possible to adjust the output according to the classification scores; as coarser labels with high confidence are sometimes more useful for agricultural practice than fine-grained but very uncertain labels. We validate the proposed method on a new, large dataset that we make public. ZueriCrop covers an area of 50 km × 48 km in the Swiss cantons of Zurich and Thurgau with a total of 116′000 individual fields spanning 48 crop classes, and 28,000 (multi-temporal) image patches from Sentinel-2. We compare our proposed hierarchical convRNN model with several baselines, including methods designed for imbalanced class distributions. The hierarchical approach performs superior by at least 9.9 percentage points in F1-score.

Crop-type mapping using time-series remote sensing data is crucial for a wide range of agricultural applications. Crop mapping during the growing season is particularly critical in timely monitoring of the agricultural system. Most existing studies focusing on within-season crop mapping leverage historical remote sensing and crop type reference data for model building, due to the difficulty in obtaining timely crop type samples for the current growing season. Yet the crop type samples from previous years may not be used directly considering the diverse patterns of crop phenology across years and locations, which hampers the scalability and transferability of the model to the current season for timely crop mapping. This article proposes an innovative within-season emergence (WISE) phenology normalized deep learning model towards scalable within-season crop mapping. The crop time-series remote sensing data are first normalized by the WISE crop emergence dates before being fed into an attention-based one-dimensional convolutional neural network classifier. Compared to conventional calendar-based approaches, the WISE-phenology normalization approach substantially helps the deep learning crop mapping model accommodate the spatiotemporal variations in crop phenological dynamics. Results in Illinois from 2017 to 2020 indicate that the proposed model outperforms calendar-based approaches and yields over 90% overall accuracy for classifying corn and soybeans at the end of season. During the growing season, the proposed model can give satisfactory performance (85% overall accuracy) one to four weeks earlier than calendar-based approaches. With WISE-phenology normalization, the proposed model exhibits more stable performance across Illinois and can be transferred to different years with enhanced scalability and robustness.

The fragmented nature of arable landscapes and diverse cropping patterns often thwart the precise mapping of crop types. Recent advances in remote-sensing technologies and data mining approaches offer a viable solution to this mapping problem. We demonstrated the potential of using hyperspectral imaging and an ensemble classification approach that combines five machine-learning classifiers to map crop types in the Anand District of Gujarat, India. We derived a set of narrow/broad-band indices from the Airborne Visible Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) imagery to represent spectral variations and identify target classes and their distribution patterns. The results showed that Maximum Entropy (MaxEnt) and Generalised Linear Model (GLM) had strong discriminatory image classification abilities with Area Under the Curve (AUC) values ranging between 0.75 and 0.93 for MaxEnt and between 0.73 and 0.92 for GLM. The ensemble model resulted in improved accuracy scores compared to individual models. We found the Photochemical Reflectance Index (PRI) and Moment Distance Ratio Right/Left (MDRRL) to be important predictors for target classes such as wheat, legumes, and eggplant. Results from the study revealed the potential of using one-class ensemble modelling approach and hyperspectral images with limited field dataset to map agricultural systems that are fragmented in nature.

Timely and accurate crop mapping is crucial for providing essential data support for agricultural production management. Reliable ground truth samples form the foundation for crop mapping using remote sensing imagery, a task that presents significant challenges in regions with limited sample availability. To address this issue, this study evaluates instance-based transfer learning methods, using the Hexi Corridor as a case study to explore crop mapping strategies in areas with scarce samples. High-confidence pixels from the United States Cropland Data Layer (CDL), along with high-density time series data derived from Sentinel-1, Sentinel-2, and Landsat-8 satellite imagery, as well as key vegetation indices, were selected as training samples for the source domain. Various algorithms, including Random Forest (RF), Extreme Gradient Boosting (XGBoost), and TrAdaBoost, were employed to transfer knowledge from the source domain to the target domain for crop type mapping. The results demonstrated that during the transfer learning process using only source domain data—without utilizing any target domain data—the overall classification accuracy reached 73.88%, with optimal accuracies for maize and alfalfa at 88.97% and 85.23%, respectively. As target domain data were gradually incorporated, the total accuracy for all models ranged from 0.77 to 0.92, with F1-scores ranging from 0.76 to 0.92, showing a consistent improvement in model performance. This study highlights the feasibility of employing transfer learning for crop mapping in the Hexi Corridor, demonstrating its potential to reduce labeling costs for target domain samples and providing a valuable reference for crop mapping in regions with limited sample availability.

Crop type information is essential for many practical applications, yet its mapping is often constrained by inherent characteristics of most farming areas, such as fragmentation and small farm plots, changes in crop morphology across the season, and cloud cover. This study investigated whether these limitations could be overcome by using time-series of Synthetic Aperture Radar (SAR) and crop phenological information combined with different Dynamic Time Warping implementation strategies. Focusing on a fragmented landscape with small farm plots in the Netherlands, we used Sentinel-1 dual polarimetry (VV + VH) and TerraSAR-X single polarimetry (HH) images and the Dutch Basic Registration of Crop Plots (BRP) dataset for training and validation. Image pre-processing was followed by the generation of radar vegetation indices and polarimetric decomposition. Crop-specific responses to incident radar signal were analyzed, as well as the accuracy of the crop classification by Time-Weighted Dynamic Time Warping (twDTW) using either backscatter bands only or backscatter bands in combination with derived indices and polarimetric decomposition features. In addition, two further modified Dynamic Time Warping strategies, namely Variable Time Weight Dynamic Time Warping (vtwDTW) and Angular Metric for Shape Similarity (AMSS), were tested for their performance. Furthermore, we investigated the accuracy performance of a decision level fusion of TerraSAR-X and Sentinel-1 classification outputs. Results show that even in a fragmented landscape with relatively small plots (around 0.08 ha), crop types can be successfully mapped by using decision level fusion of the twDTW results of both sensors, reaching an accuracy of 77.1%. When using twDTW on Sentinel-1 (VV + VH) only, including Ratio, MRVI, and DPSVI indices, overall accuracy reached 69.5%; without those indices, accuracy was slightly lower (67.5%). Merging six different grain crops with similar leaf geometry into winter grains and summer grains improved classification accuracy to 80.6%. Our findings demonstrate that twDTW on SAR imagery allows to map crop types in fragmented landscapes with relatively small farm plots, offering potential for crop type mapping in areas with smallholder farming.

Crop type mapping is relevant to a wide range of food security applications. Supervised classification methods commonly generate these data from satellite image time-series. Yet, their successful implementation is hindered by the lack of training samples. Solutions like transfer learning, development of temporal-spectral signatures of the target classes, re-utilization of existing inventories, or crowdsourcing initiatives are commonly applied to generate samples for thematically coarser classifications. These methods are rarely used for generating crop types samples. In this study, we leverage the phenology information of existing data inventories using Time-Weighted Dynamic Time Warping (TWDTW) to address the problem of automatic crop sample generation in two target areas. Resulting labeled samples are refined using proximity measures obtained from Random Forests (RF). Sentinel-2 time-series are used to obtain phenology information from two study areas. The proposed methodology achieved promising results for classes with a reduced inter-classes similarity such as sugar beets (user’s accuracy, UA, of 98% and producer’s accuracy, PA, of 100%) or grains (UA of 98% and PA of 90%). The crops with a high inter-classes similarity yielded less satisfactory results. Potatoes, for example, obtained a high PA of 95%, but a UA of only 36% because of the spectral-temporal similarity with maize. The methodology works well for areas with balanced crop samples. Yet, it favors prevalent classes in areas with imbalanced crops at the expense of a low accuracy for the minority crops. Despite these shortcomings, the proposed methodology offers a viable option to generate crop samples in regions with few ground labels.

The creation of crop type maps from satellite data has proven challenging and is often impeded by a lack of accurate in situ data. Street-level imagery represents a new potential source of in situ data that may aid crop type mapping, but it requires automated algorithms to recognize the features of interest. This paper aims to demonstrate a method for crop type (i.e., maize, wheat and others) recognition from street-level imagery based on a convolutional neural network using a bottom-up approach. We trained the model with a highly accurate dataset of crowdsourced labelled street-level imagery using the Picture Pile application. The classification results achieved an AUC of 0.87 for wheat, 0.85 for maize and 0.73 for others. Given that wheat and maize are two of the most common food crops grown globally, combined with an ever-increasing amount of available street-level imagery, this approach could help address the need for improved global crop type monitoring. Challenges remain in addressing the noise aspect of street-level imagery (i.e., buildings, hedgerows, automobiles, etc.) and uncertainties due to differences in the time of day and location. Such an approach could also be applied to developing other in situ data sets from street-level imagery, e.g., for land use mapping or socioeconomic indicators.

Minor crops are crucial for food security, especially due to their resilience against climate change related challenges (Renard & Tilman, 2019) . Consequently, accurate crop mapping is essential for monitoring policies seeking to incentivize minor crop production. However, the class imbalance problem in machine learning introduces a bias against these crops, leading to unfair classifications. This research aims to explore how this bias is mitigated through two main class imbalance correction approaches: sample balancing methods and cost-sensitive learning. Apart from investigating how these methods address the typical class imbalance problem, where there are simply less labelled samples of a specific class, we investigate how these methods can be used to address another level of bias, that created by omitted sensitive attributes. These are attributes such as parcel size, which are not explicitly considered by the classifier, yet significantly impact accuracy and contribute to the unfairness of the classification, as evidenced by notably lower accuracy for smaller parcels. By integrating these attributes into the class imbalance correction methods, we assess the potential for enhancing fairness. This approach is vital, as it corrects performance biases affecting specific sub-groups, which are not necessarily class dependent, thus addressing a critical but overlooked dimension of fairness in classification. Utilizing the BreizhCrops dataset, we create sub-sampled datasets that represent a variety of class imbalance problems. This enables us to conduct an across-the-board comparison of the selected class imbalance correction techniques, providing insights that may help streamline future research looking to employ these techniques. For the classifier architecture, we select the transformer encoder, chosen for its greater performance among deep learning methods tested on the BreizhCrops dataset (Rußwurm et al., 2020). This research contributes to the broader understanding of class imbalance correction in classification tasks, particularly for crop mapping, though the methods can also be applied in other GeoAI contexts. By evaluating sample balancing and cost-sensitive learning in varied contexts, we provide insights into optimizing classification tasks for fairness. Our work contributes to the development of responsible AI practices by offering valuable insights on how fairness can be enhanced across GeoAI applications.   Renard, D., & Tilman, D. (2019). National food production stabilized by crop diversity. Nature, 571(7764), 257–260. https://doi.org/10.1038/s41586-019-1316-y Rußwurm, M., Pelletier, C., Zollner, M., Lefèvre, S., & Körner, M. (2020). BreizhCrops: A Time Series Dataset for Crop Type Mapping (arXiv:1905.11893). arXiv. https://doi.org/10.48550/arXiv.1905.11893