Crop Classification from UAV-Based Multi-spectral Images Using Deep Learning

Accurate leaf area index estimation in sorghum using high-resolution UAV data and machine learning models

Abstract: The use of Unmanned Aerial Vehicles (UAVs) is becoming very common for last few years for monitoring agricultural crops efficiently. Low altitude remote sensing (UAV) provides people with high temporal and spatial resolution for non-destructive, accurate and timely estimation of biophysical parameters like Leaf Area Index (LAI), crop growth, plant biomass and final crop yield. Collection of the data by UAV helps to reduce errors and it fills the biasness on an observational scale in precision agriculture. The main objective of this study was to estimate real time LAI and yield of groundnut crop based on Normalized Difference Vegetation Index (NDVI) using low cost multispectral UAV. A field experiment was set up  with three different groundnut cultivars (V1= BARD-479, V2 = Chakwal-2011 and V3 = Chakwal-2016) with three replications. Field data collection regarding LAI was performed in 2019 at two different growth stages (2-3 leaf stage and Peg formation stage) of groundnut on PMAS-Arid Agriculture Research Farm (Knoot), Pakistan. Final yield was calculated at the time of crop maturity. In this study, low cost UAV platform was established with DJI Phantom 4 pro and Parrot Sequoia Sensor to develop a multispectral UAV system used as the survey platform. A Parrot Sequoia camera was mounted on the UAV used as the remote sensor. The sensor provided the information in five narrow bands including Red, Blue, Green, Near infrared (NIR) and Red Edge. The processing of UAV images was performed in the Python environment and NDVI images were created. Then regression model was performed to compare the NDVI data with the LAI and final yield of groundnut crop. The results indicated that the highest value of R 2 = 0.93 was found with NDVI and LAI at peg formation stage while value of R 2 = 0.59 was at 2-3 leaf stage. The strong and positive relationship was found between LAI and yield ( R 2 = 0.97). There was also a strong and positive relationship between NDVI and yield of groundnut with value of R 2 = 0.92. The study showed that low cost multispectral UAV can be effectively used for real time estimation of LAI and groundnut yield nondestructively and accurately. The study results show that this low cost multispectral UAV platform (DJI Phantom 4 Pro with Parrot Squoia) is robust in management decisions of agriculture such as effective fertilizer application, growth monitoring, and yield estimation accurately and timely based on the vegetation indices. This study also proved the low cost multispectral UAV practicability in estimating plant biophysical parameters at a small field experiment scale reliably. Keywords: vegetation indices, UAV, LAI, yield, multispectral camera, NDVI, groundnut, growth stages DOI:  10.33440/j.ijpaa.20200301.70  Citation: Tahir M N, Lan Y B, Zhang Y L, Wang Y K, Faisal N, Shah M A A, et al.  Real time estimation of leaf area index and groundnut yield using multispectral UAV.  Int J Precis Agric Aviat, 2020; 3(1): 1–6.

Timely and accurate extraction of crop planting units plays a critical role in crop yield estimation, soil management, food supplies, and disaster warnings. However, precisely mapping crop types is challenging in smallholder farming systems due to heterogeneous and mixed pixels, where field sizes are small, and crop types are very diverse. In this paper, the crop type system of Huocheng County in northwest China as an example area, crop classification feature variables are constructed using Sentinel-2 remote sensing images, and combining the ReliefF algorithm, three feature selection classification models are established by 3092 crop type sampling points data. The crop planting units are extracted using pixel-based and object-based classification methods, respectively, and unmanned aerial vehicle (UAV) data assess the accuracy as ground truth. The Sentinel-2 image based smallholder crops classification results indicate that: The effective combination of optimal input feature variables selection and classification models significantly improves crop classification accuracy. This is particularly evident when integrating this approach into pixel-based classification, addressing challenges such as low pixel classification accuracy in regions of this type. The validation results from UAV data also confirm these findings. At validation points 2, 3, and 4, pixel-based classification demonstrates significantly higher accuracy compared to object-based classification, with pixel-based classification accuracy improving by 9.16%, 7.83%, and 31.83%, respectively. Although the UAV validation accuracy does not reach a high level, pixel-based classification remains the optimal choice for smallholder crop classification. This method offers new insights and references for research related to the classification of smallholder crops with complex planting structures.

The convolutional neural network (CNN) architecture has shown remarkable success in image classification and segmentation. Its popularity has increased promptly due to various factors such as exponential growth in computational resources, availability of benchmark datasets, supporting libraries, and open-source software. The efficiency of CNN architecture majorly depends on the complexity of the architecture, availability of datasets, and hyperparameter selection. But, due to the huge number of parameters in CNN architecture, its selection has completely remained ad hoc in the past works. In this article, a novel encoding technique has been proposed that can represent complex CNN architecture effectively. The article defines basic building blocks to represent CNN architecture such as the genesis block, transit block, agile block, and output block. This encoding structure is used to generate dynamic length chromosome structure and initialized using evolutionary algorithms. A comparative analysis is also presented that shows compared its effectiveness with existing encoding representations on the basis of the number of encoding parameters, training cost, and efficiency.

In Indonesia, where the agricultural insurance system has been in full operation since 2016, a new damage assessment estimation formula for rice diseases was created through integrating the current damage assessment method and unmanned aerial vehicle (UAV) multispectral remote sensing data to improve the efficiency and precision of damage assessment work performed for the payments of insurance claims. The new method can quickly and efficiently output objective assessment results. In this study, UAV images and bacterial leaf blight (BLB) rice damage assessment data were acquired during the rainy and dry seasons of 2021 and 2022 in West Java, Indonesia, where serious BLB damage occurs every year. The six-level BLB score (0, 1, 3, 5, 7, and 9) and damage intensity calculated from the score were used as the BLB damage assessment data. The relationship between normalized UAV data, normalized difference vegetation index (NDVI), and BLB score showed significant correlations at the 1% level. The analysis of damage intensities and UAV data for paddy plots in all cropping seasons showed high correlation coefficients with the normalized red band, normalized near-infrared band, and NDVI, similar to the results of the BLB score analysis. However, for paddy plots with damage intensities of 70% or higher, the biased numbering of the BLB score data may have affected the evaluation results. Therefore, we conducted an analysis using an average of 1090 survey points for each BLB score and confirmed a strong relationship, with correlation coefficients exceeding 0.9 for the normalized red band, normalized near-infrared band, and NDVI. Through comparing the time required by the current assessment method with that required by the assessment method integrating UAV data, it was demonstrated that the evaluation time was reduced by more than 60% on average. We are able to propose a new assessment method for the Indonesian government to achieve complete objective enumeration.

Interstitial lung disease (ILD) refers to a group of various abnormal inflammations of lung tissues and early diagnosis of these disease patterns is crucial for the treatment. Yet it is difficult to make an accurate diagnosis due to the similarity among the clinical manifestations of these diseases. In order to assist the radiologists, computer-aided diagnosis systems have been developed. Besides, the potential of deep convolutional neural networks (CNNs) is also expected to exert on the medical image analysis in recent years. In this paper, we design a new deep convolutional neural network (CNN) architecture to achieve the classification task of ILD patterns. Furthermore, we also propose a novel two-stage transfer learning (TSTL) method to deal with the problem of the lack of training data, which leverages the knowledge learned from sufficient textural source data and auxiliary unlabeled lung CT data to the target domain. We adopt the unsupervised manner to learn the unlabeled data, by which the objective function composed of the prediction confidence and mutual information are optimized. The experimental results show that our proposed CNN architecture achieves desirable performance and outperforms most of the state-of-the-art ones. The comparative analysis demonstrates the promising feasibility and advantages of the proposed two-stage transfer learning strategy as well as the potential of the knowledge learning from lung CT data. Graphical Abstract The framework of the proposed two-stage transfer learning method.

Early and accurate identification of brain tumors is essential for determining effective treatment strategies and improving patient outcomes. Artificial intelligence (AI) and deep learning (DL) techniques have shown promise in automating diagnostic tasks based on magnetic resonance imaging (MRI). This study evaluates the performance of four pre-trained deep convolutional neural network (CNN) architectures for the automatic multi-class classification of brain tumors into four categories: Glioma, Meningioma, Pituitary, and No Tumor. The proposed approach utilizes the publicly accessible Brain Tumor MRI Msoud dataset, consisting of 7023 images, with 5712 provided for training and 1311 for testing. To assess the impact of data availability, subsets containing 25%, 50%, 75%, and 100% of the training data were used. A stratified five-fold cross-validation technique was applied. The CNN architectures evaluated include DeiT3_base_patch16_224, Xception41, Inception_v4, and Swin_Tiny_Patch4_Window7_224, all fine-tuned using transfer learning. The training pipeline incorporated advanced preprocessing and image data augmentation techniques to enhance robustness and mitigate overfitting. Among the models tested, Swin_Tiny_Patch4_Window7_224 achieved the highest classification Accuracy of 99.24% on the test set using 75% of the training data. This model demonstrated superior generalization across all tumor classes and effectively addressed class imbalance issues. Furthermore, we deployed and benchmarked the best-performing DL model on embedded AI platforms (Jetson AGX Xavier and Orin Nano), demonstrating their capability for real-time inference and highlighting their feasibility for edge-based clinical deployment. The results highlight the strong potential of pre-trained deep CNN and transformer-based architectures in medical image analysis. The proposed approach provides a scalable and energy-efficient solution for automated brain tumor diagnosis, facilitating the integration of AI into clinical workflows.

Chaos Game Representation (CGR) is an emerging means of visualising and representing genomic and proteomic sequences. There exist many open questions related to its effective application to various computational tasks. In this work, we begin to address some of these questions by comparing four variants of the Chaos Game to generate CGR imagery as part of a multi-class classification task to identify the source organism for a given protein. We propose a novel nodal configuration for icosagon and 20-flake CGRs. Using two datasets, we performed fine-tuning using seven deep convolutional neural network (CNN) architectures and report modest performance over random among the 56 test conditions, highlighting certain shortcomings in effectively leveraging CGR in conjunction with deep CNN architectures. Many of the insights from this work will serve to orient subsequent protein-related studies involving CGR-based encoding and be generally applicable to disparate domains seeking to leverage CGR for sequence-type data.

Deep learning has already been proved as a powerful state-of-the-art technique for many image understanding tasks in computer vision and other applications including remote sensing (RS) image analysis. Unmanned aircraft systems (UASs) offer a viable and economical alternative to a conventional sensor and platform for acquiring high spatial and high temporal resolution data with high operational flexibility. Coastal wetlands are among some of the most challenging and complex ecosystems for land cover prediction and mapping tasks because land cover targets often show high intra-class and low inter-class variances. In recent years, several deep convolutional neural network (CNN) architectures have been proposed for pixel-wise image labeling, commonly called semantic image segmentation. In this paper, some of the more recent deep CNN architectures proposed for semantic image segmentation are reviewed, and each model’s training efficiency and classification performance are evaluated by training it on a limited labeled image set. Training samples are provided using the hyper-spatial resolution UAS imagery over a wetland area and the required ground truth images are prepared by manual image labeling. Experimental results demonstrate that deep CNNs have a great potential for accurate land cover prediction task using UAS hyper-spatial resolution images. Some simple deep learning architectures perform comparable or even better than complex and very deep architectures with remarkably fewer training epochs. This performance is especially valuable when limited training samples are available, which is a common case in most RS applications.

In this paper, we study the performance of variants of well-known Convolutional Neural Network (CNN) architectures on different audio tasks. We show that tuning the Receptive Field (RF) of CNNs is crucial to their generalization. An insufficient RF limits the CNN's ability to fit the training data. In contrast, CNNs with an excessive RF tend to over-fit the training data and fail to generalize to unseen testing data. As state-of-the-art CNN architectures - in computer vision and other domains - tend to go deeper in terms of number of layers, their RF size increases and therefore they degrade in performance in several audio classification and tagging tasks. We study well-known CNN architectures and how their building blocks affect their receptive field. We propose several systematic approaches to control the RF of CNNs and systematically test the resulting architectures on different audio classification and tagging tasks and datasets. The experiments show that regularizing the RF of CNNs using our proposed approaches can drastically improve the generalization of models, out-performing complex architectures and pre-trained models on larger datasets. The proposed CNNs achieve state-of-the-art results in multiple tasks, from acoustic scene classification to emotion and theme detection in music to instrument recognition, as demonstrated by top ranks in several pertinent challenges (DCASE, MediaEval).

Unmanned Aerial Vehicles (UAVs) have become increasingly popular in recent years for agricultural research. High spatial and temporal resolution images obtained with UAVs are ideal for many applications in agriculture. The objective of this study was to evaluate the performance of vegetation indices (VIs) derived from UAV images for quantification of plant nitrogen (N) concentration of spring wheat, a major cereal crop worldwide. This study was conducted at three locations in Idaho, United States. A quadcopter UAV equipped with a red edge multispectral sensor was used to collect images during the 2016 growing season. Flight missions were successfully carried out at Feekes 5 and Feekes 10 growth stages of spring wheat. Plant samples were collected on the same days as UAV image data acquisition and were transferred to lab for N concentration analysis. Different VIs including Normalized Difference Vegetative Index (NDVI), Red Edge Normalized Difference Vegetation Index (NDVIred edge), Enhanced Vegetation Index 2 (EVI2), Red Edge Simple Ratio (SRred edge), Green Chlorophyll Index (CIgreen), Red Edge Chlorophyll Index (CIred edge), Medium Resolution Imaging Spectrometer (MERIS) Terrestrial Chlorophyll Index (MTCI) and Red Edge Triangular Vegetation Index (core only) (RTVIcore) were calculated for each flight event. At Feekes 5 growth stage, red edge and green based VIs showed higher correlation with plant N concentration compare to the red based VIs. At Feekes 10 growth stage, all calculated VIs showed high correlation with plant N concentration. Empirical relationships between VIs and plant N concentration were cross validated using test data sets for each growth stage. At Feekes 5, the plant N concentration estimated based on NDVIred edge showed one to one correlation with measured N concentration. At Feekes 10, the estimated and measured N concentration were highly correlated for all empirical models, but the model based on CIgreen was the only model that had a one to one correlation between estimated and measured plant N concentration. The observed high correlations between VIs derived from UAV and the plant N concentration suggests the significance of VIs deriving from UAVs for within-season N concentration monitoring of agricultural crops such as spring wheat.

Recognition of human activity based on convolutional neural network has received the interest of researchers in recent years due to its significant improvement in accuracy. A large number of algorithms based on the deep learning approach have been proposed for activity recognition purpose. However, with the increasing advancements in technologies having limited computational resources, it needs to design an efficient deep learning-based approaches with improved utilization of computational resources. This paper presents a simple and efficient 2-Dimensional convolutional neural network (2-D CNN) architecture with very small size convolutional kernel for human activity recognition. The merit of the proposed CNN architecture over standard deep learning architectures is fewer trainable parameters and lesser memory requirement which enables it to train the proposed CNN architecture on low GPU memory-based devices and also works well with smaller as well as larger size datasets. The proposed approach consists of mainly four stages: namely (1) creation of dataset and data augmentation, (2) designing 2-D convolutional neural network (CNN) architecture, (3) the proposed 2-D CNN architecture trained from scratch up to optimum stage, and (4) evaluation of the trained 2-D CNN architecture. To illustrate the effectiveness of the proposed architecture several extensive experiments are conducted on three publicly available datasets, namely IXMAS, YouTube, and UCF101 dataset. The results of the proposed method and its comparison with other state-of-the-art methods [8-12,14,18-26,29-33] demonstrate the usefulness of the proposed method.

The Normalized Difference Vegetation Index (NDVI) and Enhanced Vegetation Index (EVI) are vegetation indices widely used in remote sensing of above-ground biomass. Because both indexes are based on spectral features of plant canopy, NDVI and EVI may suffer reduced accuracy in estimating above-ground biomass when flower signals are mixed in the plant canopy. This paper addresses how flowers influence the estimation of above-ground biomass using NDVI and EVI for an alpine meadow with mixed yellow flowers of Halerpestes tricuspis (Ranunculaceae). Field spectral measurements were used in combination with simulated reflectance spectra with precisely controlled flower coverage by applying a linear spectral mixture model. Using the reflectance spectrum for the in-situ canopy with H. tricuspis flowers, we found no significant correlation between above-ground biomass and EVI (p = 0.17) or between above-ground biomass and NDVI (p = 0.78). However, both NDVI and EVI showed very good prediction of above-ground biomass with low root mean square errors (RMSE = 43 g m−2 for NDVI and RMSE = 43 g m−2 for EVI, both p < 0.01) when all the flowers were removed from the canopies. Simulation analysis based on the in-situ measurements further indicated that high variation in flower coverage among different quadrats could produce more noise in the relationship between above-ground biomass and NDVI, or EVI, which results in an evident decline in the accuracy of above-ground biomass estimation. Therefore, the study suggests that attention should be paid both to the flower fraction and the heterogeneity of flower distribution in the above-ground biomass estimation via NDVI and EVI.

Aboveground biomass (AGB) serves as a crucial indicator of the carbon sequestration capacity of coastal wetland ecosystems. Conducting extensive field surveys in coastal wetlands is both time-consuming and labor-intensive. Unmanned aerial vehicles (UAVs) and satellite remote sensing have been widely utilized to estimate regional AGB. However, the mixed pixel effects in satellite remote sensing hinder the precise estimation of AGB, while high-spatial resolution UAVs face challenges in estimating large-scale AGB. To fill this gap, this study proposed an integrated approach for estimating AGB using field sampling, a UAV, and Sentinel-2 satellite data. Firstly, based on multispectral data from the UAV, vegetation indices were computed and matched with field sampling data to develop the Field–UAV AGB estimation model, yielding AGB results at the UAV scale (1 m). Subsequently, these results were upscaled to the Sentinel-2 satellite scale (10 m). Vegetation indices from Sentinel-2 data were calculated and matched to establish the UAV–Satellite AGB model, enabling the estimation of AGB over large regional areas. Our findings revealed the AGB estimation model achieved an R2 value of 0.58 at the UAV scale and 0.74 at the satellite scale, significantly outperforming direct modeling from field data to satellite (R2 = −0.04). The AGB densities of the wetlands in Xieqian Bay, Meishan Bay, and Hangzhou Bay, Zhejiang Province, were 1440.27 g/m2, 1508.65 g/m2, and 1545.11 g/m2, respectively. The total AGB quantities were estimated to be 30,526.08 t, 34,219.97 t, and 296,382.91 t, respectively. This study underscores the potential of integrating UAV and satellite remote sensing for accurately assessing AGB in large coastal wetland regions, providing valuable support for the conservation and management of coastal wetland ecosystems.

Vegetation indices (VIs) such as the Normalized Difference Vegetation Index (NDVI) are widely used for assessing vegetation cover and condition. One of the NDVI's significant disadvantages is its sensitivity to aerosols in the atmosphere, hence several atmospherically resistant VIs were formulated using the difference in the radiance between the blue and the red spectral bands. The state‐of‐the‐art atmospherically resistant VI, which is a standard Moderate Resolution Imaging Spectroradiometer (MODIS) product, together with the NDVI, is the Enhanced Vegetation Index (EVI). A different approach introduced the Aerosol‐free Vegetation Index (AFRI) that is based on the correlation between the shortwave infrared (SWIR) and the visible red bands. The AFRI main advantage is in penetrating an opaque atmosphere influenced by biomass burning smoke, without the need for explicit correction for the aerosol effect. The objective of this research was to compare the performance of these three VIs under smoke conditions. The AFRI was applied to the 2.1 µm SWIR channel of the MODIS sensor onboard the Earth Observing System (EOS) Terra and Aqua satellites in order to assess its functionality on these imaging platforms. The AFRI performance was compared with those of NDVI and EVI. All VIs were calculated on images with and without present smoke, using the surface‐reflectance MODIS product, for three case studies of fires in Arizona, California, and Zambia. The MODIS Fire Product was embedded on the images in order to identify the exact location of the active fires. Although good correlations were observed between all VIs in the absence of smoke (in the Arizona case R 2 = 0.86, 0.77, 0.88 for the NDVI–EVI, AFRI–EVI, and AFRI–NDVI, respectively) under smoke conditions a high correlation was maintained between the NDVI and the EVI, while low correlations were found for the AFRI–EVI and AFRI–NDVI (0.21 and 0.16, for the Arizona case, respectively). A time series of MODIS images recorded over Zambia during the summer of 2000 was tested and showed high NDVI fluctuations during the study period due to oscillations in aerosol optical thickness values despite application of aerosol corrections on the images. In contrast, the AFRI showed smoother variations and managed to better assess the vegetation condition. It is concluded that, beneath the biomass burning smoke, the AFRI is more effective than the EVI in observing the vegetation conditions.