Deep Learning Models Used for Crop Analysis: A Review

Crop yield prediction is becoming increasingly crucial due to global food security concerns, as highlighted by recent reports from the World Health Organization. Accurate early predictions can mitigate famine risks by estimating food supply, which is essential for 820 million people facing hunger globally. Rice is the primary staple food consumed worldwide; therefore, global rice yield and rice area are monitored using emerging technologies such as remote sensing (RS) and machine learning (ML). These technologies provide valuable tools for enhancing rice yield predictions. RS includes critical information on crop health, soil conditions and weather patterns. In contrast, ML algorithms analyze these datasets to identify patterns and relationships that affect yield. Integrating these technologies offers promising improvements in yield forecasting accuracy, with applications showing successful yield predictions 1-3 months before harvest. Various ML techniques, including Random Forest, Support Vector Machines and deep learning models such as LSTM (Long-Short Term Memory), have been employed, often in combination with RS data. However, these models face challenges, such as data quality, managing high-dimensional RS data and accounting for spatial and temporal variability. Despite these challenges, integrating RS and ML has significant potential for advancing precision agriculture and achieving sustainable food production. This study explores the advancements, practical applications and challenges associated with using RS and ML for rice yield prediction, emphasizing the importance of these technologies in addressing global food security and promoting sustainable agricultural practices.

Deep learning-based land cover and land use (LCLU) classification systems are a significant aspiration for remote sensing communities. In nature, remote sensing images have various properties that need to be analyzed. Analyzing and interpreting image properties is difficult due to the nature of the image, the sensor technology’s capability, and other determinant variables such as seasons and weather conditions. The problem is essential for environmental monitoring, agricultural decision-making, and urban planning if it can be supported by deep learning systems. Therefore, deep learning approaches are proposed to quickly analyze and interpret the remote sensing image to classify the LCLU. The deep learning methods could be designed starting from scratch or using pre-trained networks. However, there are few comparisons of deep learning methods developed from scratch and trained on pre-trained networks. Thus, we proposed evaluating and comparing the deep learning models convolutional neural network feature extractor (CNN-FE) by developing it from scratch, transfer learning, and fine-tuning it for the LCLU classification system using remote sensed images. Using CNN-FE, TL, and fine-tuning deep learning models as examples, this paper compares and analyzes deep learning algorithms for remote sensed image classification. After developing and training each deep learning model on the UCM dataset, we evaluated and compared their performances using the performance measurement metrics accuracy, precision, recall, f1-score, and confusion matrix. The proposed deep learning algorithms can adapt and learn the features of the remote sensing images, and the TL and fine-tuning classification performances are significantly improved. As a result of the efficient time used for training the models, this paper discovered that the fine-tuned deep learning model achieved profound accuracy performance results in the UCM dataset.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

A novel deep learning change detection approach for estimating spatiotemporal crop field variations from Sentinel-2 imagery

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

In hydrology, deep learning (DL) models have already achieved remarkable breakthroughs in predicting streamflow. These DL models are fed with meteorological time-series and static catchment attributes across large samples of catchments, and predict streamflow remarkably well in both gauged and ungauged situations. In recent years, some studies have transferred the idea of constructing multi-basin/station DL models – particularly Long-Short Term Memory (LSTM) neural networks – to large-sample groundwater level modelling to explore their potential for temporal and spatial extrapolation. To the best of authors’ knowledge, existing multi-station LSTM applications are limited to three, covering 76 climate-sensitive stations in Northern France, 108 nationwide stations in Germany, and 1,800 coastal stations across nine countries/regions. Notably, spatial generalisation was investigated solely in the German study, which suggests that the model utilised static features primarily as ‘unique identifiers’ to memorise local behaviour rather than deriving the generalisable hydrological insights required for spatial extrapolation. Given the limited number of studies and the potentially biased datasets, the generalisation ability of multi-station DL models for groundwater level modelling is still under exploration.A newly released large-sample groundwater dataset by the Environment Agency of England, comprising more than 200,000 daily and 200 million sub-daily sampling observations for over 3,400 wells, offers a unique opportunity to test the generalisation ability of multi-station DL models in time and space, and whether these models can yield process-relevant insights on groundwater dynamic mechanisms. In this study, we want to investigate the following questions:1) How well can multi-station DL models simulate the groundwater variability across England?2) Which input features does the DL model use to make its predictions (especially in places where it does well)?

We developed a multimodal ultrasound (US) deep learning (DL) fusion model to automatically classify early fibrosis in patients with chronic kidney disease (CKD). This prospective study included patients with CKD who underwent continuous gray-scale US, superb microvascular imaging, and strain elastography from May to November 2022. According to the pathological tubular atrophy and interstitial fibrosis score, patients were divided into minimal and mild groups (affected area ≤10% and 11 − 25% of the total cortical volume, respectively). The dataset was divided into training (70%) and test (30%) sets. A DL model combining the features of the three US modes was developed to predict early fibrosis in patients with CKD. We compared these findings with the area under the receiver operating characteristic curve (AUC) of the clinical model by analyzing the receiver operating characteristic curve in the test set. The AUC of single-mode DL based on gray-scale US, superb microvascular imaging, and strain elastography was 0.682, 0.745, and 0.648, respectively, while that of the multimodal US DL model was 0.86. The accuracy, specificity, and sensitivity of the multimodal US DL model were 0.779, 0.767, and 0.796, respectively, and the negative and positive predictive values were 0.842 and 0.706, respectively. The AUC of the multimodal US DL model was significantly better than that of the single-mode DL and clinical models. The DL algorithm developed using multimodal US images can effectively predict early fibrosis in patients with CKD with significantly greater accuracy than single-mode DL or clinical models.

In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively. Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition DOI: 10.25165/j.ijabe.20181104.4475 Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

The output of a deep-learning model delivers different predictions depending on the input of the deep learning model. In particular, the input characteristics might affect the output of a deep learning model. When predicting data that are measured with sensors in multiple locations, it is necessary to train a deep learning model with spatiotemporal characteristics of the data. Additionally, since not all of the data measured together result in increasing the accuracy of the deep learning model, we need to utilize the correlation characteristics between the data features. However, it is difficult to interpret the deep learning output, depending on the input characteristics. Therefore, it is necessary to analyze how the input characteristics affect prediction results to interpret deep learning models. In this paper, we propose a visualization system to analyze deep learning models with air pollution data. The proposed system visualizes the predictions according to the input characteristics. The input characteristics include space-time and data features, and we apply temporal prediction networks, including gated recurrent units (GRU), long short term memory (LSTM), and spatiotemporal prediction networks (convolutional LSTM) as deep learning models. We interpret the output according to the characteristics of input to show the effectiveness of the system.

We aimed at comparing intratumoral and peritumoral deep learning, radiomics, and fusion models in predicting KRAS mutations in rectal cancer using endorectal ultrasound imaging. This study included 304 patients with rectal cancer from Fujian Medical University Union Hospital. The patients were randomly divided into a training group (213 patients) and a test group (91 patients) at a 7:3 ratio. Radiomics and deep learning models were established using primary tumor and peritumoral images. In the optimally performing regions-of-interest, two fusion strategies, a feature-based and a decision-based model, were employed to build the fusion models. The Shapley additive explanation (SHAP) method was used to evaluate the significance of features in the optimal radiomics, deep learning, and fusion models. The performance of each model was assessed using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). In the test cohort, both the radiomics and deep learning models exhibited optimal performance with a 10-pixel patch extension, yielding AUC values of 0.824 and 0.856, respectively. The feature-based DLRexpand10_FB model attained the highest AUC (0.896) across all study sets. In addition, the DLRexpand10_FB model demonstrated excellent sensitivity, specificity, and DCA. SHAP analysis underscored the deep learning feature (DL_1) as the most significant factor in the hybrid model. The feature-based fusion model DLRexpand10_FB can be employed to predict KRAS gene mutations based on pretreatment endorectal ultrasound images of rectal cancer. The integration of peritumoral regions enhanced the predictive performance of both the radiomics and deep learning models.

Deep learning breast cancer risk models demonstrate improved accuracy compared with traditional risk models but have not been prospectively tested. We compared the accuracy of a deep learning risk score derived from the patient's prior mammogram to traditional risk scores to prospectively identify patients with cancer in a cohort due for screening. We collected data on 119 139 bilateral screening mammograms in 57 617 consecutive patients screened at 5 facilities between September 18, 2017, and February 1, 2021. Patient demographics were retrieved from electronic medical records, cancer outcomes determined through regional tumor registry linkage, and comparisons made across risk models using Wilcoxon and Pearson χ2 2-sided tests. Deep learning, Tyrer-Cuzick, and National Cancer Institute Breast Cancer Risk Assessment Tool (NCI BCRAT) risk models were compared with respect to performance metrics and area under the receiver operating characteristic curves. Cancers detected per thousand patients screened were higher in patients at increased risk by the deep learning model (8.6, 95% confidence interval [CI] = 7.9 to 9.4) compared with Tyrer-Cuzick (4.4, 95% CI = 3.9 to 4.9) and NCI BCRAT (3.8, 95% CI = 3.3 to 4.3) models (P &lt; .001). Area under the receiver operating characteristic curves of the deep learning model (0.68, 95% CI = 0.66 to 0.70) was higher compared with Tyrer-Cuzick (0.57, 95% CI = 0.54 to 0.60) and NCI BCRAT (0.57, 95% CI = 0.54 to 0.60) models. Simulated screening of the top 50th percentile risk by the deep learning model captured statistically significantly more patients with cancer compared with Tyrer-Cuzick and NCI BCRAT models (P &lt; .001). A deep learning model to assess breast cancer risk can support feasible and effective risk-based screening and is superior to traditional models to identify patients destined to develop cancer in large screening cohorts.

Predicting crop yield based on the environmental, soil, water and crop parameters has been a potential research topic. Deep-learning-based models are broadly used to extract significant crop features for prediction. Though these methods could resolve the yield prediction problem there exist the following inadequacies: Unable to create a direct non-linear or linear mapping between the raw data and crop yield values; and the performance of those models highly relies on the quality of the extracted features. Deep reinforcement learning provides direction and motivation for the aforementioned shortcomings. Combining the intelligence of reinforcement learning and deep learning, deep reinforcement learning builds a complete crop yield prediction framework that can map the raw data to the crop prediction values. The proposed work constructs a Deep Recurrent Q-Network model which is a Recurrent Neural Network deep learning algorithm over the Q-Learning reinforcement learning algorithm to forecast the crop yield. The sequentially stacked layers of Recurrent Neural network is fed by the data parameters. The Q- learning network constructs a crop yield prediction environment based on the input parameters. A linear layer maps the Recurrent Neural Network output values to the Q-values. The reinforcement learning agent incorporates a combination of parametric features with the threshold that assist in predicting crop yield. Finally, the agent receives an aggregate score for the actions performed by minimizing the error and maximizing the forecast accuracy. The proposed model efficiently predicts the crop yield outperforming existing models by preserving the original data distribution with an accuracy of 93.7%.

Monte Carlo (MC)-based estimation of dose-averaged linear energy transfer (LETd) in proton therapy is accurate but computationally intensive. Although deep learning (DL) models offer efficient alternatives, few have been optimized for multi-source inputs or validated on anatomically complex tumors, such as nasopharyngeal carcinoma (NPC). This study developed and evaluated a multi-source DL framework for voxel-level LETd prediction in proton therapy for NPC, aiming to identify optimal input configurations and assess both prediction accuracy and clinical reliability. This study analyzed 100 patients with NPC, yielding 500 proton therapy fields. Ground-truth LETd maps were generated using GPU-based Monte Carlo simulations. A deep learning model was trained with different combinations of four input features: dose distribution, CT image, beam mask, and radiological depth, resulting in eight configurations (Mdose, Mdose+ct, Mdose+beam, Mdose+depth, Mdose+beam+depth, Mdose+ct+depth, Mdose+ct+beam, and Mdose+ct+beam+depth). The dataset was randomly divided into training (300 fields, 60 patients), validation (100 fields, 20 patients), and testing (100 fields, 20 patients) sets. Model performance was evaluated using mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), structural similarity index measure (SSIM), and 3D Gamma analysis. A 3D U-Net baseline was retrained under identical conditions for comparison. Model interpretability was examined with Gradient-weighted Class Activation Mapping (Grad-CAM), and prediction confidence was assessed using uncertainty maps. The Mdose+ct model achieved the best performance (MAE 0.21±0.03keV/µm, PSNR 28.99±1.33, NCC 0.98±0.01, SSIM 0.971±0.007, Gamma 3%/3mm: 94.69±3.69%), outperforming the 3D U-Net baseline (MAE 0.38±0.52keV/µm). Grad-CAM visualizations confirmed that the model focused on physically relevant regions, while uncertainty maps highlighted high-variability zones at LETd transitions. The model generated voxel-level LETd predictions in approximately 7 s per case, which was substantially faster than GPU-based MC simulations (30 s per case). The proposed multi-source deep learning framework enables rapid and accurate voxel-level LETd prediction for NPC. Combining dose with CT yielded the most reliable input configuration, and incorporating interpretability and uncertainty analyses enhances confidence in its suitability for biologically guided proton therapy planning.

Satellite remote sensing imagery is a technology that captures images of the Earth's surface using sensors mounted on satellites in Earth's orbit. It plays a vital role in numerous fields, including geographic information systems, environmental monitoring and protection, urban planning and management, disaster response, and climate change research. The versatility and efficiency of satellite remote sensing imagery make it indispensable in modern technology, providing both macroscopic and microscopic insights into Earth observation. With advancements in remote sensing technology, its applications continue to expand. However, in practice, there are discrepancies between the quality of sample data derived from satellite remote sensing imagery and real-world conditions. These discrepancies may arise from cloud cover, sensor noise, low resolution, seasonal variations, or environmental changes. For example, cloud occlusion often leads to invalid pixel regions, while time-phase mismatches and poor raw image quality severely impact downstream processing. In water environment monitoring, limitations such as rapid temporal changes in water bodies further complicate data acquisition and analysis. To address these challenges, researchers have proposed robust solutions to enhance the quality and reliability of remote sensing image processing.