Novel Land Cover Change Detection Deep Learning Framework with Very Small Initial Samples Using Heterogeneous Remote Sensing Images

AN ASSESSMENT OF SPATIAL VARIATION OF LAND SURFACE CHARACTERISTICS OF MINNA, NIGER STATE NIGERIA FOR SUSTAINABLE URBANIZATION USING GEOSPATIAL TECHNIQUES

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions.

Deep learning methods have become popular among researchers in the field of fault detection. However, their performance depends on the availability of big datasets. To overcome this problem researchers started applying transfer learning to achieve good performance from small available datasets, by leveraging multiple prediction models over similar machines and working conditions. However, the influence of negative transfer limits their application. Negative transfer among prediction models increases when the environment and working conditions are changing continuously. To overcome the effect of negative transfer, we propose a novel deep transfer learning method, coined deep boosted transfer learning, for wind turbine gearbox fault detection that prevents negative transfer and only focuses on relevant information from the source machine. The proposed method is an instance-based deep transfer learning method that updates the weights of the source and the target machine training samples separately. The weights of different source training samples are gradually decreased to reduce the impact on the final model. The proposed method is verified by the Case Western Reserve University bearing and real field wind farm datasets. The results show that the proposed method ignores negative transfer and achieves higher accuracy compared to standard deep learning and deep transfer learning methods.

This paper proposes an automatic method for land cover change detection in very-high-spatial-resolution optical remote sensing images based on the automatic selection of training samples using expectation-maximization (EM) and an extreme learning machine classifier, which has two key characteristics: (1) combining the advantages of supervised and unsupervised methods with progressive mask classification. (2) training samples of three classes (changed, unchanged, and unknown) were automatically selected by K-Means and EM, and then further refined by the likelihood function. The method was validated by one dataset of SuperView-1 imagery with a spatial resolution of 2.0 m, two datasets of TripleSat-2 imagery with a spatial resolution of 3.2 m, and one open dataset of Zi-Yuan-3 imagery with a spatial resolution of 5.8 m, and the results were compared with that of three unsupervised methods (iterative slow feature analysis, multivariate alteration detection, and adaptive object-oriented spatial-contextual extraction algorithm), two deep learning methods (convolutional-wavelet neural networks and dual-domain networks), and three supervised classifiers (support vector machine, random forest, and Naive Bayes), showing the effectiveness of this method in decrease of false-positive rates and increase of change detection accuracy. The average and maximum of the accuracy metric F1 score of our method are 0.6842 and 0.8708, respectively; the average F1 score of the unsupervised, deep learning, and supervised methods are 0.5049, 0.4943, and 0.633, respectively.

Deep learning technology plays an important role in land cover change detection (LCCD) with bitemporal remote sensing images (RSIs). In particular, the classical UNet can provide promising detection accuracies and visual performance for LCCD with RSIs. This study proposes a novel change detection approach named difference feature guidance and multi-scale feature-fusion (DFG-MFF) network based on UNet to further improve the performance of LCCD with RSIs. First, a difference feature module is adopted to guide the change feature learning process. A multi-scale fusion module is then developed to capture the multi-scale features of ground targets with various shapes and sizes. Finally, a composite loss function composed of dice loss function, focal loss function and mean squared error loss function is utilized to train the DFG-MFF network. Experiments are performed on one pairwise homogeneous dataset and two pairs of heterogeneous datasets, and the comparison with eight state-of-the-art methods well demonstrated the feasibility and advantages of the proposed network for LCCD with RSIs.

Deep learning methods are widely used in the field of rolling bearing fault diagnosis and produce good results when faced with datasets with roughly equal numbers of normal and faulty samples. However, real-world data often has a serious imbalance, with the number of fault samples being significantly less than the number of normal samples. This dataset imbalance challenges the performance of traditional deep learning methods. To address this problem, this paper proposes an efficient imbalanced data rolling bearing fault diagnosis method. The method consists of two parts: a deep learning network based on a multi-scale self-attention mechanism and a novel loss function. In terms of the deep learning network, firstly, the one-dimensional vibration signal is converted into a two-dimensional image through the Gramian angular field. This conversion maximises the inherent feature extraction capability of the network. Subsequently, the multi-scale learning capability of the network is enhanced by implementing different expansion rates for the head of the multi-scale self-attention mechanism. This nuanced approach allows the network to capture the underlying information more efficiently. Finally, the inclusion of Ghost bottlenecks and feature pyramid networks (FPNs) helps to optimise network efficiency and improve generalisation performance. A novel loss function is also proposed to make the method more suitable for imbalanced data. During the training process, the classification of samples in each class is analysed using the recall metric of imbalanced classification and the real-time recall is used as a weight to weaken the dominance of the majority class. This weighting ensures the adaptability of the method to imbalanced datasets. The proposed method is evaluated using rolling bearing datasets from Case Western Reserve University, USA, and Southeast University, China. Comparison results with other state-of-the-art deep learning methods show that the proposed method has a robust performance when dealing with imbalanced data.

Deep learning-based land cover and land use (LCLU) classification systems are a significant aspiration for remote sensing communities. In nature, remote sensing images have various properties that need to be analyzed. Analyzing and interpreting image properties is difficult due to the nature of the image, the sensor technology’s capability, and other determinant variables such as seasons and weather conditions. The problem is essential for environmental monitoring, agricultural decision-making, and urban planning if it can be supported by deep learning systems. Therefore, deep learning approaches are proposed to quickly analyze and interpret the remote sensing image to classify the LCLU. The deep learning methods could be designed starting from scratch or using pre-trained networks. However, there are few comparisons of deep learning methods developed from scratch and trained on pre-trained networks. Thus, we proposed evaluating and comparing the deep learning models convolutional neural network feature extractor (CNN-FE) by developing it from scratch, transfer learning, and fine-tuning it for the LCLU classification system using remote sensed images. Using CNN-FE, TL, and fine-tuning deep learning models as examples, this paper compares and analyzes deep learning algorithms for remote sensed image classification. After developing and training each deep learning model on the UCM dataset, we evaluated and compared their performances using the performance measurement metrics accuracy, precision, recall, f1-score, and confusion matrix. The proposed deep learning algorithms can adapt and learn the features of the remote sensing images, and the TL and fine-tuning classification performances are significantly improved. As a result of the efficient time used for training the models, this paper discovered that the fine-tuned deep learning model achieved profound accuracy performance results in the UCM dataset.

Less than 10% of patients with cancer have detectable pathogenic germline alterations, which may be partially due to incomplete pathogenic variant detection. To evaluate if deep learning approaches identify more germline pathogenic variants in patients with cancer. A cross-sectional study of a standard germline detection method and a deep learning method in 2 convenience cohorts with prostate cancer and melanoma enrolled in the US and Europe between 2010 and 2017. The final date of clinical data collection was December 2017. Germline variant detection using standard or deep learning methods. The primary outcomes included pathogenic variant detection performance in 118 cancer-predisposition genes estimated as sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The secondary outcomes were pathogenic variant detection performance in 59 genes deemed actionable by the American College of Medical Genetics and Genomics (ACMG) and 5197 clinically relevant mendelian genes. True sensitivity and true specificity could not be calculated due to lack of a criterion reference standard, but were estimated as the proportion of true-positive variants and true-negative variants, respectively, identified by each method in a reference variant set that consisted of all variants judged to be valid from either approach. The prostate cancer cohort included 1072 men (mean [SD] age at diagnosis, 63.7 [7.9] years; 857 [79.9%] with European ancestry) and the melanoma cohort included 1295 patients (mean [SD] age at diagnosis, 59.8 [15.6] years; 488 [37.7%] women; 1060 [81.9%] with European ancestry). The deep learning method identified more patients with pathogenic variants in cancer-predisposition genes than the standard method (prostate cancer: 198 vs 182; melanoma: 93 vs 74); sensitivity (prostate cancer: 94.7% vs 87.1% [difference, 7.6%; 95% CI, 2.2% to 13.1%]; melanoma: 74.4% vs 59.2% [difference, 15.2%; 95% CI, 3.7% to 26.7%]), specificity (prostate cancer: 64.0% vs 36.0% [difference, 28.0%; 95% CI, 1.4% to 54.6%]; melanoma: 63.4% vs 36.6% [difference, 26.8%; 95% CI, 17.6% to 35.9%]), PPV (prostate cancer: 95.7% vs 91.9% [difference, 3.8%; 95% CI, -1.0% to 8.4%]; melanoma: 54.4% vs 35.4% [difference, 19.0%; 95% CI, 9.1% to 28.9%]), and NPV (prostate cancer: 59.3% vs 25.0% [difference, 34.3%; 95% CI, 10.9% to 57.6%]; melanoma: 80.8% vs 60.5% [difference, 20.3%; 95% CI, 10.0% to 30.7%]). For the ACMG genes, the sensitivity of the 2 methods was not significantly different in the prostate cancer cohort (94.9% vs 90.6% [difference, 4.3%; 95% CI, -2.3% to 10.9%]), but the deep learning method had a higher sensitivity in the melanoma cohort (71.6% vs 53.7% [difference, 17.9%; 95% CI, 1.82% to 34.0%]). The deep learning method had higher sensitivity in the mendelian genes (prostate cancer: 99.7% vs 95.1% [difference, 4.6%; 95% CI, 3.0% to 6.3%]; melanoma: 91.7% vs 86.2% [difference, 5.5%; 95% CI, 2.2% to 8.8%]). Among a convenience sample of 2 independent cohorts of patients with prostate cancer and melanoma, germline genetic testing using deep learning, compared with the current standard genetic testing method, was associated with higher sensitivity and specificity for detection of pathogenic variants. Further research is needed to understand the relevance of these findings with regard to clinical outcomes.

The inverter is the most vulnerable module of photovoltaic (PV) systems. The insulated gate bipolar transistor (IGBT) is the core part of inverters and the root source of PV inverter failures. How to effectively diagnose the IGBT faults is critical for reliability, high efficiency, and safety of PV systems. Recently, deep learning (DL) methods are widely used for fault detection and diagnosis. Different from traditional diagnosis methods, DL methods use deep neural networks which can automatically extract the useful representative features from raw data. However, DL methods require large amounts of data, which leads to the high cost of communication, storage, and computation. To tackle these issues, a data-driven fault detection and diagnosis method for IGBT open-circuit faults based on compressed sensing (CS) and convolutional neural networks (CNN) is proposed in this paper. CS is adopted to compress raw signals, and the optimal value of compression ratio (CR) is determined by considering the trade-off between classification accuracy and model training time. The overlap sampling method is adopted for data segmentation. Meanwhile, overlap sampling can also increase the number of training samples and improve the sample correlation. The compressed signals are segmented and reconstructed into two-dimensional feature maps for model training. Finally, compared with CNN of the same structure, the developed CS-CNN model can compress 85% of data without accuracy loss. The performance comparison with the state-of-the-art networks demonstrates that the test accuracy is 98.68% and the model training time is much shorter than other methods.

In order to ensure the safe and stable operation of power system, enrich the means of power grid analysis and control, and expand the application of deep intelligent learning methods in power grid systems, the application of deep learning intelligent machine learning method in frequency prediction of large power grid is explored. First, on the basis of deep learning, the frequency response mode of large power grid is analyzed and the key characteristic quantities that affect the frequency response mode are extracted. Second, the deep belief neural network (DBN. DNN) frequency prediction model is constructed. Also, the training and testing of the model are introduced. Finally, the input and output based on the DBN.CNN prediction model and the network structure design of the model are analyzed. The prediction performance of the model is evaluated. The results show that when the number of neurons in the hidden layer is 50, the model achieves the optimal prediction effect. Increasing the number of training samples helps to improve the modeling ability and prediction accuracy of the model. For frequency prediction problems, the number of training samples should be set to ≥400, and the number of hidden layers corresponding to the model should be 5. When the number of hidden layer neurons is 10, the prediction accuracy of the DBN/DNN network is poor. When the number of hidden layer neurons is 50, the model can achieve the best prediction effect. Overall, the DBN.DNN prediction model has good prediction performance. The RMSE of the forecast data is O. 0073Hz can basically meet the actual application requirements. Therefore, the frequency prediction method based on deep belief neural network has certain advantages in accuracy and efficiency.

Land cover change detection (LCCD) based on bi-temporal remote sensing images plays an important role in the inventory of land cover change. Due to the benefit of having spatial dependency properties within the image space while using remote sensing images for detecting land cover change, many contextual information-based change detection methods have been proposed in past decades. However, there is still a space for improvement in accuracies and usability of LCCD. In this paper, a LCCD method based on adaptive contextual information is proposed. First, an adaptive region is constructed by gradually detecting the spectral similarity surrounding a central pixel. Second, the Euclidean distance between pairwise extended regions is calculated to measure the change magnitude between the pairwise central pixels of bi-temporal images. All the bi-temporal images are scanned pixel by pixel so the change magnitude image (CMI) can be generated. Then, the Otsu or a manual threshold is employed to acquire the binary change detection map (BCDM). The detection accuracies of the proposed approach are investigated by three land cover change cases with Landsat bi-temporal remote sensing images and aerial images with very high spatial resolution (0.5 m/pixel). In comparison to several widely used change detection methods, the proposed approach can produce a land cover change inventory map with a competitive accuracy.

Deep learning has gained huge traction in recent years because of its potential to make informed decisions. A large portion of today’s deep learning systems are based on centralized servers and fall short in providing operational transparency, traceability, reliability, security, and trusted data provenance features. Also, training deep learning models by utilizing centralized data is vulnerable to the single point of failure problem. In this paper, we explore the importance of integrating blockchain technology with deep learning. We review the existing literature focused on the integration of blockchain with deep learning. We classify and categorize the literature by devising a thematic taxonomy based on seven parameters; namely, blockchain type, deep learning models, deep learning specific consensus protocols, application area, services, data types, and deployment goals. We provide insightful discussions on the state-of-the-art blockchain-based deep learning frameworks by highlighting their strengths and weaknesses. Furthermore, we compare the existing blockchain-based deep learning frameworks based on four parameters such as blockchain type, consensus protocol, deep learning method, and dataset. Finally, we present important research challenges which need to be addressed to develop highly trustworthy deep learning frameworks.

This paper provides a comprehensive comparative study of traditional, hybrid, and deep learning (DL) methods for detecting marine species in echograms. Acoustic backscatter data obtained from multi-frequency echosounders is visualized as echograms and typically interpreted by marine biologists via manual or semi-automatic methods, which are time-consuming. Challenges related to automatic echogram interpretation are the variable size and acoustic properties of the biological targets (marine life), along with significant inter-class similarities. Our study explores and compares three types of approaches that cover the entire range of machine learning methods. Based on our experimental results, we conclude that an end-to-end DL-based framework, that can be readily scaled to accommodate new species, is overall preferable to other learning approaches for echogram interpretation, even when only a limited number of annotated training samples is available.

The clinical utility of regional strain measurements in echocardiography is challenged by suboptimal reproducibility. In this study, we aimed to evaluate the test-retest reproducibility of regional longitudinal strain (RLS) per coronary artery perfusion territory (RLSTerritory) and basal-to-apical level of the left ventricle (RLSLevel), measured by a novel fully automated deep learning (DL) method based on point tracking. We measured strain in a dual-centre test-retest data set that included 40 controls and 40 patients with suspected non-ST elevation acute coronary syndrome. Two consecutive echocardiograms per subject were recorded by different operators. The reproducibility of RLSTerritory and RLSLevel measured by the DL method and by three experienced observers using semi-automatic software (2D Strain, EchoPAC, GE HealthCare) was evaluated as minimal detectable change (MDC). The DL method had MDC for RLSTerritory and RLSLevel ranging from 3.6 to 4.3%, corresponding to a 33-35% improved reproducibility compared with the inter- and intraobserver scenarios (MDC 5.5-6.4% and 4.9-5.4%). Furthermore, the DL method had a lower variance of test-retest differences for both RLSTerritory and RLSLevel compared with inter- and intraobserver scenarios (all P < 0.001). Bland-Altman analyses demonstrated superior reproducibility by the DL method for the whole range of strain values compared with the best observer scenarios. The feasibility of the DL method was 93% and measurement time was only 1 s per echocardiogram. The novel DL method provided fully automated measurements of RLS, with improved test-retest reproducibility compared with semi-automatic measurements by experienced observers. RLS measured by the DL method has the potential to advance patient care through a more detailed, more efficient, and less user-dependent clinical assessment of myocardial function.

This review paper examines the current state of lung disease diagnosis based on deep learning (DL) methods. Lung diseases, such as Pneumonia, TB, Covid-19, and lung cancer, are significant causes of morbidity and mortality worldwide. Accurate and timely diagnosis of these diseases is essential for effective treatment and improved patient outcomes. DL methods, which utilize artificial neural networks to extract features from medical images automatically, have shown great promise in improving the accuracy and efficiency of lung disease diagnosis. This review discusses the various DL methods that have been developed for lung disease diagnosis, including convolutional neural networks (CNNs), deep neural networks (DNNs), and generative adversarial networks (GANs). The advantages and limitations of each method are discussed, along with the types of medical imaging techniques used, such as X-ray and computed tomography (CT). In addition, the review discusses the most commonly used performance metrics for evaluating the performance of DL for lung disease diagnosis: the area under the curve (AUC), sensitivity, specificity, F1-score, accuracy, precision, and the receiver operator characteristic curve (ROC). Moreover, the challenges and limitations of using DL for lung disease diagnosis, including the limited availability of annotated data, the variability in imaging techniques and disease presentation, and the interpretability and generalizability of DL models, are highlighted in this paper. Furthermore, strategies to overcome these challenges, such as transfer learning, data augmentation, and explainable AI, are also discussed. The review concludes with a call for further research to address the remaining challenges and realize DL's full potential for improving lung disease diagnosis and treatment.