Remote Sensing Image Change Detection Based on Deep Siamese Neural Network with Convolutional Lstm and Channel Attention

A multi-resolution approach for spinal metastasis detection using deep Siamese neural networks

Deep learning has substantially pushed forward remote sensing image change detection through extracting discriminative hierarchical features. However, as the increasingly high resolution remote sensing images have abundant spatial details but limited spectral information, the use of conventional backbone networks would give rise to blurry boundaries between different semantics among hierarchical features. This explains why most false alarms in the final predictions distribute around change boundaries. To alleviate the problem, we pay attention to feature refinement and propose deep learning networks that deliver improved separability (ISNet). Our ISNet reaps the advantages from two strategies applied to refining bi-temporal feature hierarchies: (i) margin maximization that clarifies the gap between changed and unchanged semantics, and (ii) targeted arrangement of attention mechanisms that directs the use of channel attention and spatial attention for highlighting semantic and positional information, respectively. Specifically, we insert channel attention modules into share-weighted backbone networks to facilitate semantic-specific feature extraction. The semantic boundaries in the extracted bi-temporal hierarchical features are then clarified by margin maximization modules, followed by spatial attention modules to enhance positional change responses. A top-down fusion pathway makes the final refined features cover multi-scale representations and have strong separability for remote sensing image change detection. Extensive experimental evaluations demonstrate that our ISNet achieves state-of-the-art performance on the LEVIR-CD, SYSU-CD, and Season-Varying datasets, in terms of Overall Accuracy (OA), Intersection-of-Union (IoU), and F1 score. Code is available at https://github.com/xingronaldo/ISNet.

ABSTRACTIn this article, we propose a novel difference image (DI) creation method for unsupervised change detection in multi-temporal multi-spectral remote-sensing images based on deep learning theory. First, we apply deep belief network to learn local and high-level features from the local neighbour of a given pixel in an unsupervised manner. Second, a back propagation algorithm is improved to build a DI based on selected training samples, which can highlight the difference on changed regions and suppress the false changes on unchanged regions. Finally, we get the change trajectory map using simple clustering analysis. The proposed scheme is tested on three remote-sensing data sets. Qualitative and quantitative evaluations show its superior performance compared to the traditional pixel-level and texture-level-based approaches.

Remote sensing image change detection (RS-CD) has made impressive progress with the help of deep learning techniques. Small object change detection (SoCD) still faces many challenges. On the one hand, when the scale of changing objects varies greatly, deep learning models with overall accuracy as the optimization goal tend to focus on large object changes and ignore small object changes to some extent. On the other hand, the RS-CD model based on deep convolutional networks needs to perform multiple spatial pooling operations on the feature map to obtain deep semantic features, which leads to the loss of small object feature-level information in the local space. Therefore, we propose a Siamese transformer change detection network with a multiscale window via an adaptive fusion strategy (SWaF-Trans). To solve the problem of ignoring small object changes, we compute self-attention in windows of different scales to model changing objects at the corresponding scales and establish semantic information links through a moving window mechanism to capture more comprehensive small object features in small-scale windows, thereby enhancing the feature representation of multiscale objects. To fuse multiscale features and alleviate the problem of small object feature information loss, we propose a channel-related fusion mechanism to model the global correlation between channels for display and adaptively adjust the fusion weights of channels to enable the network to capture more discriminative features of interest and reduce small object feature information loss. Experiments on the CDD and WHU-CD datasets show that SWaF-Trans exceeds eight advanced baseline methods, with absolute F1 scores as high as 97.10% and 93.90%, achieving maximum increases of 2% and 5.6%, respectively, compared to the baseline methods.

High-resolution remote-sensing image change detection plays an important role in areas such as land and resources investigation, natural disaster prediction, and military strategy research. Current change detection methods often focus on extracting more discriminative features, while ignoring the information loss and imbalance problems in the process of feature fusion, which results in weakness in small change objects and edge pixels of change objects. In this letter, a simple but efficient network architecture, extraction, comparison and fusion network (ECFNet), for change detection in remote-sensing images is proposed. By constraining the number of feature channels in the fusion process by the feature comparison module (FCM), ECFNet can better utilize the fine-grained information in the multiscale feature map for result prediction, which not only improves the detection performance of small objects, but also reduces the false detection around the edge pixels of change objects. Experiments on the deeply supervised image fusion network for change detection (DSIFN-CD) test set show that ECFNet achieves state-of-the-art results with a small amount of computation.

Deep Neural Network (DNN) has been widely used in remote sensing image change detectionCD in recent years. Due to the scarcity of training data, a large number of labeled data onto other fields become the source of deep neural network concept learning in remote sensing image change detection. However, the distribution of features of the change detection data and other data varies greatly, which prevents DNN from being better applied for one task to another. To solve this problem, a domain adaptive CD method based on segmentation map difference is proposed to this paper, which includes the pre-training stage and the change detection stage. In the pre-training stage, the domain adaptive UNet (Ada-UNet) is applied as the basic network of remote sensing image segmentation for network training, with the purpose of learning the concepts of different features. In the change detection stage, strict threshold segmentation results are used to train the channel attention network, which makes it more efficient to utilize the high-dimensional feature map. The probabilistic map generated by the three-channel attention networks is evaluated, and then it is used to accurately classify the changing pixels. In this paper, experiments are carried out on datasets with different feature distributions. The results show that this method has strong domain adaptability and can greatly reduce the influence of the difference in feature distributions of the CD results.

Remote sensing image change detection (RSICD) is an essential measure for monitoring the earth’s surface changes. In recent years, the explosive growth of very high-resolution (VHR) satellite sensors and the booming innovations in deep learning technology have significantly boosted RSICD development. However, most of the current RSICD models focus on locating accurate change areas while ignoring the efficiency of their method, which limits the practical application of RSICD models, especially for large-scale and emergency RSICD tasks. In this paper, we propose an Efficient Multi-scale-fusion Change Detection Network (EMS-CDNet) for bi-temporal RSICD tasks. Our EMS-CDNet pays more attention to the model’s inference speed and the accuracy-efficiency trade-off rather than only pursuing detection accuracy. We designed a multi-scale fusion module for EMS-CDNet, which adopts multi-scale and multi-branch operations to extract multi-scale features simultaneously and aggregate features at different feature levels. In addition to EMS-CDNet’s ability to achieve sufficient feature extraction, the multi-scale image input within the designed module alleviates the influence of image registration errors in practical applications, thereby strengthening EMS-CDNet’s value for practical RSICD tasks. We also integrated a novel partition unit in EMS-CDNet to lighten the model while maintaining the detection ability of small targets, thus shortening its processing time without a severe accuracy decrease. We conducted experiments on two state-of-the-art (SOTA) public RSICD datasets and our own collected dataset. The public datasets were utilized to comparatively measure the overall accuracy and efficiency measurement of EMS-CDNet, and the dataset of images we collected was used to observe EMS-CDNet’s performance under the influence of image registration errors. Our experimental results show that EMS-CDNet achieved a better accuracy-efficiency trade-off than the SOTA public datasets methods. For example, EMS-CDNet reduced the inference time by about 33% while maintaining identical detection accuracy to CLNet (the optimal method among the comparison methods). Furthermore, EMS-CDNet achieved higher accuracy on our collected dataset, with an F1 of 74% and mIoU of 0.806, demonstrating its robustness to image registration errors and showing its value for practical RSICD applications.

Change detection in remote-sensing images is used to detect changes during different time periods on the surface of the Earth. Because of the advantages of synthetic aperture radar (SAR), which is not affected by time, weather or other conditions, change-detection technology based on SAR images has important research value. At present, this technology has attracted the attention of increasingly more researchers, and has also been used extensively in diverse fields, such as urban planning, disaster assessment, and forest early warning systems. Our objective in this paper is to combine both the change detection of SAR images with the deep neural networks to compare its efficiency with fuzzy clustering method and deep belief network. Our experiments, conducted on real data sets and theoretical analysis, indicates the advantages of the proposed method. Our results appear that proposed deep-learning algorithms can further improve the change-detection process.

<p>A weakly supervised change detection method is proposed for remotely sensed multi-temporal images, by utilizing a Siamese neural<br> network architecture. The architecture of the Siamese network is a combination of two multi-filter multi-scale deep convolutional neural networks (MFMS DCNN). Initially, the Siamese network is trained by utilizing the image-level semantic labels of the image pairs in the dataset. The features of the image pairs are obtained using the trained network to generate the difference image (DI). Then, a combination of the PCA and the K-means algorithms has been used to produce the change map for the pair of images. Experiments were carried out using two remotely sensed image datasets. The weakly supervised method proposed in this paper offers better results in comparison to both weakly supervised- and unsupervised-based state-of-the-art models and techniques.</p>

As a result of its important role in video surveillance, pedestrian attribute recognition has become an attractive facet of computer vision research. Because of the changes in viewpoints, illumination, resolution and occlusion, the task is very challenging. In order to resolve the issue of unsatisfactory performance of existing pedestrian attribute recognition methods resulting from ignoring the correlation between pedestrian attributes and spatial information, in this paper, the task is regarded as a spatiotemporal, sequential, multi-label image classification problem. An attention-based neural network consisting of convolutional neural networks (CNN), channel attention (CAtt) and convolutional long short-term memory (ConvLSTM) is proposed (CNN-CAtt-ConvLSTM). Firstly, the salient and correlated visual features of pedestrian attributes are extracted by pre-trained CNN and CAtt. Then, ConvLSTM is used to further extract spatial information and correlations from pedestrian attributes. Finally, pedestrian attributes are predicted with optimized sequences based on attribute image area size and importance. Extensive experiments are carried out on two common pedestrian attribute datasets, PEdesTrian Attribute (PETA) dataset and Richly Annotated Pedestrian (RAP) dataset, and higher performance than other state-of-the-art (SOTA) methods is achieved, which proves the superiority and validity of our method.

Classification of brain tumor plays a vital role in medical imaging for accurate diagnosis, treatment, and monitoring. Deep learning approaches have gained significant traction in this industry because of their ability to extract relevant features from medical images. The research suggests employing an ensemble classifier with a weighted voting mechanism to categorize glial cell brain malignancies such as Astrocytoma, Glioblastoma multiforme, Oligodendroglioma, and Ependymoma. The proposed ensemble technique employs three main classifiers: Convolutional Neural Network (CNN), Deep Convolutional Long Short Term Memory (C-LSTM), and Deep Convolutional Neural Network + Conditional Random Fields (DCNN+CRF). Deep learning algorithms require a huge amount of input data to avoid overfitting. The Adaptive Progressive Convolutional Generative Adversarial Networks (APCGANs) are used to produce realistic artificial images to efficiently train the proposed methodology. Overall, the proposed ensemble method with weighted voting strategy consistently outperforms the other tested algorithms (CNN, C-LSTM, and DCNN+CRF). Ensemble method attained an accuracy of 99.4 %, recall - 99.1%, precision- 98.0%, and F1-score of 99.2%. Ensemble method consistently demonstrates superior performance in accurately classifying brain tumors, making it a promising algorithm for brain tumor analysis tasks.

The pandemic of novel coronavirus disease 2019 (Covid-19) has left the world to a standstill by creating a calamitous situation. To mitigate this devastating effect the inception of artificial intelligence into medical health care is mandatory. This study aims to present the educational perspective of Covid-19 and forecast the number of confirmed and death cases in the USA, India, and Brazil along with the discussion of endothelial dysfunction in epithelial cells and Angiotensin-Converting Enzyme 2 receptor (ACE2) with the Covid-19. Three different deep learning based experimental setups have been framed to forecast Covid-19. Models are (i) Bi-directional Long Short Term Memory (LSTM) (ii) Convolutional LSTM (iii) Proposed ensemble of Convolutional and Bi-directional LSTM network are known as CoBiD-Net ensemble. The educational perspective of Covid-19 has been given along with an architectural discussion of multi-organ failure due to intrusion of Covid-19 with the cell receptors of the human body. Different classification metrics have been calculated using all three models. Proposed CoBiD-Net ensemble model outperforms the other two models with respect to accuracy and mean absolute percentage error (MAPE). Using CoBiD-Net ensemble, accuracy for Covid-19 cases ranges from 98.10 to 99.13% with MAPE ranges from 0.87 to 1.90. This study will help the countries to know the severity of Covid-19 concerning education in the future along with forecasting of Covid-19 cases and human body interaction with the Covid-19 to make it the self-replicating phenomena.Supplementary InformationThe online version contains supplementary material available at 10.1007/s41324-021-00408-3.

Deep learning has achieved great success in remote sensing image change detection (CD). However, most methods focus only on the changed regions of images and cannot accurately identify their detailed semantic categories. In addition, most CD methods using convolutional neural networks (CNN) have difficulty capturing sufficient global information from images. To address the above issues, we propose a novel symmetric multi-task network (SMNet) that integrates global and local information for semantic change detection (SCD) in this paper. Specifically, we employ a hybrid unit consisting of pre-activated residual blocks (PR) and transformation blocks (TB) to construct the (PRTB) backbone, which obtains more abundant semantic features with local and global information from bi-temporal images. To accurately capture fine-grained changes, the multi-content fusion module (MCFM) is introduced, which effectively enhances change features by distinguishing foreground and background information in complex scenes. In the meantime, the multi-task prediction branches are adopted, and the multi-task loss function is used to jointly supervise model training to improve the performance of the network. Extensive experimental results on the challenging SECOND and Landsat-SCD datasets, demonstrate that our SMNet obtains 71.95% and 85.65% at mean Intersection over Union (mIoU), respectively. In addition, the proposed SMNet achieves 20.29% and 51.14% at Separated Kappa coefficient (Sek) on the SECOND and Landsat-SCD datasets, respectively. All of the above proves the effectiveness and superiority of the proposed method.

Change detection (CD) is one of the important applications of remote sensing and plays an important role in disaster assessment, land use detection, and urban sprawl tracking. High-accuracy fully supervised methods are the main methods for CD tasks at present. However, these methods require a large amount of labeled data consisting of bi-temporal images and their change maps. Moreover, creating change maps takes a lot of labor and time. To address this limitation, a simple semi-supervised change detection method based on consistency regularization and strong augmentation is proposed in this paper. First, we construct a Siamese nested UNet with graph attention mechanism (SANet) and pre-train it with a small amount of labeled data. Then, we feed the unlabeled data into the pre-trained SANet and confidence threshold filter to obtain pseudo-labels with high confidence. At the same time, we produce distorted images by performing strong augmentation on unlabeled data. The model is trained to make the CD results of the distorted images consistent with the corresponding pseudo-label. Extensive experiments are conducted on two high-resolution remote sensing datasets. The results demonstrate that our method can effectively improve the performance of change detection under insufficient labels. Our methods can increase the IoU by more than 25% compared to the state-of-the-art methods.

Remote sensing image change detection is the detection process of determining the surface change area and change feature type for the same image area from multiple time series remote sensing data. It is the core technical means of land use change detection and land cover change detection, and it is also the key research field of remote sensing applied science. In view of the problems of time-consuming, labor-intensive, and low detection accuracy in the past image change detection methods, researchers in related fields have proposed more and more cutting-edge remote sensing image change detection methods. This article first describes the development of remote sensing image change detection methods, then explains the conventional processing flow of change detection and the conventional methods of change monitoring, and finally discusses the research progress of more mainstream change detection methods.