Semantic change detection of roads and bridges: A fine-grained dataset and multimodal frequency-driven detector

Transformer-based multimodal change detection with multitask consistency constraints

Semantic change detection (SCD) extends the multiclass change detection (MCD) task to provide not only the change locations but also the detailed land-cover/land-use (LCLU) categories before and after the observation intervals. This fine-grained semantic change information is very useful in many applications. Recent studies indicate that the SCD can be modeled through a triple-branch convolutional neural network (CNN), which contains two temporal branches and a change branch. However, in this architecture, the communications between the temporal branches and the change branch are insufficient. To overcome the limitations in existing methods, we propose a novel CNN architecture for the SCD, where the semantic temporal features are merged in a deep CD unit. Furthermore, we elaborate on this architecture to reason the bi-temporal semantic correlations. The resulting bi-temporal semantic reasoning network (Bi-SRNet) contains two types of semantic reasoning blocks to reason both single-temporal and cross-temporal semantic correlations, as well as a novel loss function to improve the semantic consistency of change detection results. Experimental results on a benchmark dataset show that the proposed architecture obtains significant accuracy improvements over the existing approaches, while the added designs in the Bi-SRNet further improve the segmentation of both semantic categories and the changed areas. The codes in this article are accessible at <uri>https://github.com/ggsDing/Bi-SRNet</uri>.

Semantic change detection (SCD) in high-resolution (HR) remote sensing images faces two issues: (1) isolated network branch for binary change detection (BCD) within multi-task architecture result in suboptimal SCD performance; (2) false alarms or missed detections caused by illumination differences or seasonal transform. To address these issues, this study proposes a bi-temporal binary change enhancement network (Bi-BCENet). Specifically, we introduce a binary change enhancement (BCE) strategy based on multi-network joint learning to achieve superior SCD via improving change areas prediction. Within the network’s reasoning process, we develop a cross-attention fusion module (CAFM) to enhance the global similarity modeling via cross-network prompt fusion, and we employ a cosine similarity-based auxiliary loss to optimize non-change’s semantic consistency. The experiments on SECOND and CINA-FX datasets demonstrate that Bi-BCENet outperforms representative SCD networks, achieving 62.08%, 84.95% in FSCD and 66.88%, 83.10% in mIoUsCD, respectively. And the ablation analysis of network validates Bi-BCENet’s effectiveness in reducing false alarms and missed detections in SCD results. Moreover, for specific SCD of cropland, Bi-BCENet shows its strong potential in single-to-multi SCD.

Monitoring the spatial and temporal trends of land use land cover (LULC) change is significant for protecting land assets sustainably. However, complicated scenarios in Remote Sensing (RS) images lead to unsatisfactory outcomes, particularly for some uneven and obstructed objects. Furthermore, there is still a hurdle in efficiently extracting and fusing the complex characteristics of RS images to increase the precision of change detection. Therefore, to get better results while detecting changes in LULC, this research implements effective deep learning approaches for segmentation and feature extraction. In the beginning, a HarDNet-based technique is used to efficiently segment the edge details of the land cover data. After that, the segmented information is sent to the Dual Path Coronet to extract spatial and spectral properties and fuse them to create the feature map of the input. After that, it is sent to the Enhanced Change Prediction Module, which generates a change prediction map with precise information. Last but not least, the suggested method has been tested on the three complex remote sensing change detection datasets, Sun Yat-Sen University (SYSU-CD), High-Resolution Semantic Change Detection (HRSCD), and LEVIR-CD. The quantitative and qualitative analysis findings on the datasets reveal that the suggested technique works better than other standard change detection techniques.

Semantic change detection (SCD) represents a challenging task in the interpretation of remote sensing images (RSIs), with the goal of identifying change regions and extracting semantic information from bitemporal RSIs simultaneously. The recent integration of deep neural networks leveraging multitask learning has shown promise in enhancing SCD performance. However, there is still a challenge in improving SCD performance, specifically in designing a fine-grained network structure that can handle the two subtasks of change region localization and semantic information recognition in parallel. In this context, a novel multitask Siamese network, termed EGMS-Net, is proposed to boost the performance of SCD, which consists of three core components. First, a coarse-to-fine multitask Siamese network is constructed to obtain semantic information and change information at multiple levels. Second, an adaptive change information enhancement method based on spatial-spectral collaborative attention mechanism is proposed, which can assist the accurate localization of change regions without significantly increasing the model parameters. Third, a change information guidance module is developed to strengthen the interaction between multitask branches and reduce the difficulty of network training. Experiments on three benchmark datasets demonstrate that the proposed EGMS-Net outperforms existing state-of-the-art methods in the SCD community.

Given a pair of bitemporal very high resolution (VHR) remote sensing images, the semantic change detection task aims to locate land surface changes and identify their semantic classes. The existing algorithms use independent branches to locate and identify separately without considering the association between branches. In this article, we propose an end-to-end spatially and semantically enhanced Siamese network (SSESN) for semantic change detection. The SSESN aggregates the rich spatial and semantic information in the VHR image through a designed spatial and semantic feature aggregation module. Additionally, a change-aware module is proposed to decouple the aggregated features. Features in the binary branch are fused to the semantic branches as prior location information. This allows the spatially enhanced features to predict changed regions and the semantically enhanced features to refine the region categorizations. Experimental results show that our method provides comparable results with the state-of-the-art binary change detection and semantic change detection algorithms.

Building change detection has various applications, such as urban management and disaster assessment. Along with the exponential growth of remote sensing data and computing power, an increasing number of deep‐learning‐based remote sensing building change detection methods have been proposed in recent years. Objectively, the overwhelming majority of existing methods can perfectly deal with the change detection of low‐rise buildings. By contrast, high‐rise buildings often present a large disparity in multitemporal high‐resolution remote sensing images, which degrades the performance of existing methods dramatically. To alleviate this problem, we propose a disparity‐aware Siamese network for detecting building changes in bi‐temporal high‐resolution remote sensing images. The proposed network utilises a cycle‐alignment module to address the disparity problem at both the image and feature levels. A multi‐task learning framework with joint semantic segmentation and change detection loss is used to train the entire deep network, including the cycle‐alignment module in an end‐to‐end manner. Extensive experiments on three publicly open building change detection datasets demonstrate that our method achieves significant improvements on datasets with severe building disparity and state‐of‐the‐art performance on datasets with minimal building disparity simultaneously.

Recent years have seen rising interest in robotic mapping algorithms that operate at the level of objects, rather than two- or three-dimensional occupancy. Such “semantic maps” permit higher-level reasoning than occupancy maps, and are useful for any application that involves dealing with objects, including grasping, change detection, and object search. We describe and experimentally verify such a system aboard a mobile robot equipped with a Microsoft Kinect RGB-D sensor. Our representation is object-based, and makes uniquely weak assumptions about the quality of the perceptual data available; in particular, we perform no explicit object recognition. This allows our system to operate in large, dynamic, and uncon-strained environments, where modeling every object that occurs (or might occur) is impractical. Our dataset, which is publicly available, consists of 67 autonomous runs of our robot over a six-week period in a roughly 1600m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> office environment. We demonstrate two applications built on our system: semantic querying and change detection.

Rapid domain adaptation for disaster impact assessment: Remote sensing of building damage after the 2021 Germany floods

Deep learning has achieved great success in remote sensing image change detection (CD). However, most methods focus only on the changed regions of images and cannot accurately identify their detailed semantic categories. In addition, most CD methods using convolutional neural networks (CNN) have difficulty capturing sufficient global information from images. To address the above issues, we propose a novel symmetric multi-task network (SMNet) that integrates global and local information for semantic change detection (SCD) in this paper. Specifically, we employ a hybrid unit consisting of pre-activated residual blocks (PR) and transformation blocks (TB) to construct the (PRTB) backbone, which obtains more abundant semantic features with local and global information from bi-temporal images. To accurately capture fine-grained changes, the multi-content fusion module (MCFM) is introduced, which effectively enhances change features by distinguishing foreground and background information in complex scenes. In the meantime, the multi-task prediction branches are adopted, and the multi-task loss function is used to jointly supervise model training to improve the performance of the network. Extensive experimental results on the challenging SECOND and Landsat-SCD datasets, demonstrate that our SMNet obtains 71.95% and 85.65% at mean Intersection over Union (mIoU), respectively. In addition, the proposed SMNet achieves 20.29% and 51.14% at Separated Kappa coefficient (Sek) on the SECOND and Landsat-SCD datasets, respectively. All of the above proves the effectiveness and superiority of the proposed method.

Semantic change detection (SCD) is a critical task in the field of remote sensing, which aims to provide semantic understanding of remote sensing imagery by simultaneously detecting change regions and classifying their semantic categories. It plays a significant role in applications such as land use monitoring and ecological environment assessment. With the increasing availability of bi-temporal remote sensing datasets, a variety of deep learning-based models have been widely applied to the task of SCD detection. However, existing methods still suffer from limitations in modelling the interaction of cross-temporal semantic features and in the adaptive fusion of differential information, which constrains detection accuracy. Thus, to address these limitations, we propose ESTFNet, an efficient spatiotemporal fusion network for SCD detection. The network is built around two core modules: the Efficient Spatiotemporal Cross-Attention (ESTCA) and the (BDAF), which address feature interaction and feature fusion, respectively. Extensive experimental results demonstrate that ESTFNet outperforms several recent representative SCD models on remote sensing datasets, showing superior performance in complex scenarios. The code of our ESTFNet is available at https://github.com/loudongang/ESTFNet.

Semantic change detection (SCD) aims to identify land cover changes between bi-temporal remote sensing images and plays a crucial role in applications, such as urban monitoring and disaster assessment. Most existing methods rely on a shared-weight encoder with a dedicated change extractor, which limits their ability to precisely localize changed regions and maintain intra-class semantic consistency. Although vision foundation models (VFMs) exhibit strong generalization ability, their potential for SCD remains largely underexplored. In this paper, we introduce ChangeVFM, a novel yet effective framework that unleashes the power of vision foundation models for semantic change detection in remote sensing images. This framework is supplemented by a spatiotemporal modeling module to capture fine-grained spatial details and a query-based feature injector that integrates VFM’s semantic priors with multi-scale spatiotemporal features. The feature injector ensures that the ChangeVFM excels in both maintaining semantic consistency and multi-scale information of bi-temporal images. Without bells and whistles, ChangeVFM achieves competitive performance on the HRSCD, SECOND, and Landsat-SCD. Comprehensive quantitative and qualitative experiments further validate the effectiveness of the introduced modules and the robustness of the proposed method.

Change detection is a process of identifying changed ground objects by comparing image pairs obtained at different times. Compared with the pixel-level and object-level change detection, scene-level change detection can provide the semantic changes at image level, so it is important for many applications related to change descriptions and explanations such as urban functional area change monitoring. Automatic scene-level change detection approaches do not require ground truth used for training, making them more appealing in practical applications than nonautomatic methods. However, the existing automatic scene-level change detection methods only utilize low-level and mid-level features to extract changes between bitemporal images, failing to fully exploit the deep information. This article proposed a novel automatic binary scene-level change detection approach based on deep learning to address these issues. First, the pretrained VGG-16 and change vector analysis are adopted for scene-level direct predetection to produce a scene-level pseudo-change map. Second, pixel-level classification is implemented by using decision tree, and a pixel-level to scene-level conversion strategy is designed to generate the other scene-level pseudo-change map. Third, the scene-level training samples are obtained by fusing the two pseudo-change maps. Finally, the binary scene-level change map is produced by training a novel scene change detection triplet network (SCDTN). The proposed SCDTN integrates a late-fusion subnetwork and an early fusion subnetwork, comprehensively mining the deep information in each raw image as well as the temporal correlation between two raw images. Experiments were performed on a public dataset and a new challenging dataset, and the results demonstrated the effectiveness and superiority of the proposed approach

We use contextualized word definitions generated by large language models as semantic representations in the task of diachronic lexical semantic change detection (LSCD).In short, generated definitions are used as 'senses', and the change score of a target word is retrieved by comparing their distributions in two time periods under comparison.On the material of five datasets and three languages, we show that generated definitions are indeed specific and general enough to convey a signal sufficient to rank sets of words by the degree of their semantic change over time.Our approach is on par with or outperforms prior non-supervised sensebased LSCD methods.At the same time, it preserves interpretability and allows to inspect the reasons behind a specific shift in terms of discrete definitions-as-senses.This is another step in the direction of explainable semantic change modeling.

In this paper, we present a deep convolutional neural network (CNN) architecture for segmenting semantic changes between two images. The main objective is to segment changes at the semantic level than detecting background changes, which are irrelevant to the application. The difficulties include seasonal changes, lighting differences, artifacts due to alignment and occlusion. The existing approaches fail to address all the problems together; thus, none of them achieve state-of-the-art performance in three publicly available change detection datasets: VL-CMU-CD [1], TSUNAMI [2] and GSV [2]. Our proposed approach is a simple yet effective method that can handle even adverse challenges. In our approach, we leverage the correlation between high-level abstract CNN features to segment the changes. Compared with several traditional and other deep learning-based change detection methods, our proposed method achieves state-of-the-art performance in all three datasets.