Comment on essd-2022-155

This paper examines a feature-level fusion framework for detecting and mapping land degradation (LD) and enabling sustainable land management (SLM) in semi-arid areas using optical sensors and Synthetic Aperture Radar (SAR) satellite data. The objectives of this review were to (i) determine the trends and geographical location of land degradation mapping publications, (ii) to identify and report current challenges pertaining to mapping LD using multiscale remote sensing data, (iii) to recommend a way forward for monitoring LD using multiscale remote sensing data. The study reviewed 78 peer-reviewed research articles published over the past 24 years (1998–2022). Image fusion has the potential to be more useful in various remote sensing applications than individual sensor image data, making it more informative and valuable in the interpretation process. In addition, this review discusses the importance of SAR and optical image fusion, pixel-level techniques, applications, and major classes of quality metrics for objectively assessing fusion performance. The literature review alluded that the SAR and optical image fusion in the detection and mapping of land degradation and enabling sustainable land management has not been fully explored. Advanced techniques such as the fusion of SAR and optical satellite imageries need to be incorporated for the detection and mapping of LD, as well as the promotion of SLM in halting LD in South African drylands and around the world. We conclude that there is scope for further research on the fusion of SAR and optical images, as new micro-wave and optical sensors with higher resolution are introduced on a regular basis. The results of this review contribute to a better understanding of the applications of SAR and optical image fusion in future research in the severely degraded drylands of southern Africa. KEY RESEARCH GAPS The fusion of SAR and optical data still remains an open challenge. The future of different remote sensing applications lies in this kind of fusion. Land degradation is one of the greatest challenges amongst the environmental problems in South Africa, causing a reduction in the capacity of the land to perform ecosystem functions and services that support society and development. Yet, in South Africa, there are no studies that have widely investigated the potential for a fusion of SAR and optical data to detect and map land degradation and SLM practices. This paper established a baseline for understanding the application of a fusion of SAR and optical data as rapid tools for mapping, monitoring, and evaluating LD, as well as the impacts of SLM practices in South Africa’s degraded drylands.

Synthetic aperture radar (SAR) data from the Canadian RADARSAT satellite, along with infrared and visible data from operational weather satellites, is being collected over the ocean off the U.S. East, Gulf of Mexico, Gulf of Alaska, and Bering Sea coasts. Mesoscale features observed in the SAR data are identified using coincident Geostationary Operational Environment Satellite (GOES) data to decide whether the signature is atmospheric or oceanic in origin. Combining the two data sources allows information in the marine atmospheric boundary layer (MABL) to be observed. This technique of feature detection using multiple data sets is referred to as data fusion. This paper focuses on a single storm event that is imaged by RADARSAT SAR in the Bering Sea on February 5, 1998 at 06:00 UTM. From the fusion of SAR and GOES data, frontal boundaries can be deciphered.

Hyperspectral (HS) and multispectral (MS) image fusion aims at producing high-resolution HS (HRHS) images. However, the existing methods could not simultaneously consider the structures in both the spatial and spectral domains of the HS cube. In order to effectively preserve spatial–spectral structures in HRHS images, we propose a new low-resolution HS (LRHS) and high-resolution MS (HRMS) image fusion method based on spatial–spectral-graph-regularized low-rank tensor decomposition (SSGLRTD) in this paper. First, we reformulate the image fusion problem as a low-rank tensor decomposition model to utilize the low-rank property in the HS image. Then, two graphs are constructed in spatial and spectral domains, respectively. One of them is derived from the HRMS image for the spatial consistency, and the other is inferred from the LRHS image for spectral smoothness. Finally, the SSGLRTD fusion framework is established by combining all regularizers. With these two graphs, the spatial correlation and the spectral structure in the fused HRHS images are efficiently preserved. The experimental results on different datasets reveal that the proposed fusion method outperforms several existing fusion methods in terms of visual analysis and numerical comparison.

Hyperspectral images (HSIs) usually have high spectral and low spatial resolution. Conversely, multispectral images (MSIs) usually have low spectral and high spatial resolution. The fusion of HSI and MSI aims to create spectral images with high spectral and spatial resolution. In this paper, we propose a fusion algorithm by combining linear spectral unmixing with the local low-rank property. By taking advantage of the local low-rank property, we first partition the corresponding spectral image into patches. For each patch pair, we cast the fusion problem as a coupled spectral unmixing problem that extracts the abundance and the endmembers of MSI and HSI, respectively. It then updates the abundance and the endmember through an alternating update algorithm. In fact, the convergence of the alternative update algorithm can be mathematically and empirically supported. We also propose a multiscale postprocessing procedure to combine fusion results obtained under different patch sizes. In experiments on three data sets, the proposed fusion algorithms outperformed state-of-the-art fusion algorithms in both spatial and spectral domains.

Recent advances and new guidelines on hyperspectral and multispectral image fusion

Hyperspectral images and polarimetric synthetic aperture radar (PolSAR) data are two important data sources, yet they barely appear under the same scope, even though multi-modal data fusion is attracting more and more attention. To our best knowledge, this paper investigates for the first time semi-supervised manifold alignment (SSMA) for the fusion of the hyperspectral image and PolSAR data. The SSMA searches a latent space where different data sources are aligned, which is accomplished by using the label information and the topological structure of the data. This paper is the first attempt to apply topological data analysis (TDA), a recent mathematic sub-field of data analysis, in remote sensing. It aims to reveal relevant information from the shape of a data in its feature space, and has been proven powerful in medicine. The paper also proposes a novel algorithm, MAPPER-regularized manifold alignment, which embeds the TDA into a semi-supervised manifold alignment for the fusion of the hyper-spectral image and PolSAR data. The proposed algorithm exhibits superior performance in fusing a simulated EnMAP data set and a Sentinel-1 data set for an image of Berlin.

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral-spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

Fusion of hyperspectral images and lidar-based dems for coastal mapping

For many remote sensing applications it is preferable to have images with both high spectral and spatial resolutions. On this regards, hyperspectral and multispectral images have complementary characteristics in terms of spectral and spatial resolutions. In this paper we propose an approach for the fusion of low spatial resolution hyperspectral images with high spatial resolution multispectral images in order to obtain superresolution (spatial and spectral) hyperspectral images. The proposed approach is based on the assumption that, since both hyperspectral and multispectral images acquired on the same scene, the corresponding endmembers should be the same. On a first step the hyperspectral image is spectrally down-sampled in order to match the multispectral one. Then an endmember extraction algorithm is performed on the down-sampled hyperspectral image and the successive abundance estimation is performed on the multispectral one. Finally, the extracted endmembers are up-sampled back to the original hyperspectral space and then used to reconstruct the superresolution hyperspectral image according to the abundances obtained from the multispectral image.

As synthetic aperture radar (SAR) data is more and more popular in recent days, the fusion of SAR and optical image become inevitable and desirable problem. Image Registration is the fundamental for data fusion, and has been wildly studied. However, literature on registration between SAR and optical image remain little, which means the problem is still challenge and open. On one hand, the SAR and optical image are from totally different sensor with variant imaging mechanism. On the other hand, Simple features could not enough to maintain the accuracy and efficiency of multi-modal registration. We propose a new method for SAR and optical image registration. Different from current methods mainly based on intensity feature or point feature, our method matches two images by their boundary features. A globalized boundary detection algorithm are used for feature extraction. The extracted boundaries were then matched by coherence point drift algorithm. Experiments prove that our method could achieve pixel-level precision, and perform robust for different resolutions.

The objectives of this research are to develop robust methods for segmentation of multitemporal synthetic aperture radar (SAR) and optical data and to investigate the fusion of multitemporal ENVISAT advanced synthetic aperture radar (ASAR) and Chinese HJ-1B multispectral data for detailed urban land-cover mapping. Eight-date multiangle ENVISAT ASAR images and one-date HJ-1B charge-coupled device image acquired over Beijing in 2009 are selected for this research. The edge-aware region growing and merging (EARGM) algorithm is developed for segmentation of SAR and optical data. Edge detection using a Sobel filter is applied on SAR and optical data individually, and a majority voting approach is used to integrate all edge images. The edges are then used in a segmentation process to ensure that segments do not grow over edges. The segmentation is influenced by minimum and maximum segment sizes as well as the two homogeneity criteria, namely, a measure of color and a measure of texture. The classification is performed using support vector machines. The results show that our EARGM algorithm produces better segmentation than eCognition, particularly for built-up classes and linear features. The best classification result (80%) is achieved using the fusion of eight-date ENVISAT ASAR and HJ-1B data. This represents 5%, 11%, and 14% improvements over eCognition, HJ-1B, and ASAR classifications, respectively. The second best classification is achieved using fusion of four-date ENVISAT ASAR and HJ-1B data (78%). The result indicates that fewer multitemporal SAR images can achieve similar classification accuracy if multitemporal multiangle dual-look-direction SAR data are carefully selected.

Abstract. Hyperspectral image enhancement has been a concern for the remote sensing society for detailed end member detection. Hyperspectral remote sensor collects images in hundreds of narrow, continuous spectral channels, whereas multispectral remote sensor collects images in relatively broader wavelength bands. However, the spatial resolution of the hyperspectral sensor image is comparatively lower than that of the multispectral. As a result, spectral signatures from different end members originate within a pixel, known as mixed pixels. This paper presents an approach for obtaining an image which has the spatial resolution of the multispectral image and spectral resolution of the hyperspectral image, by fusion of hyperspectral and multispectral image. The proposed methodology also addresses the band remapping problem, which arises due to different regions of spectral coverage by multispectral and hyperspectral images. Therefore we apply algorithms to restore the spatial information of the hyperspectral image by fusing hyperspectral bands with only those bands which come under each multispectral band range. The proposed methodology is applied over Henry Island, of the Sunderban eco-geographic province. The data is collected by the Hyperion hyperspectral sensor and LISS IV multispectral sensor.

Land use and land cover (LULC) classification through remote sensing imagery serves as a cornerstone for environmental monitoring, resource management, and evidence-based urban planning. While Synthetic Aperture Radar (SAR) and optical sensors individually capture distinct aspects of Earth’s surface, their complementary nature SAR excelling in structural and all-weather observation and optical sensors providing rich spectral information—offers untapped potential for improving classification robustness. However, the intrinsic differences in their imaging mechanisms (e.g., SAR’s coherent scattering versus optical’s reflectance properties) pose significant challenges in achieving effective multimodal fusion for LULC analysis. To address this gap, we propose a multimodal deep-learning framework that systematically integrates SAR and optical imagery. Our approach employs a dual-branch neural network, with two fusion paradigms being rigorously compared: the Early Fusion strategy and the Late Fusion strategy. Experiments on the SEN12MS dataset—a benchmark containing globally diverse land cover categories—demonstrate the framework’s efficacy. Our Early Fusion strategy achieved 88% accuracy (F1 score: 87%), outperforming the Late Fusion approach (84% accuracy, F1 score: 82%). The results indicate that optical data provide detailed spectral signatures useful for identifying vegetation, water bodies, and urban areas, whereas SAR data contribute valuable texture and structural details. Early Fusion’s superiority stems from synergistic low-level feature extraction, capturing cross-modal correlations lost in late-stage fusion. Compared to state-of-the-art baselines, our proposed methods show a significant improvement in classification accuracy, demonstrating that multimodal fusion mitigates single-sensor limitations (e.g., optical cloud obstruction and SAR speckle noise). This study advances remote sensing technology by providing a precise and effective method for LULC classification.

Hyperspectral (HS) remote sensing image with finer spectral information has great advantages in feature identification and classification. However, the spatial resolution of HS image is usually low due to practical limitations. In this paper, the low-spatial-resolution HS image is fused with the high-spatial-resolution multispectral (MS) image of the same observation scene to improve its spatial resolution. A novel spectral unmixing based HS and MS image fusion approach (VSC-CNMF) is proposed, in which CNMF with minimum endmember simplex volume and abundance sparsity constraints is employed for coupled unmixing of HS and MS images. Simulative experiments are employed for verification and comparison. The experimental results illustrate that the newly proposed VSC-CNMF based HS and MS fusion algorithm outperforms several state-of-the-art unmixing based fusion approaches in cases with moderate number of endmembers.

Fusion of hyperspectral and panchromatic images using structure tensor and matting model