Cross-domain correspondence intensity modulation based on Bayesian-decision for remote sensing image pansharpening.

This paper proposes a pan-sharpening method based on multi-direction subbands and deep neural networks. First, by utilizing the multi-scale and multi-direction properties of the nonsubsampled contourlet transform (NSCT), panchromatic (PAN) image is decomposed into the low frequency subbands in different resolutions and the high frequency subbands in different directions. Pan-sharpening method aims to fuse the high frequency subband coefficients of PAN image and the low frequency subband coefficients of multispectral (MS) image. Second, in order to better extract the feature of the high frequency subbands in different directions of PAN image, the deep neural network (DNN) is trained using the image patches of high frequency subbands of PAN image. Third, in the fusion stage, we exploit NSCT on the principal component of resampled low resolution (LR) MS image. The high frequency subbands of output high resolution (HR) MS image is obtained by forward propagation of the trained DNN, which input is the high frequency subbands of LR MS image. Finally, a new subband set is obtained by fusing the reconstructed high frequency subband and the original low frequency subband of LR MS image. The HR MS image is produced by executing the inverse transform of NSCT and adaptive PCA (A-PCA) on the new subband set. The experimental results show the proposed method outperforms other well-known methods in terms of both objective measurements and visual evaluation.

Pansharpening is to fuse high-resolution panchromatic (PAN) images with low-resolution multispectral (LR-MS) images to generate high-resolution multispectral (HR-MS) images. Most of the deep learning-based pansharpening methods did not consider the inconsistency of the PAN and LR-MS images and used simple concatenation to fuse the source images, which may cause spectral and spatial distortion in the fused results. To address this problem, a multi-scale spatial-spectral attention guided fusion network for pansharpening is proposed. First, the spatial features from the PAN image and spectral features from the LR-MS image are independently extracted to obtain the shallow features. Then, a spatial-spectral attention feature fusion module (SAFFM) is constructed to guide the reconstruction of spatial-spectral features by generating a guidance map to achieve the fusion of reconstructed features at different scales. In SAFFM, the guidance map is designed to ensure the spatial-spectral consistency of the reconstructed features. Finally, considering the difference between multiply scale features, a multi-level feature integration scheme is proposed to progressively achieve fusion of multi-scale features from different SAFFMs. Extensive experiments validate the effectiveness of the proposed network against other state-of-the-art (SOTA) pansharpening methods in both quantitative and qualitative assessments. The source code will be released at https://github.com/MELiMZ/ssaff.

Pansharpening technology plays a crucial role in remote sensing image processing by integrating low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) images to generate high-resolution multispectral (HRMS) images. This process addresses the limitations of satellite sensors, which cannot directly capture HRMS images. Despite significant developments achieved by deep learning-based pansharpening methods over traditional approaches, most existing techniques either fail to account for the modal differences between LRMS and PAN images, relying on direct concatenation, or use similar network structures to extract spectral and spatial information. Additionally, many methods neglect the extraction of common features between LRMS and PAN images and lack network architectures specifically designed to extract spectral features. To address these limitations, this study proposed a novel three-branch pansharpening network that leverages both spatial and frequency domain interactions, resulting in improved spectral and spatial fidelity in the fusion outputs. The proposed method was validated on three datasets, including IKONOS, WorldView-3 (WV3), and WorldView-4 (WV4). The results demonstrate that the proposed method surpasses several leading techniques, achieving superior performance in both visual quality and quantitative metrics.

Distributed compressed sensing-based pan-sharpening with hybrid dictionary

A novel method for multispectral image pansharpening based on high dimensional model representation

Pansharpening is a process of combining a high-resolution Panchromatic (PAN) image with a low-resolution Multispectral (MS) image to obtain a high-resolution MS image. Pansharpening can not only overcome the physical and technical limitations of satellite sensors, but also improve the quality of images and obtain a more detailed description of the scene. In this paper, a novel method of Pansharpening is proposed which adopts the ARSIS concept. The spatial information missing in the low-resolution MS image can be extracted from the PAN image. This is achieved by image decomposition based on sparse representation. Then, the information is injected into the MS bands by the details injection model. In the dictionary training, we adapt Online Sparse Dictionary Learning (OSDL), which can shorten the dictionary training time and improve the fusion image effect. Experimental results show that the fusion image of the proposed method has stronger spectral performance and more detailed spatial information.

Pan-sharpening is an approach that fuse low resolution multi-spectral (LRMS) images with a high spatial detail of panchromatic (PAN) image to obtain the high resolution multispectral (HRMS) images. In this paper, we present a compressed sensing-based pan-sharpening method that include joint data fidelity and blind blurring kernel estimation. The joint data fidelity contain following three fidelity terms: (1) the LRMS images could be the decimated form of the HRMS images by convolving a blurring kernel, (2) the gradient of HRMS images in the spectrum direction could be proximity to those of the LRMS images, (3) the high frequency part of linear combination of HRMS image bands is approximate to the corresponding parts of the PAN image. Different from other methods which simply apply average blurring kernel for pan-sharpening, a blind deconvolution algorithm is introduced to estimate the blurring kernel from different satellites respectively. We also include a novel anisotropic total variation (TV) prior term to better reconstruct the image edges. The alternating direction method of multipliers (ADMM) is used to solve the proposed model efficiently. Finally, a Pleiades satellite image is employed to demonstrate that the proposed method achieve effective and efficient results simultaneously compared with other existing methods.

Pansharpening is an important technology for obtaining high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) images. Although many pansharpening models have emerged by taking advantage of deep learning (DL) technology, there remains a pressing need to further assess pansharpening accuracy and stability when LRMS images with complex land-cover types. What’s more, these models often overlook the exploitation of PAN images’ inherent high-frequency information. To address these issues, we propose a pansharpening model combining multi-level and multi-scale network architectures. The multi-level network architecture is used to build spatial-spectral dependence on LRMS-PAN pairs, and strengthen the network’s feature capture capability by keeping the multi-level texture details. The multi-scale architecture is subsequently used to extract the spatial structure and deep texture of the PAN images at different scales. Downsampled experiments and real experiments in four standard datasets show that the proposed model achieves a state-of-the-art performance.

Pansharpening is a remote sensing image fusion technique that combines a high-resolution (HR) panchromatic (PAN) image with a low-resolution (LR) multispectral (MS) image to produce an HR MS image. The primary challenge in pansharpening lies in preserving the spatial details of the PAN image while maintaining the spectral integrity of the MS image. To address this, this article presents a generative adversarial network (GAN)-based approach to pansharpening. The GAN discriminator facilitated matching the generated image’s intensity to the HR PAN image and preserving the spectral characteristics of the LR MS image. The performance in generating images was evaluated using the peak signal-to-noise ratio (PSNR). For the experiment, original LR MS and HR PAN satellite images were partitioned into smaller patches, and the GAN model was validated using an 80:20 training-to-testing data ratio. The results illustrated that the super-resolution images generated by the SRGAN model achieved a PSNR of 31 dB. These results demonstrated the developed model’s ability to reconstruct the geometric, textural, and spectral information from the images.

In this paper, a model-based approach to multiresolution fusion of remotely sensed images is presented. Given a high spatial resolution panchromatic (Pan) image and a lowspatial resolution multispectral (MS) image acquired on the same geographical area, the presented method aims to enhance the spatial resolution of the MS image to the resolution of the Pan observation. The proposed fusion technique utilizes the spatial correlation of each of the high-resolution MS channels by using an autoregressive (AR) model, whose parameters are learnt from the analysis of the Pan data. Under the assumption that the parameters of the AR model for the Pan image are the same as those that represent the MS images due to spectral correlation, the proposed technique exploits the learnt parameter values in the context of a proper regularization technique to estimate the high spatial resolution fields for the MS bands. This results in a combination of the spectral characteristics of the low-resolution MS data with the high spatial resolution of the Pan image. The main advantages of the proposed technique are: 1) unlike standard methods proposed in the literature, it requires no registration between the Pan and the MS images; 2) it models effectively the texture of the scene during the fusion process; 3) it shows very small spectral distortion (as it is less affected, compared to standard methods, by the specific digital numbers of pixels in the Pan image, since it exploits the learnt parameters from the Pan image rather than the actual Pan digital numbers for fusion); and 4) it can be used in critical situations in which the Pan and the MS images are acquired (also by different sensors) in slightly different areas. Quantitative experimental results obtained using Landsat-7 Enhanced Thematic Mapper Plus (ETM+) and Quickbird images point out the effectiveness of the proposed method

Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images for various practical tasks such as ground-object classification and change detection. Currently, Transformers have been widely utilized in the pansharpening field because of their capability to model global representations. Transformer-based pansharpening methods capture a global representation of local patch sequences in the spatial domain, but they do not aggregate each pixel in the image, which hinders their ability to capture the global context and dependencies. To solve this issue, a novel pansharpening network based on Fourier with dropout in global modeling (FDGM) is proposed, which can better model global representations and obtain higher-quality fusion images without increasing computational complexity. First, an efficient Fourier with dropout block (FDB) is designed by transforming the features from the spatial domain to the frequency domain to capture the global information from the panchromatic (PAN) and low-resolution multispectral (LRMS) images. This study represents a novel attempt to explore the generalization performance of dropout in the Fourier frequency domain. Furthermore, a residual spatial-frequency fusion module (RSFM) based on frequency channel attention is designed to enhance feature representation capability and effectively facilitate the fusion of the spatial and frequency features. Finally, the shallow features extracted from the source images are combined with the RSFM output to generate the final fusion results. The results of numerous experiments demonstrate that FDGM surpasses other state-of-the-art approaches in visual perception and quantitative evaluation.

In this article, we present a new pansharpening method, a zero-reference generative adversarial network (ZeRGAN), which fuses low spatial resolution multispectral (LR MS) and high spatial resolution panchromatic (PAN) images. In the proposed method, zero-reference indicates that it does not require paired reduced-scale images or unpaired full-scale images for training. To obtain accurate fusion results, we establish an adversarial game between a set of multiscale generators and their corresponding discriminators. Through multiscale generators, the fused high spatial resolution MS (HR MS) images are progressively produced from LR MS and PAN images, while the discriminators aim to distinguish the differences of spatial information between the HR MS images and the PAN images. In other words, the HR MS images are generated from LR MS and PAN images after the optimization of ZeRGAN. Furthermore, we construct a nonreference loss function, including an adversarial loss, spatial and spectral reconstruction losses, a spatial enhancement loss, and an average constancy loss. Through the minimization of the total loss, the spatial details in the HR MS images can be enhanced efficiently. Extensive experiments are implemented on datasets acquired by different satellites. The results demonstrate that the effectiveness of the proposed method compared with the state-of-the-art methods. The source code is publicly available at https://github.com/RSMagneto/ZeRGAN.

In order to overcome the incompatibility of spatial characteristic enhancement and spectral information preservation for remote sensing image fusion, we present a new wavelet fusion algorithm based on color compensation (CC) rule and intensity-hue-saturation (IHS) transform in this paper. First, the RGB representation of the multispectral (MS) image is transformed by IHS into the intensity, hue and saturation components. Then, the panchromatic (PAN) image and the intensity component of multispectral image are decomposed based on the wavelet transform (WT). For detail wavelet coefficients, we introduce CC into local deviation (LD) fusion, and propose a local compensation (LC) fusion rule; and for approximate wavelet coefficients, we combine CC with high pass filtering (HPF), to propose a high-pass-filtering compensation (HC) fusion rule. Finally, perform inverse IHS transform to the wavelet reconstructed new intensity component and the hue and saturation components from the original MS image to reconstruct the fused image. The visual evaluation and statistical analysis both demonstrate that the proposed algorithm not only enhances the details of result image greatly, but also preserves the spectral information of original multispectral image effectively

Interband structure modeling for Pan-sharpening of very high-resolution multispectral images

This paper presents a novel fusion approach using PCA merger based on multiscale decomposition (MSD), combined with region segmentation and support vector machine (SVM), the result is a high spatial resolution multispectral image from a high resolution panchromatic (Pan) image and low resolution multispectral (Ms) images. Principal components analysis (PCA) fusion technique is one of typical fusion methods, and PCA merger based on MSD had been proposed which can obtain better performance. As we know that, in pixel fusion level, the original images are fused as internal region regardless of the contents of images, but in this paper, we perform region segmentation after MSD, because the homogeneous regions have similar features such as color, texture and intensity. Traditionally, various fusion rules can be applied after MSD according to different conditions, however, the crucial problem is which fusion rule should be adopted under given condition, hence we use the SVM to combine the most fusion rules so that can avoid some drawbacks using single fusion rule. To validate our approach, we compare it with several typical fusion approaches, and the best result is obtained using our approach.