Multi-spectral Remote Sensing Image Fusion Method Based on Gradient Moment Matching

Computer based diagnosis of Alzheimer's disease can be performed by dint of the analysis of the functional and structural changes in the brain. Multispectral image fusion deliberates upon fusion of the complementary information while discarding the surplus information to achieve a solitary image which encloses both spatial and spectral details. This paper presents a Non-Sub-sampled Contourlet Transform (NSCT) based multispectral image fusion model for computer-aided diagnosis of Alzheimer's disease. The proposed fusion methodology involves color transformation of the input multispectral image. The multispectral image in YIQ color space is decomposed using NSCT followed by dimensionality reduction using modified Principal Component Analysis algorithm on the low frequency coefficients. Further, the high frequency coefficients are enhanced using non-linear enhancement function. Two different fusion rules are then applied to the low-pass and high-pass sub-bands: Phase congruency is applied to low frequency coefficients and a combination of directive contrast and normalized Shannon entropy is applied to high frequency coefficients. The superiority of the fusion response is depicted by the comparisons made with the other state-of-the-art fusion approaches (in terms of various fusion metrics).

Convolutional neural networks (CNNs) have recently achieved outstanding performance for hyperspectral (HS) and multispectral (MS) image fusion. However, CNNs cannot explore the long-range dependence for HS and MS image fusion because of their local receptive fields. To overcome this limitation, a transformer is proposed to leverage the long-range dependence from the network inputs. Because of the ability of long-range modeling, the transformer overcomes the sole CNN on many tasks, whereas its use for HS and MS image fusion is still unexplored. In this article, we propose a spectral-spatial transformer (SST) to show the potentiality of transformers for HS and MS image fusion. We devise first two branches to extract spectral and spatial features in the HS and MS images by SST blocks, which can explore the spectral and spatial long-range dependence, respectively. Afterward, spectral and spatial features are fused feeding the result back to spectral and spatial branches for information interaction. Finally, the high-resolution (HR) HS image is reconstructed by dense links from all the fused features to make full use of them. The experimental analysis demonstrates the high performance of the proposed approach compared with some state-of-the-art (SOTA) methods.

This paper proposes an efficient and high-fidelity image fusion method based on adaptive smoothing filtering for panchromatic (PAN) and multispectral (MS) image fusion. The scale ratio reflects the ratio of spatial resolution between the panchromatic image and the multispectral image. When facing a multiscale fusion task, traditional methods are unable to simultaneously handle the problems of spectral resolution loss resulting from high scale ratios and the issue of reduced spatial resolution due to low scale ratios. To adapt to the fusion of panchromatic and multispectral satellite images of different scales, this paper improves the problem of the insufficient filtering of high-frequency information of remote sensing images of different scales by the classic filter-based intensity modulation (SFIM) model. It uses Gaussian convolution kernels instead of traditional mean convolution kernels and builds a Gaussian pyramid to adaptively construct convolution kernels of different scales to filter out high-frequency information of high-resolution images. It can adaptively process panchromatic multispectral images of different scales, iteratively filter the spatial information in panchromatic images, and ensure that the scale transformation is consistent with the definition of multispectral images. Using 15 common fusion methods, this paper compares the experimental results of ZY-3 with scale ratio 2.7 and SV-1 with scale ratio 4 data. The results show that the method proposed in this paper retains good spatial information for image fusion at different scales and has good spectral preservation.

Hyperspectral and Multispectral (HS-MS) image fusion is a most trending technology that enhance the quality of hyperspectral image. By this technology, retrieve the precise information from both HS and MS images combined together increase spatial and spectral quality of the image. In the past decades, many image fusion techniques have been introduced in literature. Most of them using Coupled Nonnegative matrix factorization (CNMF) technique which is based on Linear Mixing Model (LMM) which neglect the nonlinearity factors in the unmixing and fusion technique of the hyperspectral images. To overcome this limitation, we are going to propose an unmixing based fusion algorithm namely Multiplicative Iterative Nonlinear Constrained Coupled Nonnegative Matrix Factorization (MINC-CNMF) that enhance the spatial quality of the image by considering the nonlinearity factor associated with the unmixing process of in the image. This method not only consider the spatial quality but also enhance the spectral data by imposing constraints known as minimum volume (MV) which helps to estimate accurate endmembers. We also measure the strength and superiority of our method against baseline methods by using four public dataset and found that our method shows outstanding performance than all the baseline methods.

In order to solve the problems that the image fusion method based on separable discrete wavelet transform is lower in spatial resolution and there is block effect in fused image, a new multispectral image fusion method based on non-separable wavelets with compactly support, symmetry, orthogonality, and dilation matrix [2,0;0,2] is proposed. A construction method of four channels 6 × 6 filter banks is presented. Using the low-pass filter constructed, multispectral images are fused. Three fusion methods called NAWS, NAWRGB and NAWL are proposed in the fusion of multispectral image and panchromatic image. Every fusion method presented outperforms the corresponding fusion method of the AWS, the AWRGB and the AWL in preserving high spatial resolution information respectively, and the higher spatial resolution fused image can be obtained. Of all fusion methods, the non-separable additive wavelet substitution (NAWS) method has the best performance in preserving higher spatial resolution information.KeywordsImage fusionNon-separable waveletsMultispectral imagePanchromatic image

The IEEE Data Fusion Technical Committee is organizing a “Data Fusion Contest” in conjunction with the IGARS Symposium. For the first issue of the contest, the fusion of multispectral and panchromatic images for optical very high resolution images is considered. A comparative study of the results provided by the participants on common data sets is presented during the conference in the “Data Fusion” session and the “winning” team(s) is (are) granted an IEEE award during the IEEE GRS-S Technical Committee and Chapter Dinner.

Biometrics recognition takes advantage of feature extraction and pattern recognition to analyze the physical and behavioral characteristics of biological individuals to achieve the purpose of individual identification. As a typical biometric technology, palm print and palm vein have the characteristics of high recognition rate, stable features, easy location and good image quality, which have attracted the attention of researchers. This paper designs and develops a multispectral palm print and palm vein acquisition platform, which can quickly acquire palm spectrum and palm vein multispectral images with seven different wavelengths. We propose a multispectral palm print palmar vein recognition framework, and feature-level image fusion is performed after extracting features of palm print palmar vein images at different wavelengths. Through the multispectral palm print palm vein image fusion experiment, a more feasible multispectral palm print and palm vein image fusion scheme is proposed. Based on the results of image fusion, we further propose an improved convolutional neural network (CNN) for model training to achieve identity recognition based on multispectral palm print palm vein images. Finally, the effects of different CNN network structures and learning rates on the recognition results were analyzed and compared experimentally.

CBERS-02B satellite has a high resolution camera HR and a multi-spectral sensor CCD.HR and CCD images can be fused to form a new image which can preserve high spatial resolution of HR and high spectral resolution of CCD.HR image can also be fused with images of other sensors.The authors used six methods to fuse HR,CCD and SPOT 5 multi-spectral images and evaluated the fusion results qualitatively and quantitatively.As a result,relatively ideal methods for fusing HR and SPOT 5 multi-spectral images were found.The results indicate the potential capability of HR images for being fused with images of other sensors.A comparison has also been made between the HR SPOT 5 multi-spectral fusion image and the HR CCD fusion image.

In recent years, the fusion of hyperspectral and multispectral images in remote sensing image processing still faces challenges, primarily due to their complexity and multimodal characteristics. Diffusion models, known for their stable training process and exceptional image generation capabilities, have shown good application potential in this field. However, when dealing with multimodal data, it may prove challenging for the models to fully capture the intricate relationships between the modalities, which may result in incomplete information integration and a small amount of remaining noise in the generated images. To address these problems, we propose a new model, KanDiff, for hyperspectral and multispectral image fusion. To address the differences in modal information between multispectral and hyperspectral images, KANDiff incorporates Kolmogorov–Arnold Networks (KAN) to guide the inputs. It helps the model understand the complex relationships between the modalities by replacing the fixed activation function in the traditional MLP with a learnable activation function. Furthermore, the image generated by the diffusion model may exhibit a small amount of the remaining noise. Convolutional Neural Networks (CNNs) effectively extract local features through their convolutional layers and achieve noise suppression via layer-by-layer feature representation. Therefore, the MergeCNN module is further introduced to enhance the fusion effect, resulting in smoother and more accurate outcomes. Experimental results on the public CAVE and Harvard datasets indicate that KanDiff has achieved improvements over current high-performance methods across several metrics, particularly showing significant enhancements in the peak signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM), thus demonstrating superior performance. Additionally, we have created an image fusion dataset of the lunar surface, and KANDiff exhibits robust performance on this dataset as well. This work introduces an effective solution for addressing the challenges posed by missing high-resolution hyperspectral images (HRHS) data, which is essential for tasks including landing site selection and resource exploration within the realm of deep space exploration.

Hyperspectral (HS) remote sensing image with finer spectral information has great advantages in feature identification and classification. However, the spatial resolution of HS image is usually low due to practical limitations. In this paper, the low-spatial-resolution HS image is fused with the high-spatial-resolution multispectral (MS) image of the same observation scene to improve its spatial resolution. A novel spectral unmixing based HS and MS image fusion approach (VSC-CNMF) is proposed, in which CNMF with minimum endmember simplex volume and abundance sparsity constraints is employed for coupled unmixing of HS and MS images. Simulative experiments are employed for verification and comparison. The experimental results illustrate that the newly proposed VSC-CNMF based HS and MS fusion algorithm outperforms several state-of-the-art unmixing based fusion approaches in cases with moderate number of endmembers.

Although the fusion of multispectral (MS) and hyperspectral (HS) images in remote sensing has become relatively mature, and different types of fusion methods have their own characteristics in terms of fusion effect, data dependency, and computational efficiency, few studies have focused on the impact of radiance extreme areas, which widely exist in real remotely sensed scenes. To this end, this paper proposed a novel method called radiance extreme area compensation fusion (RECF). Based on the architecture of spectral unmixing fusion, our method uses the reconstruction of error map to construct local smoothing constraints during unmixing and utilizes the nearest-neighbor multispectral data to achieve optimal replacement compensation, thereby eliminating the impact of overexposed and underexposed areas in hyperspectral data on the fusion effect. We compared the RECF method with 11 previous published methods on three sets of airborne hyperspectral datasets and HJ2 satellite hyperspectral data and quantitatively evaluated them using 5 metrics, including PSNR and SAM. On the test dataset with extreme radiance interference, the proposed RECF method achieved well in the overall evaluation results; for instance, the PSNR metric reached 47.6076 and SAM reached 0.5964 on the Xiong’an dataset. In addition, the result shows that our method also achieved better visual effects on both simulation and real datasets.

Hyperspectral (HS) and multispectral (MS) image fusion aims at producing high-resolution HS (HRHS) images. However, the existing methods could not simultaneously consider the structures in both the spatial and spectral domains of the HS cube. In order to effectively preserve spatial–spectral structures in HRHS images, we propose a new low-resolution HS (LRHS) and high-resolution MS (HRMS) image fusion method based on spatial–spectral-graph-regularized low-rank tensor decomposition (SSGLRTD) in this paper. First, we reformulate the image fusion problem as a low-rank tensor decomposition model to utilize the low-rank property in the HS image. Then, two graphs are constructed in spatial and spectral domains, respectively. One of them is derived from the HRMS image for the spatial consistency, and the other is inferred from the LRHS image for spectral smoothness. Finally, the SSGLRTD fusion framework is established by combining all regularizers. With these two graphs, the spatial correlation and the spectral structure in the fused HRHS images are efficiently preserved. The experimental results on different datasets reveal that the proposed fusion method outperforms several existing fusion methods in terms of visual analysis and numerical comparison.

Recent advances and new guidelines on hyperspectral and multispectral image fusion

This paper studies a new Bayesian optimization algorithm for fusing hyperspectral and multispectral images. The hyperspectral image is supposed to be obtained by blurring and subsampling a high spatial and high spectral target image. The multispectral image is modeled as a spectral mixing version of the target image. By introducing appropriate priors for parameters and hyperparameters, the fusion problem is formulated within a Bayesian estimation framework, which is very convenient to model the noise and the target image. The high spatial resolution hyperspectral image is then inferred from its posterior distribution. To compute the Bayesian maximum a posteriori estimator associated with this posterior, an alternating direction method of multipliers within block coordinate descent algorithm is proposed. Simulation results demonstrate the efficiency of the proposed fusion method when compared with several state-of-the-art fusion techniques.

There are various sensors invented to capture the images of earth surface which are used to analyze the earth surface. The images captured by different sensors have different features. To analyze the earth surface images captured at different spectrums are used. In urban area characteristic of urban object are determined by not only their spectra but also by their structure. Various sensors captures the image at various spectrums knows as multispectral image but is with low spatial resolution. Some sensors captures the high resolution image also known as panchromatic image. Therefore it becomes very necessary and useful to fuse panchromatic and multispectral image to detect different area from the image easily. The fused image is multispectral image with high spatial resolution. The various methods of image fusion studied in this paper. Comparison of them is done is done statistically and visually and it is found that fused image is of greater quality and clarity than the original one and shows the features of that area more accurately.