Adaptive steepest descent method for pan-sharpening of multispectral images

In this paper, a model-based approach to multiresolution fusion of remotely sensed images is presented. Given a high spatial resolution panchromatic (Pan) image and a lowspatial resolution multispectral (MS) image acquired on the same geographical area, the presented method aims to enhance the spatial resolution of the MS image to the resolution of the Pan observation. The proposed fusion technique utilizes the spatial correlation of each of the high-resolution MS channels by using an autoregressive (AR) model, whose parameters are learnt from the analysis of the Pan data. Under the assumption that the parameters of the AR model for the Pan image are the same as those that represent the MS images due to spectral correlation, the proposed technique exploits the learnt parameter values in the context of a proper regularization technique to estimate the high spatial resolution fields for the MS bands. This results in a combination of the spectral characteristics of the low-resolution MS data with the high spatial resolution of the Pan image. The main advantages of the proposed technique are: 1) unlike standard methods proposed in the literature, it requires no registration between the Pan and the MS images; 2) it models effectively the texture of the scene during the fusion process; 3) it shows very small spectral distortion (as it is less affected, compared to standard methods, by the specific digital numbers of pixels in the Pan image, since it exploits the learnt parameters from the Pan image rather than the actual Pan digital numbers for fusion); and 4) it can be used in critical situations in which the Pan and the MS images are acquired (also by different sensors) in slightly different areas. Quantitative experimental results obtained using Landsat-7 Enhanced Thematic Mapper Plus (ETM+) and Quickbird images point out the effectiveness of the proposed method

In this paper, we propose a novel method to enhance the pan-sharpening result of low-resolution multispectral images (MS) and high-resolution panchromatic images (Pan) by minimizing the spectral distortion engendered by the fusion process. In fact, spectral distortion is the most significant problem in many pan-sharpening techniques, due to the non linearity between Pan and MS images. In this method, an improvement of the Pan image is performed in order to enhance the correlation between Pan and MS images before pan-sharpening process. The proposed method is applied as a preprocessing by fusing the intensity image derived from MS image with the original Pan to get an improved Pan image which could be more correlated with MS image. And later, the pan-sharpening is applied on both MS and the improved Pan using any pan-sharpening technique. Simulation results of proposed method are compared in four different techniques, such as: Generalized IHS, DWT, Brovey and HPF. It has been observed that simulation results of this method preserves more spectral information and gets better visual quality compared with earlier reported techniques using original Pan.KeywordsPan-sharpeningFusionPanchromaticMultispectralSpectral distortion

A novel method for multispectral image pansharpening based on high dimensional model representation

Remote sensing image fusion using the curvelet transform

Most modern satellites can provide two types of images: 1) panchromatic (PAN) image and 2) multispectral (MS) image. The former has high spatial resolution and low spectral resolution, while the latter has high spectral resolution and low spatial resolution. To obtain images with both high spectral and spatial resolution, pansharpening has emerged to fuse the spatial information of the PAN image and the spectral information of the MS image. However, most pansharpening methods fail to preserve spatial and spectral information simultaneously. In this article, we propose a framelet-based convolutional neural network (CNN) for pansharpening which makes it possible to pursue both high spectral and high spatial resolution. Our network consists of three subnetworks: 1) feature embedding net; 2) feature fusion net; and 3) framelet prediction net. Different from conventional CNN methods directly inferring high-resolution MS images, our approach learns to predict their framelet coefficients from available PAN and MS images. The introduction of multilevel feature aggregation and hybrid residual connection makes full use of spatial information of PAN image and spectral information of MS image. Quantitative and qualitative experiments at reduced- and full-resolution demonstrate that the proposed method achieves more appealing results than other state-of-the-art pansharpening methods. The source code and trained models are available at https://github.com/TingMAC/FrMLNet.

Remote sensing image fusion (or pan-sharpening) aims at generating high resolution multi-spectral (MS) image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral image. In this paper, a deep convolutional neural network with two-stream inputs respectively for PAN and MS images is proposed for remote sensing image pan-sharpening. Firstly the network extracts features from PAN and MS images, then it fuses them to form compact feature maps that can represent both spatial and spectral information of PAN and MS images, simultaneously. Finally, the desired high spatial resolution MS image is recovered from the fused features using an encoding-decoding scheme. Experiments on Quickbird satellite images demonstrate that the proposed method can fuse the PAN and MS image effectively.

<p>Earth observation satellites capture panchromatic images at high spatial resolution and multispectral images at lower resolution to optimize the use of their onboard energy sources. This results in a technical necessity to synthesize high-resolution multispectral images from these data. Pansharpening techniques aim to combine the spatial detail of panchromatic images with the spectral information of multispectral images. However, due to the discrete nature of these images and their varying local statistical properties, many pansharpening methods suffer from numerical artifacts such as chromatic and spatial distortions. This paper introduces the L0-Norm-based pansharpening method (L0pan), which addressed these challenges by maximizing the number of similar pixels between the synthesized pansharpened image and the original panchromatic and multispectral images. L0pan was optimized using a population-based colony search algorithm, enabling it to effectively balance both chromatic fidelity and spatial resolution. Extensive experiments across nine different datasets and comparison with nine other pansharpening methods using ten quality metrics demonstrated that L0pan significantly outperformed its counterparts. Notably, the colony search algorithm yielded the best overall results, highlighting the algorithm's strength in refining pansharpening accuracy. This study contributed to the advancement of pansharpening techniques, offering a method that preserved both chromatic and spatial details more effectively than existing approaches.</p>

Interband structure modeling for Pan-sharpening of very high-resolution multispectral images

Hyperspectral images (HSI) have high spectral and low spatial resolutions. However, multispectral images (MSI) usually have low spectral and high spatial resolutions. In various applications HSI with high spectral and spatial resolutions are required. In this paper, a new method for spatial resolution enhancement of HSI using high resolution MSI based on sparse coding and linear spectral unmixing (SCLSU) is introduced. In the proposed method (SCLSU), high spectral resolution features of HSI and high spatial resolution features of MSI are fused. In this case, the sparse representation of some high resolution MSI and linear spectral unmixing (LSU) model of HSI and MSI is simultaneously used in order to construct high resolution HSI (HRHSI). The fusion process of HSI and MSI is formulated as an ill-posed inverse problem. It is solved by the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) and an orthogonal matching pursuit (OMP) algorithm. Finally, the proposed algorithm is applied to the Hyperion and ALI datasets. Compared with the other state-of-the-art algorithms such as Coupled Nonnegative Matrix Factorization (CNMF) and local spectral unmixing, the SCLSU has significantly increased the spatial resolution and in addition the spectral content of HSI is well maintained.

As a branch of Internet of Things (IoT), network geographic information (WebGIS) has the task of enhancing remote sensing images, including correction, fusion, mapping, etc. Remote sensing images captured by different satellite platforms often present earth objects with various scales. Due to the different spatial resolutions of imaging sensors, the same object will appear in different sizes. Usually panchromatic (PAN) image, without spectral diversity, has high spatial resolution and multispectral (MS) image has lower spatial resolution but more spectral information. Fusing MS and PAN images, also known as pansharpening, will increase the spatial resolution of MS image without changing its spectral information. In the fusion process, inaccurate perception and extraction of the spatial details will lead to spectral and spatial distortions in the fused image. A new multi-scale structure perception based fusion method is proposed in this paper. Firstly, an improved structure-preserving filter is designed by using MS imagery as the reference to capture the structures in the PAN image. Then the structures having different scales are progressively perceived through a multi-scale decomposition scheme and the spatial details are accurately extracted from the PAN image. Finally, the details weighted by the band-dependent gains are injected into each band of MS images. Experimental results are compared with some recently proposed edge-preserving filtering based fusion methods. Visual analysis and objective evaluations demonstrate that the proposed method can produce high-quality fused output and achieve better performance than some state-of-the-arts.KeywordsFusionRemote sensingFilterMultispectral image

Pansharpening usually improves the resolution of low-resolution multispectral (LRMS) images with spatial information from corresponding high-resolution panchromatic (HRPAN) images to produce high-resolution MS (HRMS) images. Traditional pansharpening methods use various domain transformations to make the fused image suffer varying degrees of spatial or spectral distortion because the information in the LRMS and PAN images is heterogeneous and distributed in different domains. The motivation of our proposed work is to develop a balanced and robust pansharpening method named cross-domain correspondence intensity modulation, which is based on Bayesian decision-making for remote sensing image pansharpening. First, the intensity component of the MS image is obtained via the intensity hue saturation (IHS) transform. Second, a fusion rule based on the Bayesian probabilistic model is designed to fuse the intensity component and the corresponding PAN image to obtain an intermediate component. Third, a cross-domain correspondence intensity modulation algorithm is proposed to modulate the intensity information in the intermediate component to produce the desired intensity component. Finally, an inverse IHS transformation is performed to obtain the pansharpened MS image by replacing the original intensity component with the modulated intensity component. The results on different satellite datasets show that the proposed method can effectively enhance the spatial and spectral fidelity of the fused image.

Pansharpening is a process of acquiring a multi-spectral image with high spatial resolution by fusing a low resolution multi-spectral image with a corresponding high resolution panchromatic image. In this paper, a new pansharpening method based on the Bayesian theory is proposed. The algorithm is mainly based on three assumptions: 1) the geometric information contained in the pan-sharpened image is coincident with that contained in the panchromatic image; 2) the pan-sharpened image and the original multi-spectral image should share the same spectral information; and 3) in each pan-sharpened image channel, the neighboring pixels not around the edges are similar. We build our posterior probability model according to above-mentioned assumptions and solve it by the alternating direction method of multipliers. The experiments at reduced and full resolution show that the proposed method outperforms the other state-of-the-art pansharpening methods. Besides, we verify that the new algorithm is effective in preserving spectral and spatial information with high reliability. Further experiments also show that the proposed method can be successfully extended to hyper-spectral image fusion.

In remote sensing, satellite images acquired from sensors provide either high spectral or high spatial resolution. The pansharpening framework is applied to remote-sensing systems to enhance the spatial quality of coarse-resolution multispectral (MS) images using information from panchromatic imagery. A multidecomposition pansharpening approach combining MS and panchromatic (PAN) images is proposed in this paper in order to bring the resolution of the low-resolution MS imagery up to that of the panchromatic images. In particular, multilevel wavelet decomposition is applied to the luminance-chrominance (YUV) space transformation (taking into account the red green and blue (RGB) bands) or extended-YUV transformation (taking into account the near infrared (NIR) band in addition to RGB) of the original MS channels, where geometrical details from the panchromatic image are introduced into the MS ones. Our approach contains a preprocessing step that consists of homogenizing the luminance, Y, and the panchromatic image reflectance, which are, respectively, a value integrated over a wavelength spectrum and simply a linear combination of some values in the same spectrum. Hence, as the panchromatic image reflectance and luminance reflectance correspond to different measurements, they do not correspond to the same physical information, which results in a difference between their histograms. Therefore, simple histogram matching is traditionally applied to panchromatic data to fit it to the luminance to avoid colour distortion after fusion. However, as the transformation concerns just the details of the panchromatic and MS images, a new scheme for matching the images which ignores the divergence between their approximations and maximizes the resemblance between their details is proposed in this work. After that, the fusion approach is applied, and in contrast to the original approach where the details of the fused MS luminance are set equal to the PAN luminance, we propose an adaptive approach in which just a part of the PAN details proportional to the similarity between the luminance and lowered PAN image is taken. Indeed, high-resolution geometrical details cannot be similar if the low-resolution details are not in good agreement. Besides, as the agreement between PAN and MS images depends on the occupation class, we have created a segmentation map and then computed separately the correlation in each region. Finally, the evaluation is done based on QuickBird and Pleiades-1A data sets showing rural and suburban areas. When compared to recent methods, our approach provides better results.

There are several situations that simultaneously require both high spatial and spectral resolution in a single image. In literature a certain number of methods have been proposed to merge panchromatic and multispectral data. Classical processes involve a simple linear transformation of the combined data spaces (Brovey, HSV, PCA). However standard data fusion methods may not be satisfactory to merge a high-resolution panchromatic image and a low-resolution multispectral image because they distort the spectral characteristics of the original multispectral data. It has been shown that an image fusion technique based on multiresolution wavelet decomposition (MWD) seems to be a better trade-off between spectral and spatial information. In this paper, we develop some techniques, based on MWD, for the data fusion of such images. The methods presented for reducing the fusion imperfections don't consist simply of replacing some wavelet coefficients of the multispectral image by the corresponding coefficients of the high-resolution image but tend to carry out a real. integration between two pyramidal decomposition. In fact, it is possible to fuse the wavelet coefficients of two decompositions in each scale level and with some appropriate rules. Moreover, some ad hoc algorithms have been implemented to correct spectral distortions in the synthesized high resolution multispectral image. We tested the methods to merge IKONOS panchromatic (1 m spatial resolution) and IKONOS multispectral (4 m spatial resolution) images in urban areas. The techniques presented are clearly better than classical fusion methods in preserving both spectral and spatial information on urban areas.

This paper proposes an improved framelet-based pan-sharpening algorithm for Thailand Earth Observation System (THEOS) imagery to decrease the effects of different acquisition times between panchromatic (Pan) and multispectral (MS) images, in which the smoothing filter-based intensity modulation (SFIM) is introduced into low-frequency information fusion instead of the conventional “mean” rule. Moreover, a two-layer procedure is presented to reduce the impacts of mixed pixels caused by the large difference of spatial resolutions between the Pan and MS images. The proposed method is tested on two THEOS datasets and compared with the Gram–Schmidt, SFIM and traditional framelet-based methods. The portability across contourlet transform is also examined. Both qualitative and quantitative evaluation results demonstrate that the proposed method is more independent of the illumination of the Pan image and can achieve better spectral fidelity while maintaining spatial sharpness.