A New Pan-Sharpening Method Using a Compressed Sensing Technique

This paper proposes a pan-sharpening method based on multi-direction subbands and deep neural networks. First, by utilizing the multi-scale and multi-direction properties of the nonsubsampled contourlet transform (NSCT), panchromatic (PAN) image is decomposed into the low frequency subbands in different resolutions and the high frequency subbands in different directions. Pan-sharpening method aims to fuse the high frequency subband coefficients of PAN image and the low frequency subband coefficients of multispectral (MS) image. Second, in order to better extract the feature of the high frequency subbands in different directions of PAN image, the deep neural network (DNN) is trained using the image patches of high frequency subbands of PAN image. Third, in the fusion stage, we exploit NSCT on the principal component of resampled low resolution (LR) MS image. The high frequency subbands of output high resolution (HR) MS image is obtained by forward propagation of the trained DNN, which input is the high frequency subbands of LR MS image. Finally, a new subband set is obtained by fusing the reconstructed high frequency subband and the original low frequency subband of LR MS image. The HR MS image is produced by executing the inverse transform of NSCT and adaptive PCA (A-PCA) on the new subband set. The experimental results show the proposed method outperforms other well-known methods in terms of both objective measurements and visual evaluation.

We know that COVID-19 has been considered a pandemic and various types of symptoms are analyzed by the doctors. Various cases belonging to COVID-19 are asymptomatic and due to this fact the disease is not analyzed at an initial stage and the condition of the patient will be critical. So, the purpose of this work is to provide a solution that will find out the highly precise test result of COVID-19. Magnetic Resonance Imaging, CT Scan & Lung Ultrasound are some of the methods which can provide the exact results for the testing. But the problem associated with these imaging modalities is that they are time-consuming and the data provided by these modalities are large enough to store or transmit. A compression technique is required which can reduce the time as well as data size. Computed Tomography with Compressive Sensing (CS) Technique is used as an approach to tackle the above-stated problem. To analyze the fact that this technique is efficient, we consider the Computed Tomography-based Chest images of COVID-19 infected patients and apply the CS technique (Basis Pursuit) with Discrete Cosine Transform as a representation basis and Gaussian as a measurement matrix. As a result of this study, we find out three parameters, PSNR, SSIM & FSIM, to visualize the efficiency of the reconstruction strategy. This work concludes that the Computed Tomography approach with the help of CS can be used for fast and efficient imaging for COVID-19 as well as other diseases of the same kind.

Distributed compressed sensing-based pan-sharpening with hybrid dictionary

High spatial resolution multispectral (HMS) images can provide sufficient information for researchers to analyze the potential disasters in the living environment. However, an original multispectral (MS) image is with low-space-resolution and high-spectrum-resolution, while an original panchromatic (PAN) image has the opposite property. Pansharpening aims at obtaining HMS image by retaining spectrum of the MS image and injecting details of the PAN image simultaneously. In this paper, we present a new pansharpening method. First, we use a bilateral filter (BF) to obtain the low-frequency-component (LFC) of PAN and MS images, respectively. Then the high-frequency-component (HFC) of PAN and MS images are readily obtained. Second, an adaptive intensity-hue-saturation (AIHS) based method is applied to generate the HFC of intensity. Finally, a dual-scale guided image filter (GIF) is utilized to calculate the difference between HFC of intensity and PAN to get the detail images. And then, these detail images are injected into the original MS image to achieve the HMS image. The proposed method is applied into testing various satellite data sets, and performs better effect on both visual quality and objective indictors than the existing methods.

In this article, we present a new pansharpening method, a zero-reference generative adversarial network (ZeRGAN), which fuses low spatial resolution multispectral (LR MS) and high spatial resolution panchromatic (PAN) images. In the proposed method, zero-reference indicates that it does not require paired reduced-scale images or unpaired full-scale images for training. To obtain accurate fusion results, we establish an adversarial game between a set of multiscale generators and their corresponding discriminators. Through multiscale generators, the fused high spatial resolution MS (HR MS) images are progressively produced from LR MS and PAN images, while the discriminators aim to distinguish the differences of spatial information between the HR MS images and the PAN images. In other words, the HR MS images are generated from LR MS and PAN images after the optimization of ZeRGAN. Furthermore, we construct a nonreference loss function, including an adversarial loss, spatial and spectral reconstruction losses, a spatial enhancement loss, and an average constancy loss. Through the minimization of the total loss, the spatial details in the HR MS images can be enhanced efficiently. Extensive experiments are implemented on datasets acquired by different satellites. The results demonstrate that the effectiveness of the proposed method compared with the state-of-the-art methods. The source code is publicly available at https://github.com/RSMagneto/ZeRGAN.

The fusion of high spatial resolution panchromatic (PAN) images with high spectral resolution multispectral (MS) images provides high resolution images in both spatial and spectral domains. In this work, a pansharpening method based on the fast weighted least squares (WLS) smoothing filter has been proposed. In this method, input PAN image has been decomposed into its approximation and detail sub bands by a multiscale scheme based on fast WLS filtering. The details extracted from the PAN image has been added proportionally the expanded MS image. The resulting images as well as evaluation metrics demonstrate that the proposed injection approach shows better performance than the traditional methods.

Pan-sharpening is a fundamental task for remote sensing image processing. It aims at creating a high-resolution multispectral (HRMS) image from a multispectral (MS) image and a panchromatic (PAN) image. In this article, a new band-independent encoder-decoder network is proposed for pan-sharpening. The network takes a single band of the MS (BMS) image, the PAN image, and the low-resolution PAN (LRPAN) image as inputs. The output of the network is the corresponding band of high-resolution MS (HRBMS) image. In this way, the network can process MS images with any number of bands. The overall structure of the network consists of two encoder-decoder modules at low-resolution and high-resolution, respectively. An auxiliary LRPAN image is used to speed up the training and improve the performance. The partly shared network and hierarchical structure for low-resolution and high-resolution enable a better fusion of features extracted from different scales. With a fast fine-tuning strategy, the trained model can be applied to images from different sensors. Experiments performed on different data sets demonstrate that the proposed method outperforms several state-of-the-art pan-sharpening methods in both visual appearance and objective indexes, and the single-band evaluation results further verify the superiority of the proposed method.

Pansharpening technology plays a crucial role in remote sensing image processing by integrating low-resolution multispectral (LRMS) images and high-resolution panchromatic (PAN) images to generate high-resolution multispectral (HRMS) images. This process addresses the limitations of satellite sensors, which cannot directly capture HRMS images. Despite significant developments achieved by deep learning-based pansharpening methods over traditional approaches, most existing techniques either fail to account for the modal differences between LRMS and PAN images, relying on direct concatenation, or use similar network structures to extract spectral and spatial information. Additionally, many methods neglect the extraction of common features between LRMS and PAN images and lack network architectures specifically designed to extract spectral features. To address these limitations, this study proposed a novel three-branch pansharpening network that leverages both spatial and frequency domain interactions, resulting in improved spectral and spatial fidelity in the fusion outputs. The proposed method was validated on three datasets, including IKONOS, WorldView-3 (WV3), and WorldView-4 (WV4). The results demonstrate that the proposed method surpasses several leading techniques, achieving superior performance in both visual quality and quantitative metrics.

Pansharpening is to fuse high-resolution panchromatic (PAN) images with low-resolution multispectral (LR-MS) images to generate high-resolution multispectral (HR-MS) images. Most of the deep learning-based pansharpening methods did not consider the inconsistency of the PAN and LR-MS images and used simple concatenation to fuse the source images, which may cause spectral and spatial distortion in the fused results. To address this problem, a multi-scale spatial-spectral attention guided fusion network for pansharpening is proposed. First, the spatial features from the PAN image and spectral features from the LR-MS image are independently extracted to obtain the shallow features. Then, a spatial-spectral attention feature fusion module (SAFFM) is constructed to guide the reconstruction of spatial-spectral features by generating a guidance map to achieve the fusion of reconstructed features at different scales. In SAFFM, the guidance map is designed to ensure the spatial-spectral consistency of the reconstructed features. Finally, considering the difference between multiply scale features, a multi-level feature integration scheme is proposed to progressively achieve fusion of multi-scale features from different SAFFMs. Extensive experiments validate the effectiveness of the proposed network against other state-of-the-art (SOTA) pansharpening methods in both quantitative and qualitative assessments. The source code will be released at https://github.com/MELiMZ/ssaff.

Pan-sharpening is a process of acquiring a high resolution multispectral (MS) image by combining a low resolution MS image with a corresponding high resolution panchromatic (PAN) image. In this paper, we propose a new variational pan-sharpening method based on three basic assumptions: 1) the gradient of PAN image could be a linear combination of those of the pan-sharpened image bands; 2) the upsampled low resolution MS image could be a degraded form of the pan-sharpened image; and 3) the gradient in the spectrum direction of pan-sharpened image should be approximated to those of the upsampled low resolution MS image. An energy functional, whose minimizer is related to the best pan-sharpened result, is built based on these assumptions. We discuss the existence of minimizer of our energy and describe the numerical procedure based on the split Bregman algorithm. To verify the effectiveness of our method, we qualitatively and quantitatively compare it with some state-of-the-art schemes using QuickBird and IKONOS data. Particularly, we classify the existing quantitative measures into four categories and choose two representatives in each category for more reasonable quantitative evaluation. The results demonstrate the effectiveness and stability of our method in terms of the related evaluation benchmarks. Besides, the computation efficiency comparison with other variational methods also shows that our method is remarkable.

Time delay estimation (TDE) is a hot research topic in wireless location technology. Compressed sensing (CS) theory has been widely applied to image reconstruction and direction of arrival estimation since it was proposed in 2004. The sparse model can be constructed in time domain for estimating the time delay by using the CS theory. The measurement matrix plays a crucial role in the processing of signal reconstruction which is the core problem of CS theory. Therefore the research in the measurement matrix has becomes a hotspot in recent years. The existing measurement matrix is mainly divided into two categories, i.e., random measurement matrix and deterministic measurement matrix. The performance of random measurement matrix has bottlenecks. Firstly, because of the redundant measurement matrix data, the generation and storage of the random number put forward a high requirement for hardware. Secondly the random matrix can only satisfy the restricted isometry property in a statistical sense. The research of the deterministic measurement matrix is of great value under this background. The parity check matrix of low density parity check (LDPC) code has good performance in CS theory. However, the method of randomly selecting non-zero element position has a certain probability to generate a measurement matrix with a short loop structure during generating LDPC code measurement matrix. The robustness of the reconstruction performance decreases with the increase of iteration times. A novel quasi-cyclic CS algorithm based on progressive edge-growth is constructed to estimate the time delay. The purpose of this article is to deal with the need to store a large number of data in existing measurement matrix during time delay, by using the CS theory. The algorithm presented here can achieve TDE in a high precision. First, the theoretical bridge between CS and the maximum likelihood decoding is established. And the design criterion of measurement matrix based on the LDPC code is derived. The sparse measurement matrix with quasi-cyclic structure is constructed by introducing the idea of progressive edge-growth. Finally, the orthogonal matching pursuit algorithm is used to estimate the time delay. Furthermore, the computational complexity of the algorithm and the data storage of the measurement matrix are analyzed theoretically. Simulations show that the correct reconstruction probability of the proposed approach is higher than those of the Gauss random matrix and random LDPC matrix under the same dimension. Compared with the random LDPC matrix, the proposed method can improve performance at the expense of less complexity under the condition of the same data storage.

Deep unfolding networks have obtained satisfactory performance in the pansharpening task owing to their sufficient interpretability. Inspired by the back-projection (BP) mechanism, we propose a BP-driven model, spatial-spectral dual back-project network (S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DBPN), to fuse the low spatial resolution multispectral (LR MS) and the high spatial resolution panchromatic (PAN) images by exploiting the BP in spatial and spectral domains. Specifically, the proposed S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DBPN is made up of a spatial BP network, a spectral BP network, and a reconstruction network. In the spatial BP network, spatial down- and up-projection modules are derived from BP, which is responsible for the projection of the LR MS image into the spatial domain. By analogy with the spatial BP, we reformulate the degradation between high spatial resolution multispectral (HR MS) and PAN images as spectral down- and up-projections. Then, the spectral BP network is constructed for the projection of the PAN image along the channel dimension. Finally, the features from spatial and spectral BP networks are integrated to produce the desired HR MS image through the reconstruction network. Compared to the state-of-the-art methods, extensive experiments on QuickBird, GeoEye-1, and WorldView-2 datasets demonstrate that our S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DBPN produces better HR MS images in terms of qualitative and quantitative evaluation metrics. The code of S <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> DBPN is released at: https://github.com/RSMagneto/S2DBPN.

Compressive sensing (CS) is a new technique that give an approach to reconstruct the signal with few numbers of observations or measurements. The CS is based on L1 -norm minimizations to find the sparse solutions and it is known as basis pursuit. In this investigation, CS scheme with different transforms is proposed. The three utilized transform techniques are: Discrete Fourier Transform(DFT), Discrete Cosine Transform(DCT) and Discrete Wavelet Transform(DWT) with daubechies1(DB1) and coiflets1(coif1) basis. The proposed system is tested by employing the following signals: Blocks, Heavy Sine, ‘Bumps’ and ‘Doppler’ which cover wide range of applications. The four testing signals are represented in sparse domain using different transforms. In order to threshold the coefficients of the signals in sparse domain, the universal threshold is utilized in the case of CS with FFT and DWT whereas, the universal threshold is modified to prune the DCT coefficients. The main aim of this study is to investigate the differences among CS with DFT, CS with DCT and CS with DWT, and consequently a suitable transform domain used with CS to be selected. The comparative study is established by assessing the performance of the proposed system using Root Mean Square Error (RMSE), output SNR, and the time required to reconstruct approximated signals. Simulation results have shown that the CS with DWT outperforms the CS with FFT and DCT. CS with DWT has achieved good RMSE values about (0.0014 to 3.359e-8) even when half of the signal elements are removed. CS with FFT and DCT enhanced the noisy Blocks and Bumps signals by 3dB and 1dB respectively, while it is failed to enhance noisy Heavy Sine and Doppler signals. CS with DWT of two basis and for single decomposition level have improved the noisy Blocks, Bumps, Heavy Sine and Doppler signals by 5dB, 4dB, 3dB, and 3dB respectively.

In remote sensing image fusion, pansharpening is a type of remote sensing image fusion method that aims to fuse panchromatic (PAN) images and multispectral (MS) images to produce high-resolution multispectral (HRMS) images. Deep learning based pansharpening technology offers a series of advanced unsupervised algorithms. However, there are several challenges: (1) The existing unsupervised pansharpening methods only consider the fusion of single-scale features; (2) for the fusion of MS and PAN image feature branches, the existing pansharpening methods are implemented directly by concatenation and summation, without paying attention to critical features or suppressing redundant features; (3) the semantic gap in the long skip connections of the network architecture will create unexpected results. In this paper, we design a multiscale unsupervised architecture based on generative adversarial networks (GANs) for remote sensing image pansharpening (Mun-GAN), which consists of a generator and two discriminators. The generator includes a multi-scale feature extractor (MFE), a self-adaptation weighted fusion (SWF) module, and a nest feature aggregation (NFA) module. First, the MFE is utilized to extract multiscale feature information from the input images and to then pass this information to the SWF module for adaptive weight fusion. Then, multiscale features are reconstructed by the NFA module to obtain HRMS images. The two discriminators are spectral and spatial discriminators used against the generator. Moreover, we design a hybrid loss function to aggregate the multiscale spectral and spatial feature information. Compared with other state-of-the-art methods using QuickBird, GaoFen-2 and WorldView-3 images, which demonstrate that the Mun-GAN yields better fusion results.

Pansharpening methods generally inject the missing spatial details from a high spatial resolution panchromatic (PAN) image into the corresponding co-registered low spatial resolution multispectral (MS) images while preserving the spectral information. However, most of methods extract the details only from PAN image, which may lead to distortions in the sharpened results. Motivated by this, we present a novel two-stage guided filtering based pansharpening method. In the preliminary stage, an injection model based on multi-channel guidance filtering is designed to keep the spectral fidelity of MS imagery. Then a single channel guidance filtering based injection model is proposed to enhance the details in the second stage. The proposed method is tested and verified by GeoEye-1 satellite images. Qualitative and quantitative analyses demonstrate the superiority of the proposed method compared with some state-of-the-art guided filtering based pansharpening methods.