Multiscale Fully Connected Tensor Network Decomposition With NSST Guidance for Thick Cloud Removal

ABSTRACTIn order to improve the accuracy of change detection and reduce the running time, a change detection method based on equal weight image fusion and adaptive threshold in the NSST domain is proposed. First, the logarithmic transformation is used to transform images and the mean filter is applied to the transformed images. The log-ratio method and the mean ratio method are adopted to generate two kinds of difference images. The final difference image is achieved by equal weight image fusion method. Then, an adaptive threshold denoising method based on non-subsampled shearlet transform (NSST) is used to achieve noise reduction. Finally, the k-means clustering algorithm is utilized to get the change detection results. The experimental results show that the proposed algorithm has better change detection performance than the reference algorithms in visual effect and objective parameters.

We present a novel fusion method based on improved pulse-coupled neural networks (PCNN) model in non-subsampled shearlet transform (NSST) domain for wholebody PET/CT images. Firstly, source images are decomposed using NSST into one low-pass sub-band and several highpass sub-bands. Then, an improved PCNN is used in highpass sub-bands where energy of edge and average gradient are as external input and linking strength respectively. Maximum region energy (MRE) and maximum selection (MS) rules are as fusion rules for high-and low-pass sub-bands respectively. Finally, inverse NSST is adopted to produce fused result. Experiments show the superiority of our method.

Presented is a novel technique to fuse multi‐sensor images using non‐subsampled shearlet transform (NSST) domain I 2 CM. First, multi‐scale and multi‐directional decompositions of source images can be performed by NSST. The classic ICM is improved to be I 2 CM which is then exploited to capture and extract the characteristics of each of the sub‐band images. Finally, the fused image is reconstructed from the fused sub‐images by adopting the inverse NSST. Experimental results demonstrate that the proposed image fusion scheme is very competitive compared with other conventional ones in both visual effect and objective evaluation criteria.

A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain

Combined with the shared similarity among multiple source images and depth of field in a camera, a new image fusion algorithm based on nonsubsampled shearlet transform (NSST) domain is proposed. First, NSST is utilized for decomposition of the source images, the low frequency coefficients is fused by weight votes in the structure-driven regions by using shared similarity and depth of field (SSSID), and then apply larger sum-modified-Laplacian (SML) with depth of field to the high frequency coefficients, finally the fusion image is gained after we do the inverse NSST to the fused coefficients. The algorithm can not only preserve the information of the source images well, but also suppress pixel distortion due to nonlinear operations in transform domain. Experimental results demonstrate that the proposed method outperforms the state-of-the-art transform domain fusion methods on image quality and objective fusion criteria.

There are many artificial parameters when fuse infrared and visible images, to overcome the lack of detail in the fusion image because of the artifacts, a novel fusion algorithm for infrared and visible images that is based on different constraints in non-subsampled shearlet transform (NSST) domain is proposed. There are high bands and low bands of images that are decomposed by the NSST. After analyzing the characters of the bands, fusing the high level bands by the gradient constraint, the fused image can obtain more details; fusing the low bands by the constraint of saliency in the images, the targets are more salient. Before the inverse NSST, the Nash equilibrium is used to update the coefficient. The fused images and the quantitative results demonstrate that our method is more effective in reserving details and highlighting the targets when compared with other state-of-the-art methods.

Multimodal Medical fusion imaging is a salient feature not only in image-guided medical diagnosis but also treatment and various computer guided medical procedures. This paper proposes an efficient medical fusion imaging system that is evolved from the concept of Non-Subsampled Shearlet Transform (NSST) and the Gray Wolf Optimization (GWO) technique. Optimum decomposition level is determined by GWO technique and the optimum gain parameters. Finally, one more procedure is added to the entire procedure known as denoising and enhancement process is carried out to improve its visual quality and improve detailing. Work performance of these traditional fusion techniques are hampered greatly in cases when the images are corrupted with noise. Hence there is a need to develop a fusion technique that is capable of preserving precise information even when images are corrupted. It is also challenging to achieve suppressed noise and enhanced textural simultaneously. This proposed work is a study to compare traditional spatial, transform, filter and neural network domain fusion techniques with optimized NSST fusion imaging.

Cloud detection in remote sensing images is a challenging but significant task. Due to the variety and complexity of underlying surfaces, most of the current cloud detection methods have difficulty in detecting thin cloud regions. In fact, it is quite meaningful to distinguish thin clouds from thick clouds, especially in cloud removal and target detection tasks. Therefore, we propose a method based on multiscale features-convolutional neural network (MF-CNN) to detect thin cloud, thick cloud, and noncloud pixels of remote sensing images simultaneously. Landsat 8 satellite imagery with various levels of cloud coverage is used to demonstrate the effectiveness of our proposed MF-CNN model. We first stack visible, near-infrared, short-wave, cirrus, and thermal infrared bands of Landsat 8 imagery to obtain the combined spectral information. The MF-CNN model is then used to learn the multiscale global features of input images. The high-level semantic information obtained in the process of feature learning is integrated with low-level spatial information to classify the imagery into thick, thin and noncloud regions. The performance of our proposed model is compared to that of various commonly used cloud detection methods in both qualitative and quantitative aspects. Compared to other cloud detection methods, the experimental results show that our proposed method has a better performance not only in thick and thin clouds but also in the entire cloud regions.

In this work, a novel image enhancement algorithm using NSST and SVD is proposed to improve the definition of the acquired brain images. The input brain image is computed by CLAHE, then the processed brain image and input brain image are decomposed into low- and high-frequency components by NSST, the singular value matrix of the low-frequency component is estimated. The final enhancement image is obtained by inverse NSST. Results of this experiment demonstrate that the proposed technique has good performance in terms of brain image enhancement when compared to other methods.

Synthetic aperture radar (SAR)-optical data fusion cloud removal is a highly promising cloud removal technology that has attracted considerable attention. In this field, primary researches are based on deep learning, which can be divided into two categories: convolutional neural network (CNN)-based and Transformer-based. In the cases of extensive cloud coverage, CNN-based methods, with their local spatial awareness, effectively capture local structural information in SAR data, preserving clear contours of cloud-removed land covers. However, these methods struggle to capture global land cover information in optical images, often resulting in notable color discrepancies between recovered and cloud-free regions. Conversely, Transformer-based methods, with their global modeling capability and inherent low-pass filtering properties, excel at capturing long-range spatial correlations in optical images, thereby maintaining color consistency across cloud-removal outputs. However, these methods are less effective at capturing the fine structural details in SAR data, which can lead to blurred local contours in the final cloud-removed images. In this context, a novel framework called heterogeneous parallel network for cloud removal (HPN-CR) is proposed to achieve high-quality cloud removal under extensive cloud coverage conditions. HPN-CR employs the proposed heterogeneous encoder with its SAR-optical input approach to effectively extract and fuse the local structural information in cloudy areas from SAR images, with the spectral information about land covers in cloud-free areas from the whole optical images. In particular, it uses a ResNet network with local spatial awareness to extract SAR features. It also uses the proposed Decloudformer, which globally models multiscale spatial correlations, to extract optical features. The output features are fused by the heterogeneous encoder and then reconstructed to cloud-removal images through a pixelshuffle-based decoder. Comprehensive experiments were conducted and the experimental results demonstrated the effectiveness and superiority of the proposed method. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/G-pz/HPN-CR</uri>.

The presence of thick clouds in single optical images shows the contamination of interesting objects. Besides, the difficulty of thick cloud removal is mainly the restoration of the weak boundary information from cloud-contaminated areas. Recently, many deep-learning-based frameworks were applied for cloud removal by obtain the related semantic information from the weak boundary information. However, the large-size cloud-contaminated areas lead to the artificial textures in the resulting images. Thus, obtaining the optimal semantic information from finite boundary information is the key to solve this problem. In this work, we design a deep-learning framework for cloud removal, especially large-size clouds removal (i.e., more than 30% coverage of the whole image). First, we design a cloud location model (CLM) which adopted the fully convolutional network to locate the cloud. Second, desired by theory of the coarse-to-fine restoration, we build a dense-attention network (termed as DANet) for restoring cloud contaminated areas. In the DANet, we design a dense block into the coarse network for training the features of restoring directions of each pixel from the weak boundary information. Furthermore, a contextual attention module is built into refinement network for restoring contaminated areas relying on the semantic relationship between the background and foreground information. Compared with the state-of-the-art methods, the proposed DANet achieved greater removal performances and reconstruct more natural image textures.

Cloud removal is a relevant topic in Remote Sensing, fostering medium- and high-resolution optical image usability for Earth monitoring and study. Recent applications of deep generative models and sequence-to-sequence-based models have proved their capability to advance the field significantly. Nevertheless, there are still some gaps: the amount of cloud coverage, the landscape temporal changes, and the density and thickness of clouds need further investigation. We fill some of these gaps in this work by introducing an innovative deep model. The proposed model is multi-modal, relying on both spatial and temporal sources of information to restore the whole optical scene of interest. We use the outcomes of both temporal-sequence blending and direct translation from Synthetic Aperture Radar (SAR) to optical images to obtain a pixel-wise restoration of the whole scene. The reconstructed images preserve scene details without resorting to a considerable portion of a clean image. Our approach’s advantage is demonstrated across various atmospheric conditions tested on different datasets. Quantitative and qualitative results prove that the proposed method obtains cloud-free images coping with landscape changes.

Satellite remote sensing technology is widely applied for real-time drawing maps, forecasting weather, and predicting natural disasters. However, many remote sensing satellite images contain cloud noise, resulting in difficulties in practical applications. Existing cloud removal methods are limited when an image contains lots of clouds or thick clouds. Besides, most methods need a cloudless image as a reference. Our study proposes an advanced algorithm to remove cloud noise (especially thick clouds) in remote sensing images, including a cloud segmentation model, prior knowledge refinement, and Dempster-Shafer (D-S) evidence theory. Firstly, we trained a Cloud-net model to segment cloud regions. RGB and Near Infrared (NIR) images are fed into the Cloud-net model. Then it outputs coarse segmentation images. Secondly, we introduced the prior knowledge to refine and recover segmentation results. Finally, we designed cloud removal rules based on D-S evidence theory to fuse muti-temporal remote sensing images. Our method achieves a surprising performance on GaoFen-4 (GF-4) satellite images, reducing the average percentage of cloud noise from 30\(\%\)–40\(\%\) to 2\(\%\)–8\(\%\) per image.KeywordsSatellite remote sensing imageD-S evidence theoryCloud segmentation modelMulti-temporal denoising

Cloud cover poses a major challenge to the usability of optical remote sensing images, particularly under thick clouds. This article proposes SAR-DiffCloud, a high-fidelity cloud removal framework that integrates multiband synthetic aperture radar (SAR) guidance with a conditional diffusion model and an adaptive style transfer mechanism. By leveraging the cloud-penetrating characteristics of Ka, X, and C-band SAR data, the model extracts deep scattering features via principal component analysis and fuses them with optical features through a bidirectional fusion architecture. An adaptive style transfer module based on optimal transport theory is introduced to ensure spectral consistency between generated and cloud-free regions. Extensive experiments on the AIR-MD-SAR-Map and OGSOD datasets demonstrate that SAR-DiffCloud significantly outperforms existing methods across structural, spectral, and perceptual quality metrics, particularly under extreme cloud coverage. The results confirm the effectiveness of multimodal guidance and style-aware generation for robust remote sensing image restoration.

In observations of Earth, the existence of clouds affects the quality and usability of optical remote sensing images in practical applications. Many cloud removal methods have been proposed to solve this issue. Among these methods, synthetic aperture radar (SAR)-based methods have more potential than others because SAR imaging is hardly affected by clouds, and can reflect ground information differences and changes. While SAR images used as auxiliary information for cloud removal may be blurred and noisy, the similar non-local information of spectral and electromagnetic features cannot be effectively utilized by traditional cloud removal methods. To overcome these weaknesses, we propose a novel cloud removal method using SAR-optical data fusion and a graph-based feature aggregation network (G-FAN). First, cloudy optical images and contemporary SAR images are concatenated and transformed into hyper-feature maps by pre-convolution. Second, the hyper-feature maps are inputted into the G-FAN to reconstruct the missing data of the cloud-covered area by aggregating the electromagnetic backscattering information of the SAR image, and the spectral information of neighborhood and non-neighborhood pixels in the optical image. Finally, post-convolution and a long skip connection are adopted to reconstruct the final predicted cloud-free images. Both the qualitative and quantitative experimental results from the simulated data and real data experiments show that our proposed method outperforms traditional deep learning methods for cloud removal.