A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain

The rolling guidance filtering (RGF) has a good characteristic which can smooth texture and preserve the edges, and non-subsampled shearlet transform (NSST) has the features of translation invariance and direction selection based on which a new infrared and visible image fusion method is proposed. Firstly, the rolling guidance filter is used to decompose infrared and visible images into the base and detail layers. Then, the NSST is utilized on the base layer to get the high-frequency coefficients and low-frequency coefficients. The fusion of low-frequency coefficients uses visual saliency map as a fusion rule, and the coefficients of the high-frequency subbands use gradient domain guided filtering (GDGF) and improved Laplacian sum to fuse coefficients. Finally, the fusion of the detail layers combines phase congruency and gradient domain guided filtering as the fusion rule. As a result, the proposed method can not only extract the infrared targets, but also fully preserves the background information of the visible images. Experimental results indicate that our method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments.

ABSTRACTIn order to improve the accuracy of change detection and reduce the running time, a change detection method based on equal weight image fusion and adaptive threshold in the NSST domain is proposed. First, the logarithmic transformation is used to transform images and the mean filter is applied to the transformed images. The log-ratio method and the mean ratio method are adopted to generate two kinds of difference images. The final difference image is achieved by equal weight image fusion method. Then, an adaptive threshold denoising method based on non-subsampled shearlet transform (NSST) is used to achieve noise reduction. Finally, the k-means clustering algorithm is utilized to get the change detection results. The experimental results show that the proposed algorithm has better change detection performance than the reference algorithms in visual effect and objective parameters.

We present a novel fusion method based on improved pulse-coupled neural networks (PCNN) model in non-subsampled shearlet transform (NSST) domain for wholebody PET/CT images. Firstly, source images are decomposed using NSST into one low-pass sub-band and several highpass sub-bands. Then, an improved PCNN is used in highpass sub-bands where energy of edge and average gradient are as external input and linking strength respectively. Maximum region energy (MRE) and maximum selection (MS) rules are as fusion rules for high-and low-pass sub-bands respectively. Finally, inverse NSST is adopted to produce fused result. Experiments show the superiority of our method.

Presented is a novel technique to fuse multi‐sensor images using non‐subsampled shearlet transform (NSST) domain I 2 CM. First, multi‐scale and multi‐directional decompositions of source images can be performed by NSST. The classic ICM is improved to be I 2 CM which is then exploited to capture and extract the characteristics of each of the sub‐band images. Finally, the fused image is reconstructed from the fused sub‐images by adopting the inverse NSST. Experimental results demonstrate that the proposed image fusion scheme is very competitive compared with other conventional ones in both visual effect and objective evaluation criteria.

There are many artificial parameters when fuse infrared and visible images, to overcome the lack of detail in the fusion image because of the artifacts, a novel fusion algorithm for infrared and visible images that is based on different constraints in non-subsampled shearlet transform (NSST) domain is proposed. There are high bands and low bands of images that are decomposed by the NSST. After analyzing the characters of the bands, fusing the high level bands by the gradient constraint, the fused image can obtain more details; fusing the low bands by the constraint of saliency in the images, the targets are more salient. Before the inverse NSST, the Nash equilibrium is used to update the coefficient. The fused images and the quantitative results demonstrate that our method is more effective in reserving details and highlighting the targets when compared with other state-of-the-art methods.

Thick cloud coverage presents a significant challenge for the analysis and application of multi-temporal remote sensing imagery. Although existing cloud removal methods attempt to exploit spatio-spectral-temporal correlations, they often fail to adequately capture cross-scale complementary features due to single-scale reconstruction limitations, resulting in texture degradation and color distortion effects. To address this issue, we propose a multi-scale fully connected tensor network decomposition method guided by the non-subsampled shearlet transform (NSST). The proposed approach first applies NSST in the spatial domain, followed by parameter-adaptive pulse-coupled neural network (PCNN) fusion of each decomposed subband along the temporal dimension to construct a high-quality guidance image. This guidance image is subsequently employed to impose regularization constraints on both high- and low-frequency components of the tensor network decomposition in the NSST domain, enabling a comprehensive representation of multi-scale correlations. To efficiently solve the resulting optimization model, we develop a proximal alternating minimization (PAM)-based algorithm. Extensive experiments on both simulated and real satellite datasets demonstrate that the proposed method consistently outperforms existing approaches, achieving average improvements of 1.2–3.5 dB in PSNR and 0.5–2.0% in SSIM over the best baseline across various cloud coverage scenarios, while maintaining superior detail preservation and color consistency.

Combined with the shared similarity among multiple source images and depth of field in a camera, a new image fusion algorithm based on nonsubsampled shearlet transform (NSST) domain is proposed. First, NSST is utilized for decomposition of the source images, the low frequency coefficients is fused by weight votes in the structure-driven regions by using shared similarity and depth of field (SSSID), and then apply larger sum-modified-Laplacian (SML) with depth of field to the high frequency coefficients, finally the fusion image is gained after we do the inverse NSST to the fused coefficients. The algorithm can not only preserve the information of the source images well, but also suppress pixel distortion due to nonlinear operations in transform domain. Experimental results demonstrate that the proposed method outperforms the state-of-the-art transform domain fusion methods on image quality and objective fusion criteria.

For multi-sensor fusion of infrared and visible images, it is difficult to retain the thermal radiation information of the infrared image and the texture information of the visible image in the fused image. To overcome this problem, a novel infrared and visible image fusion method based on a modified side window filter (MSWF) and an intensity transformation is proposed. First, an MSWF with effective edge-preservation ability is developed by adding four additional kernels to better decompose the source images to obtain the base and detail layers. Furthermore, to extract the edge information of the source images, we propose to further decompose the base layers to obtain the low-frequency layers and high-frequency layers (edge information) through the non-subsampled shearlet transform (NSST). Then, an S-shape intensity transformation function (ITF) is proposed to enhance the saliency information and suppress the non-saliency information in the infrared image. In the fusion process, considering the characteristics of the decomposed components, different fusion rules are designed to obtain the fused detail layer and low- and high-frequency layers. Finally, these fused components are reconstructed to obtain the final fusion image. It is experimentally demonstrated that the proposed method is superior to state-of-the-art fusion methods both in terms of subjective evaluation and objective metrics.

Infrared and visible image fusion based on semi-global weighted least squares and guided edge-aware filters

In order to more accurately realize fusion of remote sensing images, we propose a novel remote sensing images fusion algorithm combining extended non-subsampled shearlet transform (NSST) and modified pulse-coupled neural network (PCNN). Firstly, it makes histogram matching and intensity smoothing and filtering treatment on intensity component and full-color image of multi-spectral image. Secondly, such intensity component and full-color image are decomposed by extended NSST to get corresponding high-frequency and low-frequency coefficients. For low-frequency coefficients, fusion is made by sparse representation; for high-frequency coefficients, a modified pulse-coupled neural network (PCNN) strategy is put forward to process. Finally, the processed result is drawn by inverse transformation of the extended NSST and intensity-hue-saturation inverse transformation. The experimental results show that the proposed algorithm reserves as much spectral information as possible and improve spatial resolution; its visual effects and objective indexes are better than other classical fusion algorithms.

In general, since the noise deteriorates the interpolation performance in digital images, it is effective to employ denoising prior to the interpolation. In this paper, we propose a novel interpolation framework in which denoising and image sharpening methods are embedded. In the proposed framework, the image is first decomposed using the bilateral filter into the detail and base layers which represent the small and large scale features, respectively. The detail layer is adaptively smoothed to suppress the noise before interpolation and an edge-preserving interpolation method is applied to both layers. Finally, the high resolution image is obtained by combining the base and detail layers. Experimental results show that the proposed algorithm outperforms the conventional methods while suppressing blurring and jagging.

Existing image fusion algorithms have difficulty in effectively preserving valuable target features in infrared and visible images, which easily introduces blurry edges and unremarkable notable targets during their fusion process. We propose the MGFuse algorithm as a solution to this problem, which is a novel fusion algorithm that utilizes multiscale decomposition optimization and gradient-weighted local energy. Initially, non-subsampled shearlet transform (NSST) is applied to partition both the infrared and visible images into several high-frequencies and low-frequencies components. Subsequently, the acquired low frequencies continue to be decomposed via proposed optimization function to get base layers and texture layers, which can optimize the quality of image edges and preserve fine-grained details, respectively. In addition, we have formulated an intrinsic attribute-based energy (IAE) fusion scheme to merge the two base layers. The texture layers and high-frequencies are extracted by gradient-weighted local energy (GE) operator based on structure tensor, which are employed to construct the fusion strategy for these parts. At last, the acquired texture and base parts are linearly combined to get the integrated low-frequency layer on which the final image is acquired using inverse NSST. Numerous experimental observations demonstrate that our MGFuse algorithm achieve superior fusion capability than the reference nine advanced algorithms in both qualitative and quantitative assessment, and robustness to noisy images with different noise levels.

To overcome the artifact phenomenon caused by the incomplete registration of the source images, a new multi-focus image fusion approach is proposed based on sparse representation and non-subsampled shearlet transform (NSST). Firstly, the source images are decomposed to low- and high-frequency coefficients by NSST. The sparse representation is then adopted to fuse the low-frequency coefficients. For the high-frequency coefficients, a maximum sum-modified-Laplacian (SML) rule is put forward to merge them. Finally, the resultant image is obtained by the inverse NSST on the fused coefficients. Experimental results indicate that the proposed method can achieve satisfied effect compared with various existing image fusion methods.

Many fusion methods have been proposed to fuse medical image, but these approaches alway lead to fusion image distortion or image information loss. To overcome the above disadvantages, combined with nonsubsampled shearlet transform (NSST) and spiking cortical model (SCM), a new medical image fusion is proposed. First, NSST is utilized for decomposition of the source images. Secondly, the low and high frequency coefficients of NSST are all fused by fired map of improved SCM (ISCM), which motivated by larger summodified-Laplacian (SML). Finally, the fusion image is gained by inverse NSST. The algorithm can both preserve the information of the source images well and suppress pixel distortion. Experimental results demonstrate that the proposed method outperforms the state-of-the-art fusion methods.

In multi-scale geometric analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and non-subsampled shearlet transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform (WT) is used to decompose high-frequency sub-bands to obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.