MGFuse: An Infrared and Visible Image Fusion Algorithm Based on Multiscale Decomposition Optimization and Gradient-Weighted Local Energy

In multi-scale geometric analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and non-subsampled shearlet transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform (WT) is used to decompose high-frequency sub-bands to obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

There are many artificial parameters when fuse infrared and visible images, to overcome the lack of detail in the fusion image because of the artifacts, a novel fusion algorithm for infrared and visible images that is based on different constraints in non-subsampled shearlet transform (NSST) domain is proposed. There are high bands and low bands of images that are decomposed by the NSST. After analyzing the characters of the bands, fusing the high level bands by the gradient constraint, the fused image can obtain more details; fusing the low bands by the constraint of saliency in the images, the targets are more salient. Before the inverse NSST, the Nash equilibrium is used to update the coefficient. The fused images and the quantitative results demonstrate that our method is more effective in reserving details and highlighting the targets when compared with other state-of-the-art methods.

For multi-sensor fusion of infrared and visible images, it is difficult to retain the thermal radiation information of the infrared image and the texture information of the visible image in the fused image. To overcome this problem, a novel infrared and visible image fusion method based on a modified side window filter (MSWF) and an intensity transformation is proposed. First, an MSWF with effective edge-preservation ability is developed by adding four additional kernels to better decompose the source images to obtain the base and detail layers. Furthermore, to extract the edge information of the source images, we propose to further decompose the base layers to obtain the low-frequency layers and high-frequency layers (edge information) through the non-subsampled shearlet transform (NSST). Then, an S-shape intensity transformation function (ITF) is proposed to enhance the saliency information and suppress the non-saliency information in the infrared image. In the fusion process, considering the characteristics of the decomposed components, different fusion rules are designed to obtain the fused detail layer and low- and high-frequency layers. Finally, these fused components are reconstructed to obtain the final fusion image. It is experimentally demonstrated that the proposed method is superior to state-of-the-art fusion methods both in terms of subjective evaluation and objective metrics.

The purpose of this paper is to propose an algorithm and a novel method for fusion of Passive Millimeter Wave (PMMW) images with their visible images by using Non-subsampled Shearlet Transform (NSST) based on the Spiking Cortical Model (SCM) fusion rule. The parameters of NSST and Improved SCM (ISCM) are selected and optimized based on time consumption and the output of the optimization function like QAB/F. In addition, an effective thresholding method in contourlet transform for PMMW and visible images fusion, which is proposed in our previous paper, is applied to the fusion procedure. Furthermore, NSST is used for analysis of the initial images in different resolutions and directions, followed by using of the ISCM neural network as the fusion rule. Combination of the proposed fusion and thresholding processes resulted in better output images containing visual information close to that of the visible image as well as the hidden object in the millimeter wave image. Moreover, a new evaluation criterion is proposed which does not suffer the shortcomings of the available criteria and improves the previous results for detecting hidden objects by up to 23% with 5% extra time consumption. Finally, the obtained results are evaluated with the available and standard metrics, and all the images exhibited better values than those of the other methods. In fact, the value obtained for the gun image is also acceptable. The results indicate that the proposed fusion method realizes almost all the considered objectives, and the fused image show the objects distinctly in an image with visible details.

The rolling guidance filtering (RGF) has a good characteristic which can smooth texture and preserve the edges, and non-subsampled shearlet transform (NSST) has the features of translation invariance and direction selection based on which a new infrared and visible image fusion method is proposed. Firstly, the rolling guidance filter is used to decompose infrared and visible images into the base and detail layers. Then, the NSST is utilized on the base layer to get the high-frequency coefficients and low-frequency coefficients. The fusion of low-frequency coefficients uses visual saliency map as a fusion rule, and the coefficients of the high-frequency subbands use gradient domain guided filtering (GDGF) and improved Laplacian sum to fuse coefficients. Finally, the fusion of the detail layers combines phase congruency and gradient domain guided filtering as the fusion rule. As a result, the proposed method can not only extract the infrared targets, but also fully preserves the background information of the visible images. Experimental results indicate that our method can achieve a superior performance compared with other fusion methods in both subjective and objective assessments.

We present a novel fusion method based on improved pulse-coupled neural networks (PCNN) model in non-subsampled shearlet transform (NSST) domain for wholebody PET/CT images. Firstly, source images are decomposed using NSST into one low-pass sub-band and several highpass sub-bands. Then, an improved PCNN is used in highpass sub-bands where energy of edge and average gradient are as external input and linking strength respectively. Maximum region energy (MRE) and maximum selection (MS) rules are as fusion rules for high-and low-pass sub-bands respectively. Finally, inverse NSST is adopted to produce fused result. Experiments show the superiority of our method.

Structure tensor and nonsubsampled shearlet transform based algorithm for CT and MRI image fusion

To overcome the artifact phenomenon caused by the incomplete registration of the source images, a new multi-focus image fusion approach is proposed based on sparse representation and non-subsampled shearlet transform (NSST). Firstly, the source images are decomposed to low- and high-frequency coefficients by NSST. The sparse representation is then adopted to fuse the low-frequency coefficients. For the high-frequency coefficients, a maximum sum-modified-Laplacian (SML) rule is put forward to merge them. Finally, the resultant image is obtained by the inverse NSST on the fused coefficients. Experimental results indicate that the proposed method can achieve satisfied effect compared with various existing image fusion methods.

Many fusion methods have been proposed to fuse medical image, but these approaches alway lead to fusion image distortion or image information loss. To overcome the above disadvantages, combined with nonsubsampled shearlet transform (NSST) and spiking cortical model (SCM), a new medical image fusion is proposed. First, NSST is utilized for decomposition of the source images. Secondly, the low and high frequency coefficients of NSST are all fused by fired map of improved SCM (ISCM), which motivated by larger summodified-Laplacian (SML). Finally, the fusion image is gained by inverse NSST. The algorithm can both preserve the information of the source images well and suppress pixel distortion. Experimental results demonstrate that the proposed method outperforms the state-of-the-art fusion methods.

To preserve more useful information in visible and infrared images and improve the quality of the fused image, a method based on the nonsubsampled shearlet transform (NSST) and fuzzy C-means clustering is proposed. First, the source images are decomposed by NSST so as to get their own low- and high-frequency subbands. Second, the low-frequency subbands are divided into the infrared target part and the background part by fuzzy C-means clustering while different fusion rules are applied to the infrared target part and background part, respectively. Then, a choose-max fusion rule based on the sum-modified Laplacian of source images and local energy of coefficient is proposed to integrate the high-frequency subbands. Finally, the fused image is obtained by inverse NSST. The comparison experiment with the other three state-of-the-art fusion methods shows that the proposed method has good subjective visual effects and superior objective evaluations.

The fusion of infrared and visible images may result in low contrast, which is unsuitable for observation by human eyes. Thus, we propose a contrast-enhanced fusion algorithm with nonsubsampled shearlet transform (NSST) frames, in which the NSST is first employed to decompose each of the source images into one low frequency sub-band and a series of high frequency sub-bands. To improve the fusion performance, we designed two measures for fusion of the low frequency and the high frequency: the low frequency is divided into salient and nonsalient regions in accordance with the human visual system to improve the global contrast by targeted fusion and the high frequency requires a local contrast fusion strategy. Finally, the merged sub-bands are constructed according to the selection principles, and the final fused image is produced by applying the inverse NSST on these merged sub-bands. Experimental results demonstrate the effectiveness and superiority of the proposed method over the state-of-the-art fusion methods in terms of both visual effect and objective evaluation results.

Multi-focus fusion technique is used to combine images obtained from single or different cameras with different focal distance, etc. In the proposed method, the non-subsampled shearlet transform (NSST) is employed to decompose the input image data into the low-frequency and high-frequency bands. These low-frequency and high-frequency bands are combined using sparse representation (SR) and modified difference based fusion rules, respectively. Then, inverse NSST is employed to get the fused image. Both qualitative and quantitative results confirm that the proposed approach yields a better performance as compared to state-of-the-art fusion schemes.

A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain

In order to more accurately realize fusion of remote sensing images, we propose a novel remote sensing images fusion algorithm combining extended non-subsampled shearlet transform (NSST) and modified pulse-coupled neural network (PCNN). Firstly, it makes histogram matching and intensity smoothing and filtering treatment on intensity component and full-color image of multi-spectral image. Secondly, such intensity component and full-color image are decomposed by extended NSST to get corresponding high-frequency and low-frequency coefficients. For low-frequency coefficients, fusion is made by sparse representation; for high-frequency coefficients, a modified pulse-coupled neural network (PCNN) strategy is put forward to process. Finally, the processed result is drawn by inverse transformation of the extended NSST and intensity-hue-saturation inverse transformation. The experimental results show that the proposed algorithm reserves as much spectral information as possible and improve spatial resolution; its visual effects and objective indexes are better than other classical fusion algorithms.

Infrared images and visible images can obtain different image information in the same scene, especially in low-light scenes, infrared images can obtain image information that cannot be obtained by visible images. In order to obtain more useful information in the environment such as glimmer, infrared and visible images can be fused. In this paper, an image fusion method based on anisotropic diffusion and fast guided filter is proposed. Firstly, the source images are decomposed into base layers and detail layers by anisotropic dispersion. Secondly, the visible images and the infrared images are passed through the side window Gaussian filter to obtain the saliency map, and then the saliency map is passed through fast guided filter to obtain the fusion weight. Thirdly, the fused base layers and the fused detail layers are reconstructed to obtain the final fusion image. The application of the side window Gaussian filter helps to reduce the artifact information of the fused image. The results of the proposed algorithm are compared with similar algorithms. The fusion results reveal that the proposed method are outstanding in subjective evaluation and objective evaluation, and are better than other algorithms in standard deviation(STD) and entropy(EN), and other quality metrics are close to the optimal comparison algorithm.