Multi-focus Image Fusion Based on Non-subsampled Shearlet Transform and Sparse Representation

Multi-focus fusion technique is used to combine images obtained from single or different cameras with different focal distance, etc. In the proposed method, the non-subsampled shearlet transform (NSST) is employed to decompose the input image data into the low-frequency and high-frequency bands. These low-frequency and high-frequency bands are combined using sparse representation (SR) and modified difference based fusion rules, respectively. Then, inverse NSST is employed to get the fused image. Both qualitative and quantitative results confirm that the proposed approach yields a better performance as compared to state-of-the-art fusion schemes.

Combined with the shared similarity among multiple source images and depth of field in a camera, a new image fusion algorithm based on nonsubsampled shearlet transform (NSST) domain is proposed. First, NSST is utilized for decomposition of the source images, the low frequency coefficients is fused by weight votes in the structure-driven regions by using shared similarity and depth of field (SSSID), and then apply larger sum-modified-Laplacian (SML) with depth of field to the high frequency coefficients, finally the fusion image is gained after we do the inverse NSST to the fused coefficients. The algorithm can not only preserve the information of the source images well, but also suppress pixel distortion due to nonlinear operations in transform domain. Experimental results demonstrate that the proposed method outperforms the state-of-the-art transform domain fusion methods on image quality and objective fusion criteria.

Many fusion methods have been proposed to fuse medical image, but these approaches alway lead to fusion image distortion or image information loss. To overcome the above disadvantages, combined with nonsubsampled shearlet transform (NSST) and spiking cortical model (SCM), a new medical image fusion is proposed. First, NSST is utilized for decomposition of the source images. Secondly, the low and high frequency coefficients of NSST are all fused by fired map of improved SCM (ISCM), which motivated by larger summodified-Laplacian (SML). Finally, the fusion image is gained by inverse NSST. The algorithm can both preserve the information of the source images well and suppress pixel distortion. Experimental results demonstrate that the proposed method outperforms the state-of-the-art fusion methods.

Multi-scale geometric analysis is a popular tool that is widely used in the field of multi-focus image fusion. It plays a large role in extracting the features from input images. In this paper, a novel multi-focus image fusion method based on a non-fixed-base dictionary and multi-measure optimization is presented in the non-subsampled shearlet transform (NSST) domain. The proposed approach contains the following four steps. The input images are first decomposed by NSST into low- and high-frequency coefficients. Second, a sparse representation (SR)-based framework is proposed and applied to merge the low-frequency coefficients of the input images. In this framework, a non-fixed-base dictionary iteratively trained by the previous dictionary is constructed, which represents the complex details of the source images. Third, a type-2 fuzzy logic scheme is introduced for the high-frequency coefficient fusion, which can effectively select the high-quality coefficients from the source images so that it can achieve better performance. Finally, an inverse NSST operation is conducted in the merged low- and high-pass subbands and thus obtains the initial fused image. A multi-measure optimization method is then employed to optimize the initial fused image, and thus, the final fused result is achieved. In this method, three different measures, namely, pixel difference, visual saliency, and similarity, are designed to select the focus regions more completely. The experimental results demonstrate that our proposed approach yields a better effect than other methods in both the visual quality and the objective assessment.

In multi-scale geometric analysis (MGA)-based fusion methods for infrared and visible images, adopting the same representation for the two types of images will result in the non-obvious thermal radiation target in the fused image, which can hardly be distinguished from the background. To solve the problem, a novel fusion algorithm based on nonlinear enhancement and non-subsampled shearlet transform (NSST) decomposition is proposed. Firstly, NSST is used to decompose the two source images into low- and high-frequency sub-bands. Then, the wavelet transform (WT) is used to decompose high-frequency sub-bands to obtain approximate sub-bands and directional detail sub-bands. The “average” fusion rule is performed for fusion for approximate sub-bands. And the “max-absolute” fusion rule is performed for fusion for directional detail sub-bands. The inverse WT is used to reconstruct the high-frequency sub-bands. To highlight the thermal radiation target, we construct a non-linear transform function to determine the fusion weight of low-frequency sub-bands, and whose parameters can be further adjusted to meet different fusion requirements. Finally, the inverse NSST is used to reconstruct the fused image. The experimental results show that the proposed method can simultaneously enhance the thermal target in infrared images and preserve the texture details in visible images, and which is competitive with or even superior to the state-of-the-art fusion methods in terms of both visual and quantitative evaluations.

To preserve more useful information in visible and infrared images and improve the quality of the fused image, a method based on the nonsubsampled shearlet transform (NSST) and fuzzy C-means clustering is proposed. First, the source images are decomposed by NSST so as to get their own low- and high-frequency subbands. Second, the low-frequency subbands are divided into the infrared target part and the background part by fuzzy C-means clustering while different fusion rules are applied to the infrared target part and background part, respectively. Then, a choose-max fusion rule based on the sum-modified Laplacian of source images and local energy of coefficient is proposed to integrate the high-frequency subbands. Finally, the fused image is obtained by inverse NSST. The comparison experiment with the other three state-of-the-art fusion methods shows that the proposed method has good subjective visual effects and superior objective evaluations.

We present a novel fusion method based on improved pulse-coupled neural networks (PCNN) model in non-subsampled shearlet transform (NSST) domain for wholebody PET/CT images. Firstly, source images are decomposed using NSST into one low-pass sub-band and several highpass sub-bands. Then, an improved PCNN is used in highpass sub-bands where energy of edge and average gradient are as external input and linking strength respectively. Maximum region energy (MRE) and maximum selection (MS) rules are as fusion rules for high-and low-pass sub-bands respectively. Finally, inverse NSST is adopted to produce fused result. Experiments show the superiority of our method.

Technique for image fusion between gray-scale visual light and infrared images based on NSST and improved RF

There are many artificial parameters when fuse infrared and visible images, to overcome the lack of detail in the fusion image because of the artifacts, a novel fusion algorithm for infrared and visible images that is based on different constraints in non-subsampled shearlet transform (NSST) domain is proposed. There are high bands and low bands of images that are decomposed by the NSST. After analyzing the characters of the bands, fusing the high level bands by the gradient constraint, the fused image can obtain more details; fusing the low bands by the constraint of saliency in the images, the targets are more salient. Before the inverse NSST, the Nash equilibrium is used to update the coefficient. The fused images and the quantitative results demonstrate that our method is more effective in reserving details and highlighting the targets when compared with other state-of-the-art methods.

Multi-focus image fusion method can fuse images taken from the same view point with different focal settings, and obtain an image with every object in focus. In this paper, a novel multi-focus image fusion via non-subsampled shearlet transform (NSST) with non-fixed base dictionary learning is presented. First, low frequency coefficients and high frequency coefficients are obtained by NSST. Then, a new strategy, which can enhance the information of spatial detail for the fused image is proposed to process two different coefficients. The low frequency coefficients are fused via a non-fixed base dictionary, which makes the K-SVD algorithm more efficient, and the high frequency coefficients are fused with spatial frequency, which is effective in the fused image. Experiment results demonstrate that the results of proposed method obtain more spatial details and have almost zero residuals compared with several conventional methods in terms of both visual quality and objective measurements.

Aiming at the problems of information richness and insufficient edge definition of multi-focus fusion images, this paper proposes a multi-focus image fusion algorithm based on non-subsampled shear wave transform (NSST) and consistent optimization strategy. Firstly, the source image is decomposed by NSST transform in multi-scale and multi-direction to obtain high and low frequency subband coefficients. Secondly, Locally improved Laplacian energy and spatial consistency optimization strategy are used for low-frequency subband coefficients to construct low-frequency subband coefficient fusion weights; for high-frequency subband coefficients, spatial frequency and energy based on correlation coefficients are used, and phase consistency optimization is used. The strategy is to construct high-frequency sub-band coefficient fusion weights; finally, the final fusion image is obtained through NSST inverse transformation. The experimental results show that the proposed algorithm can improve the information retention and visual clarity of the fused image, whether in subjective vision or in objective evaluation.

Multi-focus image fusion aims to integrate clear segments from different partially focused images, creating an ‘all-in-focus’ composite. Due to the lack of ground-truth for multi-focus image fusion, supervised deep learning methods are deemed inappropriate for this task. In this paper, we present an unsupervised approach for multi-focus image fusion, named Fusion2Void. Fusion2Void ingeniously tackles the challenge of missing ground-truth by framing image inpainting as an auxiliary task. Specifically, Fusion2Void utilizes a fusion network to merge focused regions from multiple source images. Following the fusion process, image patches in the source images are randomly dropped to construct an additional image inpainting task. Subsequently, an image inpainting network uses the fused image as a guide to restore the missing content in the source images. The missing content in the source images includes both focused and defocused regions. Restoring focused image patches is significantly more challenging than restoring their defocused counterparts due to their inclusion of more high-frequency details. If the focused image patches are effectively restored, the repair of the defocused image patches becomes notably easier. Therefore, the image inpainting network implicitly compels the fused image to incorporate all focused content from the source images, as these can be utilized to restore the missing focused regions in the source images perfectly. Based on image inpainting, the fusion network generates ‘all-in-focus’ images in an unsupervised manner. Experiments on several synthetic and real-world datasets highlight Fusion2Void’s state-of-the-art performance relative to other methods.

Traditional and deep learning-based fusion methods generated the intermediate decision map to obtain the fusion image through a series of post-processing procedures. However, the fusion results generated by these methods are easy to lose some source image details or results in artifacts. Inspired by the image reconstruction techniques based on deep learning, we propose a multi-focus image fusion network framework without any post-processing to solve these problems in the end-to-end and supervised learning way. To sufficiently train the fusion model, we have generated a large-scale multi-focus image dataset with ground-truth fusion images. What's more, to obtain a more informative fusion image, we further designed a novel fusion strategy based on unity fusion attention, which is composed of a channel attention module and a spatial attention module. Specifically, the proposed fusion approach mainly comprises three key components: feature extraction, feature fusion and image reconstruction. We firstly utilize seven convolutional blocks to extract the image features from source images. Then, the extracted convolutional features are fused by the proposed fusion strategy in the feature fusion layer. Finally, the fused image features are reconstructed by four convolutional blocks. Experimental results demonstrate that the proposed approach for multi-focus image fusion achieves remarkable fusion performance compared to 19 state-of-the-art fusion methods.

In order to more accurately realize fusion of remote sensing images, we propose a novel remote sensing images fusion algorithm combining extended non-subsampled shearlet transform (NSST) and modified pulse-coupled neural network (PCNN). Firstly, it makes histogram matching and intensity smoothing and filtering treatment on intensity component and full-color image of multi-spectral image. Secondly, such intensity component and full-color image are decomposed by extended NSST to get corresponding high-frequency and low-frequency coefficients. For low-frequency coefficients, fusion is made by sparse representation; for high-frequency coefficients, a modified pulse-coupled neural network (PCNN) strategy is put forward to process. Finally, the processed result is drawn by inverse transformation of the extended NSST and intensity-hue-saturation inverse transformation. The experimental results show that the proposed algorithm reserves as much spectral information as possible and improve spatial resolution; its visual effects and objective indexes are better than other classical fusion algorithms.

Presented is a novel technique to fuse multi‐sensor images using non‐subsampled shearlet transform (NSST) domain I 2 CM. First, multi‐scale and multi‐directional decompositions of source images can be performed by NSST. The classic ICM is improved to be I 2 CM which is then exploited to capture and extract the characteristics of each of the sub‐band images. Finally, the fused image is reconstructed from the fused sub‐images by adopting the inverse NSST. Experimental results demonstrate that the proposed image fusion scheme is very competitive compared with other conventional ones in both visual effect and objective evaluation criteria.