Multimodal medical image fusion using L0 gradient smoothing with sparse representation

Multimodality medical image fusion is the process of combining multiple images from single or multiple modalities of imaging. Medical image fusion methods are adopted to increase the quality of medical images by attaining the salient features in the fusion results. Hence, they raise the clinical applicability of medical images for appraisal and diagnosis problems. This purpose is achieved by capturing the complementary information presented in two or more images of different modalities in the fusion result. Medical image fusion is generally concerned with Magnetic Resonance Imaging (MRI), Magnetic Resonance Angiogram (MRA), Positron Emission Tomography (PET), Structural Positron Emission Tomography (SPET), Computerized Tomography (CT), and Single-Photon Emission Computed Tomography (SPECT) modalities. Each modality has its merits and drawbacks. This induces new fusion methods for merging information from multiple imaging modalities. Researchers have presented several methods for medical image fusion, and these methods achieved good results. However, medical image fusion is a resurgent field that needs to be enhanced to conquer the increasing challenges. This paper presents a comprehensive survey of some existing medical image fusion methods. It is expected that this study will be useful for the researchers scrutinizing medical image fusion. Furthermore, it is expected to establish a concrete foundation for developing more powerful fusion methods for medical applications.

Image fusion is to get a fused image that contains all important information from source images of the same scene. Meanwhile, multi-scale transforms and sparse representation (SR) are the two most effective techniques for image fusion. However, the SR-based image fusion methods are time-consuming and do not take the structural information of the source images into consideration. In addition, different multi-scale transform-based methods have their inevitable defects waiting to be solved till now. Therefore, in this paper, a new image fusion method combining nonsubsampled contourlet transform (NSCT) with SR is proposed. A decision map for the low-frequency coefficients according to the high-frequency coefficients is made to overcome these problems. Furthermore, it can reduce the calculation cost of the fusion algorithm and retain the useful information of source images as far as possible. Comparing with conventional multi-scale transform based methods and sparse representation based methods with a fixed or learned dictionary, the proposed method has better fusion performance in the field of medical image fusion.

Medical imaging has played an essential role in medicine and disease diagnosis. Medical images from a single modality contain limited data about the organ. On the other hand, images from different modalities contain valuable structural and functio nal data about an organ. Medical image fusion (MIF) strategies integrate complementary information from two medical images captured using distinct modalities. This paper offered a new multimodal MIF approach using the parameter-adaptive pulse-coupled neural networks (PA-PCNN) within the non-subsampled contourlet transform (NSCT). The NSCT decomposes those images into high- and low-frequency bands. PA-PCNN combines those bands. The fused image was created using the inverse of the NSCT approach. To prove the proposed approach’s performance, we appoint a variety of medical images like computed tomography (CT), magnetic resonance imaging (MRI), single-photon emission CT (SPECT), and positron emission tomography (PET). Our experiments use five fusion metrics to validate the proposed approach’s performance, such as entropy (EN), mutual information (MI), weighted edge information (QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}), nonlinear correlation information entropy (Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}), and average gradient (AG). Outcomes show that the proposed approach achieves high overall performance in visual and objective characteristics when compared with five well-known MIF methods. The average values for EN, MI, QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}, Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}, and AG with the proposed approach are 5.2144,3.1282,.6600,.8071, and 8.9874, respectively.

Image processing technology requires moreover the full image or the part of image which is to be processed from the user’s point of view like the radius of object etc. The main purpose of fusion is to diminish dissimilar error between the fused image and the input images. With respect to the medical diagnosis, the edges and outlines of the concerned objects is more important than extra information. So preserving the edge features of the image is worth for investigating the image fusion. The image with higher contrast contains more edge-like features. Here we propose a new medical image fusion scheme namely Local Energy Match NSCT based on discrete contourlet transformation, which is constructive to give the details of curve edges. It is used to progress the edge information of fused image by dropping the distortion. This transformation lead to crumbling of multimodal image addicted to finer and coarser details and finest details will be decayed into unusual resolution in dissimilar orientation. The input multimodal images namely CT and MRI images are first transformed by Non Sub sampled Contourlet Transformation (NSCT) which decomposes the image into low frequency and high frequency elements. In our system, the Low frequency coefficient of the image is fused by image averaging and Gabor filter bank algorithm. The processed High frequency coefficients of the image are fused by image averaging and gradient based fusion algorithm. Then the fused image is obtained by inverse NSCT with local energy match based coefficients. To evaluate the image fusion accuracy, Peak Signal to Noise Ratio (PSNR), Root Mean Square Error (RMSE) and Correlation Coefficient parameters are used in this work .

Image fusion is the process of gathering all the important information of multiple images into a single fused image. It is used in many fields like medical, military, remote sensing etc... Image fusion in a medical field is extensively used by physicians for analysis and treatment, as the fused image contains complementary features present in different medical images obtained from imaging devices. The wave atom transform based medical image fusion is proposed as the potential capabilities of wave atoms have been exploded in many applications like image denoising, fingerprint identification, compression etc. Medical image techniques became important tool for medical diagnosis and treatment. Medical images of different modalities such as MRI, CT, PET, SPECT provide information in a limited domain. In medical resources, MRI and CT images contain structural information whereas PET and SPECT images contain functional Information. Thus, single modality does not provide complete and accurate information, both structural as well as functional information is required. So far, many medical image fusion methods have been developed. The easiest way to perform image fusion is to perform weighted average of input images pixel by pixel. But this method produces side effects such as artifacts & contrast reduction. The proposed method is experimented on various medical images and compared with recent state of fusion methods. Results prove that images obtained from proposed method have better clarity and enhanced information and are practically more helpful for better treatment and quick analysis.

Deep learning methods for medical image fusion: A review

Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform

Due to the obvious advantages, the technology of medical imaging has been widely utilized in the medical areas. Commonly, the medical images of different imaging mechanism are able to guide the radiologist for the precise diagnosis of disease and for more effective interventional treatment procedures. Consequently, the fusion of different medical images of the same scene is necessary. This paper proposes a novel fusion technique for medical images based on non-subsampled contourlet transform (NSCT). Due to the better competence of image information capturing, NSCT is utilized to conduct the multi-scale and multi-directional decompositions of source images. In addition, the models of both region average energy and region coefficient deviation are used for the fusion for low-frequency sub-images and high-frequency ones, respectively. In comparison with several current typical fusion techniques, the proposed one has remarked superiorities in terms of both subjective and objective evaluations.

In this modern era, medical image processing is an indispensable part of many applications and practices in the medical domain. The images that are used should meet certain criteria, including having more accurate details and information than each individual image, which can help medical scientists with analysis and treatment. Medical image fusion is among the techniques that offer high-quality images, which are combined from different modalities. Multimodal medical image fusion provides remarkable improvement in the quality of the fused images. In this paper, we describe an image fusion method for magnetic resonance imaging (MRI) and computed tomography (CT) utilizing local features and fuzzy logic methods. The aim of the proposed technique is to create the maximum combination of useful information present in MRI and CT images. Image local features are distinguished and combined with fuzzy logic to calculate weights for each pixel. Simulation outcomes show that the proposed method produces considerably better results compared to cutting-edge techniques. The method is also used to detect and highlight tumorous areas, followed by morphology filters used to eliminate any noise and disturbance.

The synthesis of visual information from multiple medical imaging inputs to a single fused image without any loss of detail and distortion is known as multimodal medical image fusion. It improves the quality of biomedical images by preserving detailed features to advance the clinical utility of medical imaging meant for the analysis and treatment of medical disorders. This study develops a novel approach to fuse multimodal medical images utilizing anisotropic diffusion (AD) and non-subsampled contourlet transform (NSCT). First, the method employs anisotropic diffusion for decomposing input images to their base and detail layers to coarsely split two features of input images such as structural and textural information. The detail and base layers are further combined utilizing a sum-based fusion rule which maximizes noise filtering contrast level by effectively preserving most of the structural and textural details. NSCT is utilized to further decompose these images into their low and high-frequency coefficients. These coefficients are then combined utilizing the principal component analysis/Karhunen-Loeve (PCA/KL) based fusion rule independently by substantiating eigenfeature reinforcement in the fusion results. An NSCT-based multiresolution analysis is performed on the combined salient feature information and the contrast-enhanced fusion coefficients. Finally, an inverse NSCT is applied to each coefficient to produce the final fusion result. Experimental results demonstrate an advantage of the proposed technique using a publicly accessible dataset and conducted comparative studies on three pairs of medical images from different modalities and health. Our approach offers better visual and robust performance with better objective measurements for research development since it excellently preserves significant salient features and precision without producing abnormal information in the case of qualitative and quantitative analysis.

Multimodal medical image fusion integrates effective information from different modal images and integrates salient and complementary features, which can more comprehensively describe the condition of lesions and make medical diagnosis results more reliable. This paper proposes a multimodal medical image fusion method based on image detail enhancement and dual‐branch feature fusion (DEDF). First, the source images are preprocessed by guided filtering to enhance important details and improve the fusion and visualization effects. Then, local extreme maps are used as guides to smooth the source images. Finally, a DEDF mechanism based on guided filtering and bilateral filtering is established to obtain multiscale bright and dark feature maps, as well as base images of different modalities, which are fused to obtain a more comprehensive medical image and improve the accuracy of medical diagnosis results. Extensive experiments, compared qualitatively and quantitatively with various state‐of‐the‐art medical image fusion methods, validate the superior fusion performance and effectiveness of the proposed method.

In the current era of technological development, medical imaging plays an important part in several applications of medical diagnosis and therapy. This requires more precise images with much more details and information for correct medical diagnosis and therapy. Medical image fusion is one of the solutions for obtaining much spatial and spectral information in a single image. This article presents an optimization‐based contourlet image fusion approach in addition to a comparative study for the performance of both multi‐resolution and multi‐scale geometric effects on fusion quality. An optimized multi‐scale fusion technique based on the Non‐Subsampled Contourlet Transform (NSCT) using the Modified Central Force Optimization (MCFO) and local contrast enhancement techniques is presented. The first step in the proposed fusion approach is the histogram matching of one of the images to the other to allow the same dynamic range for both images. The NSCT is used after that to decompose the images to be fused into their coefficients. The MCFO technique is used to determine the optimum decomposition level and the optimum gain parameters for the best fusion of coefficients based on certain constraints. Finally, an additional contrast enhancement process is applied on the fused image to enhance its visual quality and reinforce details. The proposed fusion framework is subjectively and objectively evaluated with different fusion quality metrics including average gradient, local contrast, standard deviation (STD), edge intensity, entropy, peak signal‐to‐noise ratio, Q ab/f , and processing time. Experimental results demonstrate that the proposed optimized NSCT medical image fusion approach based on the MCFO and histogram matching achieves a superior performance with higher image quality, average gradient, edge intensity, STD, better local contrast and entropy, a good quality factor, and much more details in images. These characteristics help for more accurate medical diagnosis in different medical applications.

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

Nonsubsampled contourlet transform (NSCT) provides flexible multiresolution, anisotropy, and directional expansion for images. When the NSCT is introduced to image fusion, more information for fusion can be obtained and the impacts of mis-registration on the fused results can also be reduced effectively. Therefore, the NSCT is more suitable for image fusion. Based on NSCT and adaptive Genetic Algorithm (AGA), this paper proposes a new adaptive fusion method. This method can optimize decomposition level of the NSCT, lowpass subband coefficients and bandpass directional subband coefficients of the fused image. Finally, the algorithm is tested and compared with existing algorithms. The results show that the algorithm can produce high-quality fused images rapidly.

Medical image fusion based on enhanced three-layer image decomposition and Chameleon swarm algorithm