Image Fusion using Waveatom Transform for Medical Applications

Multimodality medical image fusion is the process of combining multiple images from single or multiple modalities of imaging. Medical image fusion methods are adopted to increase the quality of medical images by attaining the salient features in the fusion results. Hence, they raise the clinical applicability of medical images for appraisal and diagnosis problems. This purpose is achieved by capturing the complementary information presented in two or more images of different modalities in the fusion result. Medical image fusion is generally concerned with Magnetic Resonance Imaging (MRI), Magnetic Resonance Angiogram (MRA), Positron Emission Tomography (PET), Structural Positron Emission Tomography (SPET), Computerized Tomography (CT), and Single-Photon Emission Computed Tomography (SPECT) modalities. Each modality has its merits and drawbacks. This induces new fusion methods for merging information from multiple imaging modalities. Researchers have presented several methods for medical image fusion, and these methods achieved good results. However, medical image fusion is a resurgent field that needs to be enhanced to conquer the increasing challenges. This paper presents a comprehensive survey of some existing medical image fusion methods. It is expected that this study will be useful for the researchers scrutinizing medical image fusion. Furthermore, it is expected to establish a concrete foundation for developing more powerful fusion methods for medical applications.

Multimodal medical image fusion technique is an important and essential coalescing technique for the medical images with different modalities. The fused medical image carries more useful information than two or more relevant individual medical images of modality. To enhance, preserve edge and feature information and to remove noise of the source images a novel medical image fusion method has been developed with multiscale edge preserving decomposition L0 smoothing with sparse representation (SR) in nonsubsampled contourlet transform (NSCT) domain. The NSCT‐based image fusion method provides richer information in the spatial and spectral domains simultaneously. In this method, initially, L0 gradient smoothing filter is applied in two different modal clinical images separately to decompose the source images into two layers such as low frequency layer (LFL) and high frequency layer (HFL) which preserves the information and improves quality. To maintain the curve edges and the energy of the source medical images, the LFLs of different modal images are combined by using the NSCT ‐ SR fusion rule also to protect the detailed information of input medical images and reduce redundant information, the HFLs are combined by max‐absolute combination rule. By combining the reconstructed LFL and HFL, the resultant fused image is obtained. The experimental results show that the proposed work can provide better results than current methodologies in terms of both visual consistency and quantitative analysis.

In this modern era, medical image processing is an indispensable part of many applications and practices in the medical domain. The images that are used should meet certain criteria, including having more accurate details and information than each individual image, which can help medical scientists with analysis and treatment. Medical image fusion is among the techniques that offer high-quality images, which are combined from different modalities. Multimodal medical image fusion provides remarkable improvement in the quality of the fused images. In this paper, we describe an image fusion method for magnetic resonance imaging (MRI) and computed tomography (CT) utilizing local features and fuzzy logic methods. The aim of the proposed technique is to create the maximum combination of useful information present in MRI and CT images. Image local features are distinguished and combined with fuzzy logic to calculate weights for each pixel. Simulation outcomes show that the proposed method produces considerably better results compared to cutting-edge techniques. The method is also used to detect and highlight tumorous areas, followed by morphology filters used to eliminate any noise and disturbance.

Medical imaging has played an essential role in medicine and disease diagnosis. Medical images from a single modality contain limited data about the organ. On the other hand, images from different modalities contain valuable structural and functio nal data about an organ. Medical image fusion (MIF) strategies integrate complementary information from two medical images captured using distinct modalities. This paper offered a new multimodal MIF approach using the parameter-adaptive pulse-coupled neural networks (PA-PCNN) within the non-subsampled contourlet transform (NSCT). The NSCT decomposes those images into high- and low-frequency bands. PA-PCNN combines those bands. The fused image was created using the inverse of the NSCT approach. To prove the proposed approach’s performance, we appoint a variety of medical images like computed tomography (CT), magnetic resonance imaging (MRI), single-photon emission CT (SPECT), and positron emission tomography (PET). Our experiments use five fusion metrics to validate the proposed approach’s performance, such as entropy (EN), mutual information (MI), weighted edge information (QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}), nonlinear correlation information entropy (Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}), and average gradient (AG). Outcomes show that the proposed approach achieves high overall performance in visual and objective characteristics when compared with five well-known MIF methods. The average values for EN, MI, QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}, Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}, and AG with the proposed approach are 5.2144,3.1282,.6600,.8071, and 8.9874, respectively.

In the current era of technological development, medical imaging plays an important role in many applications of medical diagnosis and therapy. In this regard, medical image fusion could be a powerful tool to combine multi-modal images by using image processing techniques. But, conventional approaches failed to provide the effective image quality assessments and robustness of fused image. To overcome these drawbacks, in this work three-stage multiscale decomposition (TSMSD) using pulse-coupled neural networks with adaptive arguments (PCNN-AA) approach is proposed for multi-modal medical image fusion. Initially, nonsubsampled shearlet transform (NSST) is applied onto the source images to decompose them into low frequency and high frequency bands. Then, low frequency bands of both the source images are fused using nonlinear anisotropic filtering with discrete Karhunen–Loeve transform (NLAF-DKLT) methodology. Next, high frequency bands obtained from NSST are fused using PCNN-AA approach. Now, fused low frequency and high frequency bands are reconstructed using NSST reconstruction. Finally, band fusion rule algorithm with pyramid reconstruction is applied to get final fused medical image. Extensive simulation outcome discloses the superiority of proposed TSMSD using PCNN-AA approach as compared to state-of-the-art medical image fusion methods in terms of fusion quality metrics such as entropy (E), mutual information (MI), mean (M), standard deviation (STD), correlation coefficient (CC) and computational complexity.

Image compression or fusion is the concept of identifying in-depth parameters of disease variables, and requires output images that preserve all the viable and prominent information that is gathered from source images without any further introduction of artifacts or unnecessary distortions. Measurement of images for prospective evaluation and image fusion depends on various performance measures, such as structure similarity index, standard deviation, edge detection, correlation coefficient and high pass correlation, average gradient, root-mean-square error, peak signal-to-noise ratio, entropy, etc. This review discusses various medical image fusion modalities focused on Principal Component Analysis, Independent Component Analysis, and wavelet transform. An introduction to the usefulness of such modalities is presented, suggesting safe hybrid modality combinations that could greatly enhance the image fusion process. Novel trends in medical image fusion techniques to achieve a perfectly desired, quality image, the future prospects of an ideal technique for medical imaging, and recognition of diseases are covered.

Nowadays, there are a lot of medical images and their numbers are increasing day by day. These medical images are stored in the large database. To minimize the redundancy and optimize the storage capacity of images, medical image fusion is used. The main aim of medical image fusion is to combine complementary information from multiple imaging modalities (e.g. CT, MRI, PET, etc.) of the same scene. After performing medical image fusion, the resultant image is more informative and suitable for patient diagnosis. There are some fusion techniques which are described in this chapter to obtain fused image. This chapter presents two approaches to image fusion, namely spatial domain Fusion technique and transforms domain Fusion technique. This chapter describes Techniques such as Principal Component Analysis which is spatial domain technique and Discrete Wavelet Transform and Stationary Wavelet Transform which are Transform domain techniques. Performance metrics are implemented to evaluate the performance of image fusion algorithm.

Deep learning methods for medical image fusion: A review

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation

Wavelet transforms have emerged as a powerful tool in image fusion. However, the study and analysis of medical image fusion is still a challenging area of research. Therefore, in this paper, we propose a multiscale fusion of multimodal medical images in wavelet domain. Fusion of medical images has been performed at multiple scales varying from minimum to maximum level using maximum selection rule which provides more flexibility and choice to select the relevant fused images. The experimental analysis of the proposed method has been performed with several sets of medical images. Fusion results have been evaluated subjectively and objectively with existing state-of-the-art fusion methods which include several pyramid- and wavelet-transform-based fusion methods and principal component analysis (PCA) fusion method. The comparative analysis of the fusion results has been performed with edge strength (Q), mutual information (MI), entropy (E), standard deviation (SD), blind structural similarity index metric (BSSIM), spatial frequency (SF), and average gradient (AG) metrics. The combined subjective and objective evaluations of the proposed fusion method at multiple scales showed the effectiveness and goodness of the proposed approach.

Although medical imaging is frequently used to diagnose diseases, in complex diagnostic situations, specialists typically need to look at different modalities of image information. Creating a composite multimodal medical image can aid professionals in making quick and accurate diagnoses of diseases. The fused images of many medical image fusion algorithms, however, are frequently unable to precisely retain the functional and structural information of the source image. This work develops an end-to-end model based on GAN (U-Patch GAN) to implement the self-supervised fusion of multimodal brain images in order to enhance the fusion quality. The model uses the classical network U-net as the generator, and it uses the dual adversarial mechanism based on the Markovian discriminator (PatchGAN) to enhance the generator's attention to high-frequency information. To ensure that the network satisfies the Lipschitz continuity, we apply the spectral norm to each layer of the network. We also propose better adversarial loss and feature loss (feature matching loss and VGG-16 perceptual loss) based on the F-norm, which significantly enhance the quality of fused images. On public data sets, we performed a lot of tests. First, we studied how clinically useful the fused image was. The model's performance in single-slice images and continuous-slice images was then confirmed by comparison with other six most popular mainstream fusion approaches. Finally, we verify the effectiveness of the adversarial loss and feature loss.

A novel dictionary learning approach for multi-modality medical image fusion

Medical image fusion plays an important role in clinical applications such as image-guided surgery, image-guided radiotherapy, noninvasive diagnosis, and treatment planning. In this paper, we propose a novel multi-modal medical image fusion method based on simplified pulse-coupled neural network and quaternion wavelet transform. The proposed fusion algorithm is capable of combining not only pairs of computed tomography (CT) and magnetic resonance (MR) images, but also pairs of CT and proton-density-weighted MR images, and multi-spectral MR images such as T1 and T2. Experiments on six pairs of multi-modal medical images are conducted to compare the proposed scheme with four existing methods. The performances of various methods are investigated using mutual information metrics and comprehensive fusion performance characterization (total fusion performance, fusion loss, and modified fusion artifacts criteria). The experimental results show that the proposed algorithm not only extracts more important visual information from source images, but also effectively avoids introducing artificial information into fused medical images. It significantly outperforms existing medical image fusion methods in terms of subjective performance and objective evaluation metrics.

Industrial Internet of Things (IIoT) refers to the extension and utilization of the Internet of Things (IoT) in industrial sectors and applications. Medical image fusion and classification has been utilized to get valuable significant multimodality medical image data. Recently, deep learning methods offer an effective way to design an end-to-end model that can determine the final classification labels. This chapter introduces a Multimodality Image Fusion Classification (MMIFC) by the incorporation of image fusion, feature extraction, and classification techniques. Initially, the input medical images were fused by the use of optimal shearlet transform where the coefficients of shearlets are optimized by the use of Enhanced Monarch Butterfly Optimization (EMBO) algorithm. In the next stage, the fused images were classified based on the feature extractor and deep learning model to check the test image is malignant and benign. To further optimize the performance of the deep learning model, hyperparameter tuning process takes place by the use of Bayesian optimization model to optimize the weights of structure. Finally, this ODNN model classifies the MMFI as class 1 or 0. A detailed simulation result takes place to ensure the effective performance of the proposed method. From the experimental results, the proposed strategy accomplishes better fusion rate and classification results compared with other supervised and non-supervised learning techniques.