Multiscale Medical Image Fusion in Wavelet Domain

Background: During the two past decades, medical image fusion has become an essential part ofmodern medicine due to the availability of numerous imaging modalities (e.g., MRI, CT, SPECT,etc.). This paper presents a new medical image fusion algorithm based on PCA and DTCWT,which uses different fusion rules to obtain a new image containing more information than any ofthe input images.Methods: A new image fusion algorithm improves the visual quality of the fused image, based onfeature selection and Principal Component Analysis (PCA) in the Dual-Tree Complex WaveletTransform (DTCWT) domain. It is called Feature Selection with Principal Component Analysisand Dual-Tree Complex Wavelet Transform (FSPCA-DTCWT). Using different fusion rules in asingle algorithm result in correctly reconstructed image (fused image), this combination willproduce a new technique, which employs the advantages of each method separately. The DTCWTpresents good directionality since it considers the edge information in six directions and providesapproximate shift invariant. The main goal of PCA is to extract the most significant characteristics(represented by the wavelet coefficients) in order to improve the spatial resolution. The proposedalgorithm fuses the detailed wavelet coefficients of input images using features selection rule.Results: Several experiments have been conducted over different sets of multimodal medicalimages such as CT/MRI, MRA/T1-MRI. However, due to pages-limit on a paper, only results ofthree sets have been presented. The FSPCA-DTCWT algorithm is compared to recent fusionmethods presented in the literature (eight methods) in terms of visual quality and quantitativelyusing well-known fusion performance metrics (five metrics). Results showed that the proposedalgorithm outperforms the existing ones regarding visual and quantitative evaluations.Conclusion: This paper focuses on medical image fusion of different modalities. A novel imagefusion algorithm based on DTCWT to merge multimodal medical images has been proposed.Experiments have been performed using two different sets of multimodal medical images. Theresults show that the proposed fusion method significantly outperforms the recent fusiontechniques reported in the literature.

Multimodality medical image fusion is the process of combining multiple images from single or multiple modalities of imaging. Medical image fusion methods are adopted to increase the quality of medical images by attaining the salient features in the fusion results. Hence, they raise the clinical applicability of medical images for appraisal and diagnosis problems. This purpose is achieved by capturing the complementary information presented in two or more images of different modalities in the fusion result. Medical image fusion is generally concerned with Magnetic Resonance Imaging (MRI), Magnetic Resonance Angiogram (MRA), Positron Emission Tomography (PET), Structural Positron Emission Tomography (SPET), Computerized Tomography (CT), and Single-Photon Emission Computed Tomography (SPECT) modalities. Each modality has its merits and drawbacks. This induces new fusion methods for merging information from multiple imaging modalities. Researchers have presented several methods for medical image fusion, and these methods achieved good results. However, medical image fusion is a resurgent field that needs to be enhanced to conquer the increasing challenges. This paper presents a comprehensive survey of some existing medical image fusion methods. It is expected that this study will be useful for the researchers scrutinizing medical image fusion. Furthermore, it is expected to establish a concrete foundation for developing more powerful fusion methods for medical applications.

A method for feature level image fusion for multimodal medical images in second generation wavelet domain (lifting wavelet transform domain) is proposed. The feature fused is edge and boundary information of input images that is extracted using wavelet transform modulus maxima criterion. The image fusion performance is evaluated by standard deviation, entropy, cross entropy and gradient parameters. Experimental results show that the proposed method gives better results for image fusion as image contrast, average information content and detail information of fused image are increased. This method has further advantages of fast implementation, flexibility, saving of auxiliary memory, property of perfect reconstruction and simplicity as we have used lifting wavelet transform. The reduction in computational complexity has been achieved by a factor of two as compared to the nonlifted wavelet transform.

Abstract: Multi-modality medical or clinical image fusion is a field of study aimed at enhancing diagnostic accuracy and aid in decisions to be taken by medical professional. Various fusion techniques such as pixel-based, region-based, and transformbased approaches are applied in image fusion to provide accurate fusion. Different devices which take scans of body such as MRI, CT, PET, SPECT, Ultrasound hold and carry different features, and different medical sensors obtain different information of the particular part of the body. Each of these imaging modalities offer only specific information that is used for the detection and analysis of specific problem. The idea behind fusion is to achieve and get better contrast and better fused image. The algorithm is making use of the common pyramid type and similarity type fusion algorithm with the neural networks model to achieve a better and more flexible fusion method. The advantages of image fusion medically are widespread. It plays a pivotal role in tumour localization, surgical planning and in treatment assessment.

An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion

Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation

In this modern era, medical image processing is an indispensable part of many applications and practices in the medical domain. The images that are used should meet certain criteria, including having more accurate details and information than each individual image, which can help medical scientists with analysis and treatment. Medical image fusion is among the techniques that offer high-quality images, which are combined from different modalities. Multimodal medical image fusion provides remarkable improvement in the quality of the fused images. In this paper, we describe an image fusion method for magnetic resonance imaging (MRI) and computed tomography (CT) utilizing local features and fuzzy logic methods. The aim of the proposed technique is to create the maximum combination of useful information present in MRI and CT images. Image local features are distinguished and combined with fuzzy logic to calculate weights for each pixel. Simulation outcomes show that the proposed method produces considerably better results compared to cutting-edge techniques. The method is also used to detect and highlight tumorous areas, followed by morphology filters used to eliminate any noise and disturbance.

The objective of any multimodal medical image fusion algorithm is to assist a radiologist for better decision-making during the diagnosis and therapy by integrating the anatomical (magnetic resonance imaging) and functional (positron emission tomography/ single-photon emission computed tomography) information. We proposed a new medical image fusion method based on content-based decomposition, Principal Component Analysis (PCA), and sigmoid function. We considered Empirical Wavelet Transform (EWT) for content-based decomposition purposes since it can preserve crucial medical image information such as edges and corners. PCA is used to obtain initial weights corresponding to each detail layer. In our experiments, we found that direct usage of PCA for detail layer fusion introduces severe artifacts into the fused image due to weight scaling issues. In order to tackle this, we considered using the sigmoid function for better weight scaling. We considered 24 pairs of MRI-PET and 24 pairs of MRI-SPECT images for fusion, and the results are measured using four significant quantitative metrics. Finally, we compared our proposed method with other state-of-the-art transformbased fusion approaches, using traditional and recent performance measures. An appreciable improvement is observed in both qualitative and quantitative results compared to other fusion methods.

This paper presents a novel medical image fusion technique named as hybrid technique developed by combining the features of principal component analysis (PCA) and stationary wavelet transform (SWT) fusion rules. Magnetic resonance imaging (MRI) and computed tomography (CT) images are merged to acquire contemporary image that augments the supplementary information to the consultant for better diagnosis. In this work, fusion results of proposed hybrid technique are compared with those using PCA and SWT techniques. The performance evaluation is carried out in terms of Mean (M), Standard deviation(SD), Entropy (H), Average gradient(AG), Root mean square error (RMSE), Peak signal to noise ratio (PSNR), Mutual Information (MI) and spatial frequency(SF). Comparison result demonstrates a better performance using hybrid technique than that of other two methods (PCA and SWT).

Medical imaging has played an essential role in medicine and disease diagnosis. Medical images from a single modality contain limited data about the organ. On the other hand, images from different modalities contain valuable structural and functio nal data about an organ. Medical image fusion (MIF) strategies integrate complementary information from two medical images captured using distinct modalities. This paper offered a new multimodal MIF approach using the parameter-adaptive pulse-coupled neural networks (PA-PCNN) within the non-subsampled contourlet transform (NSCT). The NSCT decomposes those images into high- and low-frequency bands. PA-PCNN combines those bands. The fused image was created using the inverse of the NSCT approach. To prove the proposed approach’s performance, we appoint a variety of medical images like computed tomography (CT), magnetic resonance imaging (MRI), single-photon emission CT (SPECT), and positron emission tomography (PET). Our experiments use five fusion metrics to validate the proposed approach’s performance, such as entropy (EN), mutual information (MI), weighted edge information (QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}), nonlinear correlation information entropy (Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}), and average gradient (AG). Outcomes show that the proposed approach achieves high overall performance in visual and objective characteristics when compared with five well-known MIF methods. The average values for EN, MI, QAB/F\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$^{AB/F}$$\\end{document}, Qncie\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{ncie}$$\\end{document}, and AG with the proposed approach are 5.2144,3.1282,.6600,.8071, and 8.9874, respectively.

Deep learning-based image fusion techniques have been widely applied in multi-modal medical image fusion. However, existing methods often face challenges such as insufficient preservation of image structure and quality, loss of multi-scale structural information, and noise and artifacts during the fusion process. To address these issues, this paper proposes an uncertainty and gradient-guided deep medical image fusion network (UAGFusion), which includes an uncertainty-guided fusion network (UAFN) and a gradient-guided modulation gate refinement network (GGMGFN), with the aim of exploring methods to improve medical image fusion performance. The UAFN incorporates an uncertainty-guided feature enhancement module, cross-scale attention mechanism, and multi-level feature fusion (FF) module to enhance the integration and expression capabilities of multi-modal features. The GGMGFN employs a gradient-guided detail extractor to capture edge and texture information and uses a modulated gated fusion module to integrate multi-source feature streams. To guide network training, this paper designs a hybrid loss function comprising content loss, edge loss, multi-scale structural similarity loss, perceptual loss, and uncertainty perceptual loss to achieve a balance between fusion quality and uncertainty. Experimental results demonstrate that the proposed method achieves competitive performance compared to existing approaches across several objective metrics, while offering improvements in structural information preservation and noise robustness.

Image compression or fusion is the concept of identifying in-depth parameters of disease variables, and requires output images that preserve all the viable and prominent information that is gathered from source images without any further introduction of artifacts or unnecessary distortions. Measurement of images for prospective evaluation and image fusion depends on various performance measures, such as structure similarity index, standard deviation, edge detection, correlation coefficient and high pass correlation, average gradient, root-mean-square error, peak signal-to-noise ratio, entropy, etc. This review discusses various medical image fusion modalities focused on Principal Component Analysis, Independent Component Analysis, and wavelet transform. An introduction to the usefulness of such modalities is presented, suggesting safe hybrid modality combinations that could greatly enhance the image fusion process. Novel trends in medical image fusion techniques to achieve a perfectly desired, quality image, the future prospects of an ideal technique for medical imaging, and recognition of diseases are covered.

The emergence of multimodal medical imaging technology greatly increases the accuracy of clinical diagnosis and etiological analysis. Nevertheless, each medical imaging modal unavoidably has its own limitations, so the fusion of multimodal medical images may become an effective solution. In this paper, a novel fusion method on the multimodal medical images exploiting convolutional neural network (CNN) and extreme learning machine (ELM) is proposed. As a typical representative in deep learning, CNN has been gaining more and more popularity in the field of image processing. However, CNN often suffers from several drawbacks, such as high computational costs and intensive human interventions. To this end, the model of convolutional extreme learning machine (CELM) is constructed by incorporating ELM into the traditional CNN model. CELM serves as an important tool to extract and capture the features of the source images from a variety of different angles. The final fused image can be obtained by integrating the significant features together. Experimental results indicate that, the proposed method is not only helpful to enhance the accuracy of the lesion detection and localization, but also superior to the current state-of-the-art ones in terms of both subjective visual performance and objective criteria.

NUC-Fuse: Multimodal medical image fusion using nuclear norm & classification of brain tumors using ARBFN

Image processing technology requires moreover the full image or the part of image which is to be processed from the user’s point of view like the radius of object etc. The main purpose of fusion is to diminish dissimilar error between the fused image and the input images. With respect to the medical diagnosis, the edges and outlines of the concerned objects is more important than extra information. So preserving the edge features of the image is worth for investigating the image fusion. The image with higher contrast contains more edge-like features. Here we propose a new medical image fusion scheme namely Local Energy Match NSCT based on discrete contourlet transformation, which is constructive to give the details of curve edges. It is used to progress the edge information of fused image by dropping the distortion. This transformation lead to crumbling of multimodal image addicted to finer and coarser details and finest details will be decayed into unusual resolution in dissimilar orientation. The input multimodal images namely CT and MRI images are first transformed by Non Sub sampled Contourlet Transformation (NSCT) which decomposes the image into low frequency and high frequency elements. In our system, the Low frequency coefficient of the image is fused by image averaging and Gabor filter bank algorithm. The processed High frequency coefficients of the image are fused by image averaging and gradient based fusion algorithm. Then the fused image is obtained by inverse NSCT with local energy match based coefficients. To evaluate the image fusion accuracy, Peak Signal to Noise Ratio (PSNR), Root Mean Square Error (RMSE) and Correlation Coefficient parameters are used in this work .