Survey on Multi-Modal Medical Image Fusion

Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation

Multimodal medical image fusion plays an important role in clinical applications. However, multimodal medical image fusion methods ignore the feature dependence among modals, and the feature fusion ability with different granularity is not strong. A Long-Range Correlation-Guided Dual-Encoder Fusion Network for Medical Images is proposed in this paper. The main innovations of this paper are as follows: Firstly, A Cross-dimension Multi-scale Feature Extraction Module (CMFEM) is designed in the encoder, by extracting multi-scale features and aggregating coarse-to-fine features, the model realizes fine-grained feature enhancement in different modalities. Secondly, a Long-range Correlation Fusion Module (LCFM) is designed, by calculating the long-range correlation coefficient between local features and global features, the same granularity features are fused by the long-range correlation fusion module. long-range dependencies between modalities are captured by the model, and different granularity features are aggregated. Finally, this paper is validated on clinical multimodal lung medical image dataset and brain medical data dataset. On the lung medical image dataset, IE, AG, {textbf {Q}}^{{textbf {AB/F}}}, and EI metrics are improved by 4.53%, 4.10%, 6.19%, and 6.62% respectively. On the brain medical image dataset, SF, VIF, and {textbf {Q}}^{{textbf {AB/F}}} metrics are improved by 3.88%, 15.71%, and 7.99% respectively. This model realizes better fusion performance, which plays an important role in the fusion of multimodal medical images.

Multimodal medical image fusion is to extract information from different modal images into a single one and obtain the organizational characteristics of source images. Different medical imaging measures such as CT and MRI often bring different visual morphology, however, the salient features of tissues are basically the same from the perspective of human eyes. According to this characteristic, an improved image fusion algorithm based on visual salience detection is proposed in this paper. First, the GBVS algorithm was introduced to calculate visual salience of two registered source images, and then decompose the source images in NSST domain to obtain their low-frequency and high-frequency sub-bands. For the low-frequency sub-bands, local energy and GBVS graph are input into fuzzy logic system to obtain the respective weights for the fused low-frequency sub-band. For the high-frequency sub-bands, the NSML values of each sub-band were calculated and compared to obtain the fused high-frequency sub-band. The final fused image was obtained by using the inverse NSST transformation. Applying this method to multimodal medical image fusion, the visual quality of the image can be enhanced effectively and the salient features of tissues can be preserved well. Experiments on multimodal fusion of different gray-scale medical images show that the proposed method has advantages in retention of image salient features and the overall image contrast, and has better objective index than the comparison models.

Wavelet transforms have emerged as a powerful tool in image fusion. However, the study and analysis of medical image fusion is still a challenging area of research. Therefore, in this paper, we propose a multiscale fusion of multimodal medical images in wavelet domain. Fusion of medical images has been performed at multiple scales varying from minimum to maximum level using maximum selection rule which provides more flexibility and choice to select the relevant fused images. The experimental analysis of the proposed method has been performed with several sets of medical images. Fusion results have been evaluated subjectively and objectively with existing state-of-the-art fusion methods which include several pyramid- and wavelet-transform-based fusion methods and principal component analysis (PCA) fusion method. The comparative analysis of the fusion results has been performed with edge strength (Q), mutual information (MI), entropy (E), standard deviation (SD), blind structural similarity index metric (BSSIM), spatial frequency (SF), and average gradient (AG) metrics. The combined subjective and objective evaluations of the proposed fusion method at multiple scales showed the effectiveness and goodness of the proposed approach.

The emergence of multimodal medical imaging technology greatly increases the accuracy of clinical diagnosis and etiological analysis. Nevertheless, each medical imaging modal unavoidably has its own limitations, so the fusion of multimodal medical images may become an effective solution. In this paper, a novel fusion method on the multimodal medical images exploiting convolutional neural network (CNN) and extreme learning machine (ELM) is proposed. As a typical representative in deep learning, CNN has been gaining more and more popularity in the field of image processing. However, CNN often suffers from several drawbacks, such as high computational costs and intensive human interventions. To this end, the model of convolutional extreme learning machine (CELM) is constructed by incorporating ELM into the traditional CNN model. CELM serves as an important tool to extract and capture the features of the source images from a variety of different angles. The final fused image can be obtained by integrating the significant features together. Experimental results indicate that, the proposed method is not only helpful to enhance the accuracy of the lesion detection and localization, but also superior to the current state-of-the-art ones in terms of both subjective visual performance and objective criteria.

An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion

A novel dictionary learning approach for multi-modality medical image fusion

Nowadays, medical imaging becomes a common part of everyday clinical practices. Despite enormous progresses, still there is no single modality which can represent all aspects of the human body. For example, CT is suitable to view dense structures while MRI provides high resolution for soft tissue. In this paper, we propose a novel method for fusion of multimodal medical images. First, the surfacelet transform is used to decompose the source images. Then, we effectively combine the low and high frequency coefficients. Finally, inverse transform would provide the fused image. Experimental results exhibited the superior solution quality of our approach in comparison to a number of well-known counterpart algorithms.

In medical science, the role of medical images is very crucial to identify the diagnosis. Variety of medical images such as CT (Computed Tomography) images, MRI (magnetic resonance imaging) images are available but many times these different modalities need more informatics images. Therefore, multi-modality medical image fusion technique is one of the solutions which can provide more informatics details by fusing multi-modality medical images. In this paper, a new approach of multi-modality medical image fusion approach is proposed where fuzzy local information based C-means clustering and local entropy fusion approach is performed in SWT (stationary wavelet transform) Domain. To measure the performance of proposed scheme, a comparative study is also performed over the existing methods. The result analysis is performed in both visually and also by performance metrics such as mutual information, edge based similarity and blind image quality. From experimental evaluation, it can be analyzed that proposed scheme gives better outcomes in compare to existing methods.

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

NUC-Fuse: Multimodal medical image fusion using nuclear norm & classification of brain tumors using ARBFN

The synthesis of visual information from multiple medical imaging inputs to a single fused image without any loss of detail and distortion is known as multimodal medical image fusion. It improves the quality of biomedical images by preserving detailed features to advance the clinical utility of medical imaging meant for the analysis and treatment of medical disorders. This study develops a novel approach to fuse multimodal medical images utilizing anisotropic diffusion (AD) and non-subsampled contourlet transform (NSCT). First, the method employs anisotropic diffusion for decomposing input images to their base and detail layers to coarsely split two features of input images such as structural and textural information. The detail and base layers are further combined utilizing a sum-based fusion rule which maximizes noise filtering contrast level by effectively preserving most of the structural and textural details. NSCT is utilized to further decompose these images into their low and high-frequency coefficients. These coefficients are then combined utilizing the principal component analysis/Karhunen-Loeve (PCA/KL) based fusion rule independently by substantiating eigenfeature reinforcement in the fusion results. An NSCT-based multiresolution analysis is performed on the combined salient feature information and the contrast-enhanced fusion coefficients. Finally, an inverse NSCT is applied to each coefficient to produce the final fusion result. Experimental results demonstrate an advantage of the proposed technique using a publicly accessible dataset and conducted comparative studies on three pairs of medical images from different modalities and health. Our approach offers better visual and robust performance with better objective measurements for research development since it excellently preserves significant salient features and precision without producing abnormal information in the case of qualitative and quantitative analysis.

Human visual system inspired multi-modal medical image fusion framework

In this modern era, medical image processing is an indispensable part of many applications and practices in the medical domain. The images that are used should meet certain criteria, including having more accurate details and information than each individual image, which can help medical scientists with analysis and treatment. Medical image fusion is among the techniques that offer high-quality images, which are combined from different modalities. Multimodal medical image fusion provides remarkable improvement in the quality of the fused images. In this paper, we describe an image fusion method for magnetic resonance imaging (MRI) and computed tomography (CT) utilizing local features and fuzzy logic methods. The aim of the proposed technique is to create the maximum combination of useful information present in MRI and CT images. Image local features are distinguished and combined with fuzzy logic to calculate weights for each pixel. Simulation outcomes show that the proposed method produces considerably better results compared to cutting-edge techniques. The method is also used to detect and highlight tumorous areas, followed by morphology filters used to eliminate any noise and disturbance.

A method for feature level image fusion for multimodal medical images in second generation wavelet domain (lifting wavelet transform domain) is proposed. The feature fused is edge and boundary information of input images that is extracted using wavelet transform modulus maxima criterion. The image fusion performance is evaluated by standard deviation, entropy, cross entropy and gradient parameters. Experimental results show that the proposed method gives better results for image fusion as image contrast, average information content and detail information of fused image are increased. This method has further advantages of fast implementation, flexibility, saving of auxiliary memory, property of perfect reconstruction and simplicity as we have used lifting wavelet transform. The reduction in computational complexity has been achieved by a factor of two as compared to the nonlifted wavelet transform.