Long-range correlation-guided dual-encoder fusion network for medical images

Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation

Abstract: Multi-modality medical or clinical image fusion is a field of study aimed at enhancing diagnostic accuracy and aid in decisions to be taken by medical professional. Various fusion techniques such as pixel-based, region-based, and transformbased approaches are applied in image fusion to provide accurate fusion. Different devices which take scans of body such as MRI, CT, PET, SPECT, Ultrasound hold and carry different features, and different medical sensors obtain different information of the particular part of the body. Each of these imaging modalities offer only specific information that is used for the detection and analysis of specific problem. The idea behind fusion is to achieve and get better contrast and better fused image. The algorithm is making use of the common pyramid type and similarity type fusion algorithm with the neural networks model to achieve a better and more flexible fusion method. The advantages of image fusion medically are widespread. It plays a pivotal role in tumour localization, surgical planning and in treatment assessment.

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

In medical science, the role of medical images is very crucial to identify the diagnosis. Variety of medical images such as CT (Computed Tomography) images, MRI (magnetic resonance imaging) images are available but many times these different modalities need more informatics images. Therefore, multi-modality medical image fusion technique is one of the solutions which can provide more informatics details by fusing multi-modality medical images. In this paper, a new approach of multi-modality medical image fusion approach is proposed where fuzzy local information based C-means clustering and local entropy fusion approach is performed in SWT (stationary wavelet transform) Domain. To measure the performance of proposed scheme, a comparative study is also performed over the existing methods. The result analysis is performed in both visually and also by performance metrics such as mutual information, edge based similarity and blind image quality. From experimental evaluation, it can be analyzed that proposed scheme gives better outcomes in compare to existing methods.

The emergence of multimodal medical imaging technology greatly increases the accuracy of clinical diagnosis and etiological analysis. Nevertheless, each medical imaging modal unavoidably has its own limitations, so the fusion of multimodal medical images may become an effective solution. In this paper, a novel fusion method on the multimodal medical images exploiting convolutional neural network (CNN) and extreme learning machine (ELM) is proposed. As a typical representative in deep learning, CNN has been gaining more and more popularity in the field of image processing. However, CNN often suffers from several drawbacks, such as high computational costs and intensive human interventions. To this end, the model of convolutional extreme learning machine (CELM) is constructed by incorporating ELM into the traditional CNN model. CELM serves as an important tool to extract and capture the features of the source images from a variety of different angles. The final fused image can be obtained by integrating the significant features together. Experimental results indicate that, the proposed method is not only helpful to enhance the accuracy of the lesion detection and localization, but also superior to the current state-of-the-art ones in terms of both subjective visual performance and objective criteria.

Human visual system inspired multi-modal medical image fusion framework

The synthesis of visual information from multiple medical imaging inputs to a single fused image without any loss of detail and distortion is known as multimodal medical image fusion. It improves the quality of biomedical images by preserving detailed features to advance the clinical utility of medical imaging meant for the analysis and treatment of medical disorders. This study develops a novel approach to fuse multimodal medical images utilizing anisotropic diffusion (AD) and non-subsampled contourlet transform (NSCT). First, the method employs anisotropic diffusion for decomposing input images to their base and detail layers to coarsely split two features of input images such as structural and textural information. The detail and base layers are further combined utilizing a sum-based fusion rule which maximizes noise filtering contrast level by effectively preserving most of the structural and textural details. NSCT is utilized to further decompose these images into their low and high-frequency coefficients. These coefficients are then combined utilizing the principal component analysis/Karhunen-Loeve (PCA/KL) based fusion rule independently by substantiating eigenfeature reinforcement in the fusion results. An NSCT-based multiresolution analysis is performed on the combined salient feature information and the contrast-enhanced fusion coefficients. Finally, an inverse NSCT is applied to each coefficient to produce the final fusion result. Experimental results demonstrate an advantage of the proposed technique using a publicly accessible dataset and conducted comparative studies on three pairs of medical images from different modalities and health. Our approach offers better visual and robust performance with better objective measurements for research development since it excellently preserves significant salient features and precision without producing abnormal information in the case of qualitative and quantitative analysis.

Medical image fusion is very important for the diagnosis and treatment of diseases. In recent years, there have been a number of different multi-modal medical image fusion algorithms that can provide delicate contexts for disease diagnosis more clearly and more conveniently. Recently, nuclear norm minimization and deep learning have been used effectively in image processing. A multi-modality medical image fusion method using a rolling guidance filter (RGF) with a convolutional neural network (CNN) based feature mapping and nuclear norm minimization (NNM) is proposed. At first, we decompose medical images to base layer components and detail layer components by using RGF. In the next step, we get the basic fused image through the pretrained CNN model. The CNN model with pre-training is used to obtain the significant characteristics of the base layer components. And we can compute the activity level measurement from the regional energy of CNN-based fusion maps. Then, a detail fused image is gained by NNM. That is, we use NNM to fuse the detail layer components. At last, the basic and detail fused images are integrated into the fused result. From the comparison with the most advanced fusion algorithms, the results of experiments indicate that this fusion algorithm has the best effect in visual evaluation and objective standard. The fusion algorithm using RGF and CNN-based feature mapping, combined with NNM, can improve fusion effects and suppress artifacts and blocking effects in the fused results.

Multimodal medical image fusion is to extract information from different modal images into a single one and obtain the organizational characteristics of source images. Different medical imaging measures such as CT and MRI often bring different visual morphology, however, the salient features of tissues are basically the same from the perspective of human eyes. According to this characteristic, an improved image fusion algorithm based on visual salience detection is proposed in this paper. First, the GBVS algorithm was introduced to calculate visual salience of two registered source images, and then decompose the source images in NSST domain to obtain their low-frequency and high-frequency sub-bands. For the low-frequency sub-bands, local energy and GBVS graph are input into fuzzy logic system to obtain the respective weights for the fused low-frequency sub-band. For the high-frequency sub-bands, the NSML values of each sub-band were calculated and compared to obtain the fused high-frequency sub-band. The final fused image was obtained by using the inverse NSST transformation. Applying this method to multimodal medical image fusion, the visual quality of the image can be enhanced effectively and the salient features of tissues can be preserved well. Experiments on multimodal fusion of different gray-scale medical images show that the proposed method has advantages in retention of image salient features and the overall image contrast, and has better objective index than the comparison models.

Nowadays, medical imaging becomes a common part of everyday clinical practices. Despite enormous progresses, still there is no single modality which can represent all aspects of the human body. For example, CT is suitable to view dense structures while MRI provides high resolution for soft tissue. In this paper, we propose a novel method for fusion of multimodal medical images. First, the surfacelet transform is used to decompose the source images. Then, we effectively combine the low and high frequency coefficients. Finally, inverse transform would provide the fused image. Experimental results exhibited the superior solution quality of our approach in comparison to a number of well-known counterpart algorithms.

Background: During the two past decades, medical image fusion has become an essential part ofmodern medicine due to the availability of numerous imaging modalities (e.g., MRI, CT, SPECT,etc.). This paper presents a new medical image fusion algorithm based on PCA and DTCWT,which uses different fusion rules to obtain a new image containing more information than any ofthe input images.Methods: A new image fusion algorithm improves the visual quality of the fused image, based onfeature selection and Principal Component Analysis (PCA) in the Dual-Tree Complex WaveletTransform (DTCWT) domain. It is called Feature Selection with Principal Component Analysisand Dual-Tree Complex Wavelet Transform (FSPCA-DTCWT). Using different fusion rules in asingle algorithm result in correctly reconstructed image (fused image), this combination willproduce a new technique, which employs the advantages of each method separately. The DTCWTpresents good directionality since it considers the edge information in six directions and providesapproximate shift invariant. The main goal of PCA is to extract the most significant characteristics(represented by the wavelet coefficients) in order to improve the spatial resolution. The proposedalgorithm fuses the detailed wavelet coefficients of input images using features selection rule.Results: Several experiments have been conducted over different sets of multimodal medicalimages such as CT/MRI, MRA/T1-MRI. However, due to pages-limit on a paper, only results ofthree sets have been presented. The FSPCA-DTCWT algorithm is compared to recent fusionmethods presented in the literature (eight methods) in terms of visual quality and quantitativelyusing well-known fusion performance metrics (five metrics). Results showed that the proposedalgorithm outperforms the existing ones regarding visual and quantitative evaluations.Conclusion: This paper focuses on medical image fusion of different modalities. A novel imagefusion algorithm based on DTCWT to merge multimodal medical images has been proposed.Experiments have been performed using two different sets of multimodal medical images. Theresults show that the proposed fusion method significantly outperforms the recent fusiontechniques reported in the literature.

Medical image fusion plays an important role in clinical applications such as image-guided surgery, image-guided radiotherapy, noninvasive diagnosis, and treatment planning. In this paper, we propose a novel multi-modal medical image fusion method based on simplified pulse-coupled neural network and quaternion wavelet transform. The proposed fusion algorithm is capable of combining not only pairs of computed tomography (CT) and magnetic resonance (MR) images, but also pairs of CT and proton-density-weighted MR images, and multi-spectral MR images such as T1 and T2. Experiments on six pairs of multi-modal medical images are conducted to compare the proposed scheme with four existing methods. The performances of various methods are investigated using mutual information metrics and comprehensive fusion performance characterization (total fusion performance, fusion loss, and modified fusion artifacts criteria). The experimental results show that the proposed algorithm not only extracts more important visual information from source images, but also effectively avoids introducing artificial information into fused medical images. It significantly outperforms existing medical image fusion methods in terms of subjective performance and objective evaluation metrics.

DFENet: A dual-branch feature enhanced network integrating transformers and convolutional feature learning for multimodal medical image fusion

Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform

Multimodal medical image fusion, as a powerful tool for the clinical applications, has developed with the advent of various imaging modalities in medical imaging. The main motivation is to capture most relevant information from sources into a single output, which plays an important role in medical diagnosis. In this paper, a novel fusion framework is proposed for multimodal medical images based on Gradient Orientation Bracketed pairs. The source medical images are first transformed by Image separation (DOF) followed by combining background and foreground parts frequency components. The Standard Dynamic Range (SDR) device traces image details like contrasts and gradient filter direction in a series of SDR images with different levels. The image overall luminance levels are calculated with the image series, which maximizes the observable distinctions, and then the gradients embedded in these images. The fusion algorithm techniques are used for fusion of images based on Luminance and gradient level. This is done in a multi-resolution of illumination variation in the sequence. Experimental results and comparative study show that the proposed fusion framework provides an effective way to enable more accurate analysis of multimodality images.