SOLID: a novel similarity metric for mono-modal and multi-modal deformable image registration

Improvement of medical diagnosis, aided computer surgeries and tumor identification requires an accurate image registration approaches. The registration of multimodal medical images is more complicated than the registration of unimodal medical images due to the variation in luminance between the images. In this paper, an accurate multimodal image registration approach using artificial immune system (AIS) is proposed and the affine transformation model is used in contrast to the most of the related works which assumed rigid transformation model or similarity transformation model. In the proposed approach the LL bands of the discrete wavelet transform (DWT) for the images are used and the normalized mutual information (NMI) is used as a fitness function. The proposed approach achieves good result in the case of noiseless images, noisy images and partial data loss from one of the images. Moreover, the proposed approach does not need any feature extraction or refinement step. To demonstrate the robustness of the proposed approach, it has been compared with two multimodal medical image registration approaches.

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions – A comparative study on generalizability

Over the past few decades, fast<strong>-</strong>growth has occurred in the area of medical image acquisition devices, and physicians now rely on the utilization of medical images for the diagnosis, treatment plans, and surgical guidance. Researchers have classified medical images according to two structures: anatomical and functional structures. Due to this classification, the data obtained from two or more images of the same object frequently provide complementary and more abundant information through a process known as multimodal medical model registration. Image registration is spatially mapping the coordinate system of the two images obtained from a different viewpoint and utilizing various sensors. Several automatic multimodal medical image registration algorithms have been introduced based on types of medical images and their applications to increase the reliability, robustness, and accuracy. Due to the diversity in imaging and the different demands<strong> </strong>for applications, there is no single registration algorithm that can do that. This paper introduces a novel method for developing a multimodal medical image registration system that can select the most accepted registration algorithm from a group of registration algorithms for a variety of input datasets. The method described here is based on a machine learning technique that selects the most promising candidate. Several experiments have been conducted, and the results reveal that the novel approach leads to considerably faster reliability, accuracy, and more robustness registration algorithm selection.

Deep learning based data-adaptive descriptor for non-rigid multi-modal medical image registration

Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients' data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients' data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework.

A relatively large class of information theoretical measures, including e.g. mutual information or normalized entropy, has been used in multi-modal medical image registration. Even though the mathematical foundations of the different measures were very similar, the final expressions turned out to be surprisingly different. Therefore one of the main aims of this paper is to enlight the relationship of different objective functions by introducing a mathematical framework from which several known optimization objectives can be deduced. Furthermore we extend existing measures in order to be applicable on image features different than image intensities and introduce efficiency as a very general concept to qualify such features. The presented framework is very general and not at all restricted to medical images. Still we want to discuss the possible impact of our theoretical framework for the particular problem of medical image registration, where the feature space has traditionally been fixed to image intensities. Our theoretical approach is very general though and can be used for any kind of multi-modal signals, such as for the broad field of multi-media applications.

Analysis of multispectral or multitemporal images requires proper geometric alignment of the images to compare corresponding regions in each image volume. Retrospective three-dimensional alignment or registration of multimodal medical images based on features intrinsic to the image data itself is complicated by their different photometric properties, by the complexity of the anatomical objects in the scene and by the large variety of clinical applications in which registration is involved. While the accuracy of registration approaches based on matching of anatomical landmarks or object surfaces suffers from segmentation errors, voxel-based approaches consider all voxels in the image without the need for segmentation. The recent introduction of the criterion of maximization of mutual information, a basic concept from information theory, has proven to be a breakthrough in the field. While solutions for intrapatient affine. registration based on this concept are already commercially available, current research in the field focuses on interpatient nonrigid matching.

Joint segmentation and registration of images is a focused area of research nowadays. Jointly segmenting and registering noisy images and images having weak boundaries/intensity inhomogeneity is a challenging task. In medical image processing, joint segmentation and registration are essential methods that aid in distinguishing structures and aligning images for precise diagnosis and therapy. However, these methods encounter challenges, such as computational complexity and sensitivity to variations in image quality, which may reduce their effectiveness in real-world applications. Another major issue is still attaining effective joint segmentation and registration in the presence of artifacts or anatomical deformations. In this paper, a new nonparametric joint model is proposed for the segmentation and registration of multi-modality images having weak boundaries/noise. For segmentation purposes, the model will be utilizing local binary fitting data term and for registration, it is utilizing conditional mutual information. For regularization of the model, we are using linear curvature. The new proposed model is more efficient to segmenting and registering multi-modality images having intensity inhomogeneity, noise and/or weak boundaries. The proposed model is also tested on the images obtained from the freely available CHOAS dataset and compare the results of the proposed model with the other existing models using statistical measures such as the Jaccard similarity index, relative reduction, Dice similarity coefficient and Hausdorff distance. It can be seen that the proposed model outperforms the other existing models in terms of quantitatively and qualitatively.

In this thesis, we focus on two important medical image processing and analysis problems: multimodal image registration and cerebrovascular segmentation. First, for the purpose of improving the robustness of image registration, we propose two novel registration objective functions and a new technique for better optimization performance. The first novel objective function is a combination of the multi-dimensional mutual information and an angle measure on the maximum distance-gradient (MDG) vector field. The MDG vector field is a new spatial feature field designed for registration tasks. It encodes both local edge information and globally defined spatial information related to the intensity difference, the distance, and the direction of a voxel to a certain object boundary point. This objective function then integrates both the magnitude and the orientation information of the MDG feature into the image registration process. The other novel objective function can be obtained if the statistical joint intensity mappings estimated from the pre-aligned training image pairs are available. This a priori knowledge can then be used as reference to guide the multi-resolution based registration process, by minimizing a new Kullback-Leibler distance (KLD) based similarity measure between the observed joint intensity distribution and the reference joint intensity distribution. To achieve better optimization, which evidently affects the robustness of a registration process, we propose a general technique based on random transformation disturbance to assist optimization procedures to avoid local optima, and hence to achieve robust optimization results. The technique is based on different properties of the local neighborhood of a global optimum and those of local optima. Second, this thesis presents a statistical segmentation technique for the extraction of vasculatures in three-dimensional rotational angiography (3D-RA). This method uses maximum intensity projections (MIP) to improve the accuracy and robustness of threshold estimation using the expectation maximization (EM) algorithm. It is fully automatic and computationally efficient. Finally, encouraging experimental results of the proposed approaches on both synthetic and clinical medical images are presented in the thesis.

CoCycleReg: Collaborative cycle-consistency method for multi-modal medical image registration

The multiresolution approach is commonly used to speed up the mutual-information (MI) based registration process. Conventionally, a Gaussian pyramid is often used as a multiresolution representation. However, in multi-modal medical image registration, MI-based methods with Gaussian pyramid may suffer from the problem of short capture ranges especially at the lower resolution levels. This paper proposes a novel and straightforward multimodal image registration method based on wavelet representation, in which two matching criteria are used including sum of difference (SAD) for improving the registration robustness and MI for assuring the registration accuracy. Experimental results show that the proposed method obtains a longer capture range than the traditional MI-based Gaussian pyramid method meanwhile maintaining comparable accuracy.

In this research paper, authors propose multimodal brain image registration using discrete wavelet transform(DWT) followed by Gaussian pyramids. The reference and target images are decomposed into their LL, LH, HL and LL DWT coefficients and then are processed for image registration using Gaussian pyramids. The image registration is also done using Gaussian pyramids only and wavelets transforms only for comparison. The quality of registration is measured by comparing the maximum MI values used by the three methods and also by comparing their correlation coefficients. Our proposed technique proves to show better results when compared with the other two methods.

In this paper, we introduce SAMmorph, a novel multimodal medical image registration framework addressing challenges in aligning cross‐modality images due to significant appearance differences and heavy reliance on expensive annotations. It comprises three key components: the VoxelSAM module for generating pseudo‐labels, the FusionLayer which takes moving/fixed images and their pseudo‐labels as input, integrates them via two optional schemes (stack: concatenating along the channel dimension to form a 4‐channel tensor; enhance: modulating image intensity with mask confidence to generate a 3‐channel tensor) to produce combined features for registration, and the deformable registration network which includes two architectures (joint and independent registration networks) to implement the bidirectional registration strategy. Experiments demonstrate that SAMmorph outperforms state‐of‐the‐art methods across various metrics on the ‐RegPro dataset, achieving a dice similarity coefficient (DSC) of 0.906, target registration error (TRE) of 6.06 mm, compared to those being 0.789 and 7.632 mm achieved by the strongest tested baseline method CorrMLP, with statistical significance confirmed by Wilcoxon signed‐rank tests (). This work presents a promising approach for improving medical image registration accuracy and efficiency, with potential applications in diverse medical imaging scenarios. The source codes have been released at: https://github.com/qilin‐world/SAMmorph .

Registration of multi-modal medical images is an essential pre-processing step, for example, for fusion or image guided-interventions. However, the alignment process is prone to high variability in tissue appearance between modalities, in addition to local intensity variations and artefacts. This work introduces a robust multi-modal registration approach that mitigates the undesirable effect of such variability. Robustness is achieved using Huber’s loss function for the data fidelity and regularization terms. We propose a novel approach using Huber’s criterion, which enables a jointly convex estimation of the motions and the associated scale parameters. We formulate the problem as a complex 2D transformation estimation and investigate a robust total-variation smoothing, as well as a dictionary learning-based data fidelity term. Experiments are conducted using two datasets of multi-contrast MR brain images.

Multimodality image registration using local linear embedding and hybrid entropy