Robust multimodal medical image registration and statistical cerebrovascular segmentation

Joint segmentation and registration of images is a focused area of research nowadays. Jointly segmenting and registering noisy images and images having weak boundaries/intensity inhomogeneity is a challenging task. In medical image processing, joint segmentation and registration are essential methods that aid in distinguishing structures and aligning images for precise diagnosis and therapy. However, these methods encounter challenges, such as computational complexity and sensitivity to variations in image quality, which may reduce their effectiveness in real-world applications. Another major issue is still attaining effective joint segmentation and registration in the presence of artifacts or anatomical deformations. In this paper, a new nonparametric joint model is proposed for the segmentation and registration of multi-modality images having weak boundaries/noise. For segmentation purposes, the model will be utilizing local binary fitting data term and for registration, it is utilizing conditional mutual information. For regularization of the model, we are using linear curvature. The new proposed model is more efficient to segmenting and registering multi-modality images having intensity inhomogeneity, noise and/or weak boundaries. The proposed model is also tested on the images obtained from the freely available CHOAS dataset and compare the results of the proposed model with the other existing models using statistical measures such as the Jaccard similarity index, relative reduction, Dice similarity coefficient and Hausdorff distance. It can be seen that the proposed model outperforms the other existing models in terms of quantitatively and qualitatively.

CoCycleReg: Collaborative cycle-consistency method for multi-modal medical image registration

Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients' data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients' data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework.

Multimodality image registration quality assurance for conformal three-dimensional treatment planning

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions – A comparative study on generalizability

BackgroundMultimodal histology image registration is a process that transforms into a common coordinate system two or more images obtained from different microscopy modalities. The combination of information from various modalities can contribute to a comprehensive understanding of tissue specimens, aiding in more accurate diagnoses, and improved research insights. Multimodal image registration in histology samples presents a significant challenge due to the inherent differences in characteristics and the need for tailored optimization algorithms for each modality.ResultsWe developed MMIR a cloud-based system for multimodal histological image registration, which consists of three main modules: a project manager, an algorithm manager, and an image visualization system.ConclusionOur software solution aims to simplify image registration tasks with a user-friendly approach. It facilitates effective algorithm management, responsive web interfaces, supports multi-resolution images, and facilitates batch image registration. Moreover, its adaptable architecture allows for the integration of custom algorithms, ensuring that it aligns with the specific requirements of each modality combination. Beyond image registration, our software enables the conversion of segmented annotations from one modality to another.

Image registration is an elementary task in medical image processing and analysis, which can be divided into monomodal and multimodal. Direct 3D multimodal registration in volumetric medical images can provide more insight into the interpretation of subsequent image processing applications than 2D methods. This paper is dedicated to the development of a 3D multimodal image registration algorithm based on a viscous fluid model associated with the Bhattacharyya distance. In our approach, a modified Navier-Stoke's equation is exploited as the foundation of the multimodal image registration framework. The hopscotch method is numerically implemented to solve the velocity field, whose values at the explicit locations are first computed and the values at the implicit positions are solved by transposition. The differential of the Bhattacharyya distance is incorporated into the body force function, which is the main driving force for deformation, to enable multimodal registration. A variety of simulated and real brain MR images were utilized to assess the proposed 3D multimodal image registration system. Preliminary experimental results indicated that our algorithm produced high registration accuracy in various registration scenarios and outperformed other competing methods in many multimodal image registration tasks.Clinical Relevance- This facilitates the disease diagnosis and treatment planning that requires accurate 3D multimodal image registration without massive image data and extensive training regardless of the imaging modality.

Similarity measure plays a critical role in image registration. Mutual information (MI) has been proved to be a promising measure used widely in multi-modality image registration. However, applying mutual information to original intensities only takes statistical information into consideration, while spatial information is not even considered. In this paper, a novel approach is proposed to incorporate spatial information into MI through gradient vector flow (GVF). Mutual information now is calculated from the GVF-intensity (GVFI) map of the original images instead of their intensity values. Multi-modality brain image registration was performed to test the accuracy and robustness of the proposed method. Experimental results showed that the success rate of our method is higher than that of traditional MI-based registration.

Medical image registration is an integral part of various clinical applications including image guidance, motion tracking, therapy assessment and diagnosis. We present a robust approach for mono-modal and multi-modal medical image registration. To this end, we propose the novel shape operator based local image distance (SOLID) which estimates the similarity of images by comparing their second-order curvature information. Our similarity metric is rigorously tailored to be suitable for comparing images from different medical imaging modalities or image contrasts. A critical element of our method is the extraction of local features using higher-order shape information, enabling the accurate identification and registration of smaller structures. In order to assess the efficacy of the proposed similarity metric, we have implemented a variational image registration algorithm that relies on the principle of matching the curvature information of the given images. The performance of the proposed algorithm has been evaluated against various alternative state-of-the-art variational registration algorithms. Our experiments involve mono-modal as well as multi-modal and cross-contrast co-registration tasks in a broad variety of anatomical regions. Compared to the evaluated alternative registration methods, the results indicate a very favorable accuracy, precision and robustness of the proposed SOLID method in various highly challenging registration tasks.

Deep learning based data-adaptive descriptor for non-rigid multi-modal medical image registration

Maximization of mutual information (MI) (MMI) has a wide application in multimodal image registration. The conventional MMI algorithm is based on the statistics derived from image sampling and does not use spatial features. In this paper, we proposed an improved MMI registration algorithm that combines the spatial information through a feature-based selection mechanism. A Harris corner filter is chosen to classify the pixels. The classification results are used to focus sample selection, joint entropy estimation, and weighted MI information calculation. Although features play an important role, conventional feature matching is not used, which removes an important challenge to the use of feature-based information. The advantages of the proposed algorithm are improved robustness by incorporation of spatial features and significant reduction in computing time for registration. Experiments with challenging synthetic images and multimodal airborne infrared images are provided to demonstrate the improvement.

The multiresolution approach is commonly used to speed up the mutual-information (MI) based registration process. Conventionally, a Gaussian pyramid is often used as a multiresolution representation. However, in multi-modal medical image registration, MI-based methods with Gaussian pyramid may suffer from the problem of short capture ranges especially at the lower resolution levels. This paper proposes a novel and straightforward multimodal image registration method based on wavelet representation, in which two matching criteria are used including sum of difference (SAD) for improving the registration robustness and MI for assuring the registration accuracy. Experimental results show that the proposed method obtains a longer capture range than the traditional MI-based Gaussian pyramid method meanwhile maintaining comparable accuracy.

Over the past few decades, fast<strong>-</strong>growth has occurred in the area of medical image acquisition devices, and physicians now rely on the utilization of medical images for the diagnosis, treatment plans, and surgical guidance. Researchers have classified medical images according to two structures: anatomical and functional structures. Due to this classification, the data obtained from two or more images of the same object frequently provide complementary and more abundant information through a process known as multimodal medical model registration. Image registration is spatially mapping the coordinate system of the two images obtained from a different viewpoint and utilizing various sensors. Several automatic multimodal medical image registration algorithms have been introduced based on types of medical images and their applications to increase the reliability, robustness, and accuracy. Due to the diversity in imaging and the different demands<strong> </strong>for applications, there is no single registration algorithm that can do that. This paper introduces a novel method for developing a multimodal medical image registration system that can select the most accepted registration algorithm from a group of registration algorithms for a variety of input datasets. The method described here is based on a machine learning technique that selects the most promising candidate. Several experiments have been conducted, and the results reveal that the novel approach leads to considerably faster reliability, accuracy, and more robustness registration algorithm selection.

RA-MMIR : Multi-modal image registration by Robust Adaptive Variation Attention Gauge Field

Objective To evaluate the image quality and registration accuracy of a three-dimensional (3D) dynamic phantom in four-dimensional computed tomography (4DCT) and four-dimensional cone-beam computed tomography (4DCBCT). Methods The Computerized Imaging Reference Systems Dynamic Thorax Phantom Model 008A was scanned to get 4DCT and 4DCBCT images. Two balls with different diameters (O= 1 cm and O= 2 cm) were used to simulate tumors with different sizes. The motion mode of the balls was 3D sinusoidal motion at 0.25 Hz (the amplitudes along the x, y, and z axes were ±1.0 cm, ±0.4 cm, and ±0.2 cm, respectively). Gross target volumes (GTVs) from 10-phase bins, internal gross target volumes (IGTV), and target volumes on maximum intensity projection (MIP) and mean intensity projection (MeanIP) images were contoured and calculated. Target volumes on 4DCT or 4DCBCT images were compared with the static and dynamic volumes of the balls (VS and VD). The matching index (MI) of target volumes between the 4DCT and 4DCBCT images was analyzed after rigid image registration. Results The GTV in each phase of the image was larger than VS. The difference between the average GTV derived from 10 phases of 4DCT or 4DCBCT images and Vs of the small ball was larger than that of the large ball (35.03% vs. 22.66%; 32.62% vs. 17.00%). All the IGTVs and target volumes on MIP images were slightly larger than VD, but target volumes on MeanIP images were smaller than VD. The average MI of 10-phase bins of the small ball was smaller than that of the large ball (66.76% vs. 82.21%). Moreover, MIs of IGTV, MIP, and MeanIP of the small ball were also smaller than those of the large ball (77.39% vs. 90.29%; 75.90% vs. 89.28%; 74.47% vs. 82.74%). Conclusions In the case of a relatively small tumor volume and a relatively large motion amplitude, 4DCT and 4DCBCT should be used with caution for comparison of image registration. Key words: Tomography, X-ray computed; Tomography, X-ray computed, cone-beam; Imaging registration; Phantom