Symmetric transformer-based network for unsupervised image registration

Nonrigid multimodal image registration remains a challenging task in medical image processing and analysis. The structural representation (SR)-based registration methods have attracted much attention recently. However, the existing SR methods cannot provide satisfactory registration accuracy due to the utilization of hand-designed features for structural representation. To address this problem, the structural representation method based on the improved version of the simple deep learning network named PCANet is proposed for medical image registration. In the proposed method, PCANet is firstly trained on numerous medical images to learn convolution kernels for this network. Then, a pair of input medical images to be registered is processed by the learned PCANet. The features extracted by various layers in the PCANet are fused to produce multilevel features. The structural representation images are constructed for two input images based on nonlinear transformation of these multilevel features. The Euclidean distance between structural representation images is calculated and used as the similarity metrics. The objective function defined by the similarity metrics is optimized by L-BFGS method to obtain parameters of the free-form deformation (FFD) model. Extensive experiments on simulated and real multimodal image datasets show that compared with the state-of-the-art registration methods, such as modality-independent neighborhood descriptor (MIND), normalized mutual information (NMI), Weber local descriptor (WLD), and the sum of squared differences on entropy images (ESSD), the proposed method provides better registration performance in terms of target registration error (TRE) and subjective human vision.

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions – A comparative study on generalizability

Brain tumor detection and medical image registration are critical tasks in neuroimaging, enabling accurate diagnosis, treatment planning, and follow-up monitoring. Magnetic Resonance Imaging (MRI) remains the preferred modality for visualizing tumor morphology, while image registration facilitates the alignment of multimodal scans for enhanced clinical interpretation. This survey reviews recent advances in deep learning-based methods for brain tumor classification and medical image registration, focusing on Convolutional Neural Networks (CNNs), transfer learning architectures (e.g., VGG-19, ResNet, DenseNet201), and modern registration frameworks including supervised, unsupervised, GAN-based, and reinforcement learning approaches. A comprehensive analysis of 50+ peer-reviewed studies highlights state-of-the-art performance, with brain tumor classification achieving up to 97.5% precision (CNN + FCM) and 96.97% accuracy (DWT+PCA+DNN), while multimodal MRI registration methods report Dice Similarity Coefficient (DSC) values exceeding 0.90 and reduced Target Registration Error (TRE) compared to traditional deformable registration techniques. Trends indicate a shift toward hybrid models combining segmentation and registration, domain adaptation for cross-dataset generalization, and lightweight frameworks for real-time clinical deployment. In conclusion, deep learning approaches have significantly advanced the accuracy, efficiency, and automation of brain tumor detection and medical image registration. However, challenges such as limited annotated multimodal datasets, robust similarity measures for heterogeneous modalities, and the integration of clinically interpretable AI models remain open for future research.

Deep learning based data-adaptive descriptor for non-rigid multi-modal medical image registration

3D medical image registration is of great clinical importance. However, supervised learning methods require a large amount of accurately annotated corresponding control points (or morphing), which are very difficult to obtain. Unsupervised learning methods ease the burden of manual annotation by exploiting unlabeled data without supervision. In this article, we propose a new unsupervised learning method using convolutional neural networks under an end-to-end framework, Volume Tweening Network (VTN), for 3D medical image registration. We propose three innovative technical components: (1)An end-to-end cascading scheme that resolves large displacement; (2)An efficient integration of affine registration network; and (3)An additional invertibility loss that encourages backward consistency. Experiments demonstrate that our algorithm is 880x faster (or 3.3x faster without GPU acceleration) than traditional optimization-based methods and achieves state-of-the-art performance in medical image registration.

Medical image segmentation and registration are two tasks to analyze the anatomical structures in clinical research. Still, deep-learning solutions utilizing the connections between segmentation and registration remain underdiscovered. This article designs a joint learning framework named RSegNet that can realize concurrent deformable registration and segmentation by minimizing an integrated loss function, including three parts: diffeomorphic registration loss, segmentation similarity loss, and dual-consistency supervision loss. The probabilistic diffeomorphic registration branch could benefit from the auxiliary segmentations available from the segmentation branch to achieve anatomical consistency and better deformation regularity by dual-consistency supervision. Simultaneously, the segmentation performance could also be improved by data augmentation based on the registration with well-behaved diffeomorphic guarantees. Experiments on the human brain 3-D magnetic resonance images have been implemented to demonstrate the effectiveness of our approach. We trained and validated RSegNet with 1000 images and tested its performances on four public datasets, which shows that our method successfully yields concurrent improvements of both segmentation and registration compared with separately trained networks. Specifically, our method can increase the accuracy of segmentation and registration by 7.0% and 1.4%, respectively, in terms of Dice scores. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Registration and segmentation of medical images are two significant tasks in medical research and clinical application. However, most existing approaches consider these two tasks independently while neglecting the potential association between them. Therefore, we suggest a new approach that combines these two tasks into one joint deep learning framework, boosting registration, and segmentation performance by introducing dual-consistency supervision. Besides, our framework could generate outputs within 1 s by taking an affinely aligned medical image pair as input, which is suitable for time-critical requirements in a clinic. We tested it on four public datasets and achieved state-of-the-art performance to demonstrate the proposed method’s feasibility and robustness. Furthermore, our proposed RSegNet is a general learning framework suitable for various image modalities and anatomical structures. Hence, we expect our framework to serve as a practical clinical tool to speed up medical image analysis procedures and improve diagnostic accuracy.

Medical image registration is crucial in medical image analysis applications. Recently, U-Net-style networks have been commonly used for unsupervised image registration, predicting dense displacement fields in full-resolution space. However, this process is resource-intensive and time-consuming for high-resolution volumetric image data. To address this challenge, this paper proposes a novel model named RegFSC-Net, which utilizes Fourier transform with spatial reorganization (SR) and channel refinement (CR) network for registration. We embed efficient feature extraction modules SR and CR modules into the encoder, and adopt a parameter-free model to drive the decoder to improve the U-shaped network. Precisely, RegFSC-Net does not directly predict the full-resolution displacement field in space but learns the low-dimensional representation of the displacement field in the bandlimited Fourier domain, which is beneficial in reducing network parameters, memory usage, and computational costs. Experimental results show that RegFSC-Net outperforms various state-of-the-art methods. Specifically, in comparison to the widely recognized Transformer-based method TransMorph, RegFSC-Net utilizes only around 8.2% of its parameters, resulting in a 1.95% higher Dice score and significantly faster inference speeds of 126.67% and 419.99% on GPU and CPU, respectively. Furthermore, we also designed three variants of RegFSC-Net and demonstrated their potential applications in computer-aided diagnosis.

Medical image registration is essential and a key step in many advanced medical image tasks. In recent years, medical image registration has been applied to many clinical diagnoses, but large deformation registration is still a challenge. Deep learning‐based methods typically have higher accuracy but do not involve spatial transformation, which ignores some desirable properties, including topology preservation and the invertibility of transformation, for medical imaging studies. On the other hand, diffeomorphic registration methods achieve a differentiable spatial transformation, which guarantees topology preservation and invertibility of transformation, but registration accuracy is low. Therefore, a diffeomorphic deformation registration with CNN is proposed, based on a symmetric architecture, simultaneously estimating forward and inverse deformation fields. CNN with Efficient Channel Attention is used to better capture the spatial relationship. Deformation fields are optimized explicitly and implicitly to enhance the invertibility of transformations. An extensive experimental evaluation is performed using two 3D datasets. The proposed method is compared with different state‐of‐the‐art methods. The experimental results show excellent registration accuracy while better guaranteeing the diffeomorphic transformation.

Although a considerable amount of research has been devoted to the registration of medical images, a good proportion of this has been performed on rigid regions of the anatomy. This has tended to limit the practical value of such research. The registration of medical images which are non-rigid in nature is a seemingly difficult and not fully investigated problem. This paper presents an elastic transformation method which is used to perform matching of non-rigid medical images, that is, two-dimensional images of a region of the body such as the thorax or abdomen which exhibits a high level of elasticity. Some forms of elastic transformation may be more localized in scope than others, which leads to the categorization of elastic transformation methods into local and semi-local methods. This paper demonstrates the use of local elastic spatial transformations in matching non-rigid medical images. These transformations are illustrated by matching corresponding computed tomography and magnetic resonance images of the thorax.

Segmentation is one of the important and challenging tasks in medical image analysis. Region of Interest (RoI) needs to be segmented from the background in almost all types of medical images for further analysis. This assist doctors in perfect diagnosis of diseases. In this paper convolutional neural network (CNN) has been used for automatic segmentation of RoI in medical imaging modalities such as carotid artery ultrasound images. The results obtained through proposed CNN are compared with other machine learning algorithm such as support vector machine (SVM) and radial basis function (RBF). CNN exhibits an edge in performance with 10-fold cross validation over the other two networks with an accuracy of 99.3%. The results are also compared with other standard CNN architectures.

The image navigation and registration (INR) performance assessment for the Geostationary Operational Environmental Satellite-R (GOES-R) Advanced Baseline Imager (ABI) is an image registration process at the subpixel level, which involves the registration of images from GOES-R ABI with corresponding images from Landsat at the same scene. We present an optimized image registration algorithm for GOES-R ABI INR performance assessment (GAIPA): a gradient descent algorithm with a Sobel edge-enhancement procedure and a normalized lookup table (NLUT) image transformation (GDSN), which is an area-based image registration approach with Pearson correlation as the similarity metric. Sobel edge enhancements increase the sensitivity of Pearson correlations that lead to much improved registration accuracy. NLUT transformations minimize radiance differences between GOES-R ABI images acquired at different times and Landsat reference images on image registration outcomes, which lead to improvements in correlation values and quality of image registration outputs. Evaluations of registration accuracy and quality metrics in GDSN show considerable improvements in computational efficiency, registration accuracy, and quality over the approach implemented on the image INR performance assessment tool set. Pearson correlations can be effective and robust similarity metrics in image registration at the subpixel level with implementing Sobel edge enhancement and NLUT transformation. The statistical analysis of the image registration outputs for GAIPA shows the new algorithm provides considerable improvements in reducing the uncertainties in the registration outputs.

Automatic image registration is a fundamental task in medical image processing, and significant advances have occurred in the last decade. However, one major problem with advanced registration techniques is their high computational cost. Due to this restraint, these methods have found limited application to clinical situations where real time or near real time execution is required, e.g., intraoperative imaging, or high volumes of data need to be processed periodically. High performance in image registration can be achieved by reduction in data and search spaces. However, to obtain a significant increase in performance, these approaches must be complemented with parallel processing. Parallel processing is associated with expensive supercomputers and computer clusters that are unaffordable for most public medical institutions. This chapter will describe how to take advantage of an existing computational infrastructure and achieve high performance image registration in a practical and affordable way. More specifically, it will outline the implementation of a fast and robust Internet subtraction service, using a distributed evolutionary algorithm and a service-oriented architecture.KeywordsEvolutionary AlgorithmDifferential EvolutionImage RegistrationProjective TransformationJoint Probability Density FunctionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Mutual information (MI) is a very important concept in medical image registration for a diagnosis. During the past decade, there have been a large number of studies on multimodal image registration. This extensive attention is due to its high performance in terms of accuracy and robustness. However, there are so many different perspectives on MI that have been discussed and proposed during the last two decades. As a result, it is very difficult to have a holistic view of the field and to select an appropriate method to solve a specific problem in medical image analysis. In order to solve this problem, this paper proposes a novel taxonomy of MI approaches used in medical image registration. In our work, we extensively and comprehensively investigate works proposed in the literature and systematically classify them based on methods of computational processing. Our taxonomy will help to enhance an understanding of various principles of mutual information.

Image registration is the process of aligning two or more images in a single coordinate. Nowadays, medical image registration plays a significant role in computer-assisted disease diagnosis, treatment, and surgery. The different modalities available in the medical image make medical image registration an essential step in Computer Assisted Diagnosis (CAD), Computer- Aided Therapy (CAT) and Computer-Assisted Surgery (CAS). Recently, many learning-based methods were employed for disease detection and classification, but those methods were not suitable for real-time due to delayed response and the need for pre-alignment and labeling. The proposed research constructed a deep learning model with Rigid transform and B-Spline transform for medical image registration for an automatic brain tumour finding. The proposed research consists of two steps. The first step uses Rigid transformation based Convolutional Neural Network and the second step uses B-Spline transform-based Convolutional Neural Network. The model is trained and tested with 3624 MR (Magnetic Resonance) images to assess the performance. The researchers believe that MR images help in the success of the treatment of patients with brain tumour. The result of the proposed method is compared with the Rigid Convolutional Neural Network (CNN), Rigid CNN + Thin-Plat Spline (TPS), Affine CNN, Voxel morph, ADMIR (Affine and Deformable Medical Image Registration) and ANT(Advanced Normalization Tools) using DICE score, Average Symmetric surface Distance (ASD), and Hausdorff distance. The RBCNN model will help the physician to automatically detect and classify the brain tumor quickly (18 Sec) and efficiently without doing pre-alignment and labeling.

Deformable image registration is a basic image processing task, especially widely used in medical image processing and analysis. Different from rigid registration, its purpose is to find the optimal nonlinear transformation between two images and establish corresponding relationship, so as to achieve image consistency. In recent years, deformable registration methods based on deep learning have been studied a lot. Compared with traditional methods, they show great advantages in registration performance. This paper proposes an attention-based residual neural network for deformable image registration, which utilizes the U-Net encoder-decoder structure to design a convolutional neural network to predict the deformation field, and uses the residual module and attention mechanism enhances the ability of the model to extract features, and finally uses the spatial transformation function to obtain the registered image, and the entire network is trained in an unsupervised manner. We conducted experiments on the MNIST dataset and 2D brain magnetic resonance images (MRI) respectively. The experimental results show that the deformable registration network proposed in this paper has good performance and shows good results in registration accuracy.