Gradient based image registration using importance sampling

Background: Non-rigid registration of medical images with intensity distortions is a difficult problem due to the change in pixel intensity. It is caused by contrast agent or intensity bias field. Methods: In some cases, this problem can be solved using Residual Complexity (RC) method. However, relative modification of parameter in residual complexity would result in completely different experimental effect. Another drawback is sensitivity to noise. To handle this problem, a new intensity-based similarity measure, Weighted Residual Complexity (WRC) has been proposed for effective medical image registration in this paper. Specifically, the local entropy image of two images is computed to be aligned respectively. Then, a weighting function using a function of the local entropy difference is modeled. The weighting function is used to weight the residual image in residual complexity adaptively. The residual image is defined as the difference between reference image and warped floating image. Results: The weighting function assigns smaller weight to residual image if the corresponding pixel value is larger in local entropy difference. The proposed technique was applied to simulative and real medical images. The contrast experiments were made with mutual information, diffeomorphic demons and residual complexity. Conclusion: Also, the analysis of experimental results was made qualitatively and quantitatively, which indicates that this new approach gives a better performance than diffeomorphic demons, mutual information and residual complexity. Keywords: Weighting function, local entropy, residual complexity, intensity distortion, image registration, WRC.

In this article, we combined intensity- and feature-based similarity measures to co-register very-high-resolution (VHR) optical and SAR images. The global translation difference between optical and SAR images is minimized by applying a mutual information (MI) intensity-based similarity measure from coarsest to finest pyramid images constructed from the images in order. Matching points are then extracted considering the spatial distance and gradient orientation of linear features extracted from each image. To increase the reliability of the registration result, a quadtree-based structure is constructed (1) to mask out regions from the similarity measurement such as dense urban or heterogeneous areas, which can cause large differences in geometric and radiometric properties in two images; and (2) to extract evenly distributed and precise matching points by considering regional properties of the study site. To evaluate the proposed method’s generalization, various VHR optical and SAR sensors are used and compounded to construct the study sites. Evenly distributed matching points across the whole image were extracted, and reliable registration results by a non-linear transformation constructed from the points were derived from the proposed method.

Similarity measure is an important key in image registration. Most traditional intensity-based similarity measures (e.g. sum-of-squared-differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non-stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this assumption, a novel similarity measure is proposed here based on sparse representation in a mono-modal setting. The zero norm (l 0 ) in the transform domain is introduced as a new similarity measure in the presence of non-stationary intensity distortion. The present study defines a sparsity similarity measure that indicates the complexity of the residual image between two registered images in the transform domain such as discrete cosine transform or wavelet. It is attempted here to analytically and statistically illustrate that the proposed similarity measure has important properties such as metric properties in vector space, from correntropy perspective. This measure produces accurate registration results on both artificial and real-world problems which are examined in the present study.

Mutual information is a popular intensity-based image similarity measure mainly used in image registration. This measure has been also very successful as the similarity metric in our knowledge-based computer-assisted detection (CADe) system for the detection of masses in screening mammograms. Our CADe system is designed to assess a new, query case based on its similarity with known cases stored in the knowledge database. However, intensity-based mutual information captures only relationships between the gray level values of corresponding pixels. This study presents a novel advancement of our CADe system by incorporating neighborhood textural information when estimating the mutual information of two images. Specifically, an entropy filter is applied to the images, effectively replacing each image pixel value with its neighborhood entropy. This pixel-based entropy is a localized measure of image texture. Then, the information-theoretic CAD system is asked to make a decision regarding the query case using the texture-based mutual information similarity metric. The entropy-based image enhancement and MI-based decision making processes are repeated at different neighborhood scales. Finally, an artificial network merges intensity-based and texture-based decisions to investigate possible improvements in mass detection performance. Given a database of 1,820 regions of interest (ROIs) extracted from screening mammograms (901 depicting a biopsy-proven mass and 919 depicting normal parenchyma) and a leave-one out sampling scheme, the study showed that our CADe system achieves an ROC area of 0.87±0.01 using the intensity-based ROC. The ROC performance for the texture-based CADe system ranges from 0.69±0.01 to 0.83±0.01 depending on the scale of analysis. The synergistic approach of the ANN using both intensity-based and texture-based information resulted in statistically significantly better performance with an ROC area index of 0.93±0.01.

2D/3D image registration is one of the key technologies to realize pose estimation in computer-aided surgery. In order to improve the global and local search performance of the model in the pose parameter space, an improved multi-resolution 2D/3D registration method is proposed in this paper. Firstly, aiming at the problem that the intensity-based similarity measure is not sensitive to small offset between images, the gradient information with strong sensitivity to image texture edge is introduced, and the Intensity and Gradient Weighted Correlation (IGWC) coefficient similarity measure is proposed; Secondly, aiming at the problem of slow convergence of global optimization algorithm and small capture range of local optimization algorithm, a global-local combined registration optimization strategy is proposed. The experimental results show that this method improves the registration accuracy and success rate.

Intensity-based similarity measures assume that the original signal intensity of different modality images can provide statistically consistent information regarding the two modalities to be co-registered. In multi-modal registration problems, however, intensity-based similarity measures are often inadequate to identify an optimal transformation. Texture features can improve the performance of the multi-modal co-registration by providing more similar appearance representations of the two images to be co-registered, compared to the signal intensity representations. Furthermore, texture features extracted at different length scales (neighborhood sizes) can reveal similar underlying structural attributes between the images to be co-registered similarities that may not be discernible on the signal intensity representation alone. However one limitation of using texture features is that a number of them may be redundant and dependent and hence there is a need to identify non-redundant representations. Additionally it is not clear which features at which specific scales reveal similar attributes across the images to be co-registered. To address this problem, we introduced a novel approach for multimodal co-registration that employs new multi-scale image representations. Our approach comprises 4 distinct steps: (1) texure feature extraction at each length scale within both the target and template images, (2) independent component analysis (ICA) at each texture feature length scale, and (3) spectrally embedding (SE) the ICA components (ICs) obtained for the texture features at each length scale, and finally (4) identifying and combining the optimal length scales at which to perform the co-registration. To combine and co-register across different length scales, -mutual information (-MI) was applied in the high dimensional space of spectral embedding vectors to facilitate co-registration. To validate our multi-scale co-registration approach, we aligned 45 pairs of prostate MRI and histology images corresponding to the prostate with the objective of mapping extent of prostate cancer annotated by a pathologist on the pathology onto the pre-operative MRI. The registration results showed higher correlation between template and target images with average correlation ratio of 0.927 compared to 0.914 for intensity-based registration. Additionally an improvement in the dice similarity coefficient (DSC) of 13.6% was observed for the multi-scale registration compared to intensity-based registration and an 1.26% DSC improvement compared to registration involving the best individual scale.

Nonrigid registration method for longitudinal chest CT images in COVID-19

Mutual Information (MI) has been extensively used as a similarity measure in image registration and motion estimation, and it is particularly robust for 3D multimodal medical image registration. However, MI estimators are known i) to have a high variance and ii) to be computationally costly. In order to overcome these drawbacks, we propose a new similarity measure based on an Edgeworth-based third order expansion of MI and named 3-EMI in the following. This kind of approximation is well known in signal processing, and especially in Independent Components Analysis (ICA), but its computation is easier since data can be prewhitened contrary to images in registration. The performance of affine and non-rigid registrations based on the 3-EMI metric is studied through computer results in the context of cardiac multislice computed tomography. In fact, an estimate of the 3-EMI metric using sample statistics is compared to a histogram-based estimate of the standard normalized MI, showing a better robustness of the 3-EMI measure with respect to the range of the searched deformation. In addition, in practice, one part of the floating image may be missing regarding the reference image. Computer results show that our approach is less sensitive to such a practical problem.

We describe a new MR-based attenuation correction (MRAC) method for neurological studies performed using integrated PET/MR scanners. The method, combining the advantages of image segmentation and atlas-based approaches to generate a high-resolution template, is based on the widely available SPM8 software and provides robust and accurate linear attenuation coefficients (LACs) for head while requiring minimal user interaction. Atlas generation: 3T MR and CT images from 15 glioblastoma subjects were used to generate the high-resolution atlas. MR images were segmented into 6 tissue classes: GM, WM, CSF, soft tissue, bone and air)[1]. Tissue classes were then coregistered using an iterative diffeomorphic image registration algorithm [2] to form the template. Atlas validation: The template was validated on 16 subjects. SyN [3] and IRTK [4], considered state-of-the-art for non-rigid image registration[5], were used for comparison. Final attenuation maps were created from the warped CT atlas following [6]. PET images were then reconstructed using the proposed methods as well as the manufacturer’s built-in method (dual-echo Dixon-VIBE sequence) [7] and compared to the gold standard CT-based attenuation correction (CTAC). The qualitative and quantitative analysis of the attenuation maps revealed that the SPM8-based method produces very robust results (Figure ​(Figure1).1). In terms of the PET data quantification, we observed improvements of > 70% compared to the VIBE-based method (Table ​(Table11 and Figure ​Figure2).2). When compared to SyN-based image registration, the SPM8 approach showed improved global results on the brain area (Figures ​(Figures11 and ​and22). Figure 1 Comparison of LACs from a validation subject for our proposed method (A), the SyN method (B) and the manufacturer’s built-in Dixon method (C) to the gold standard CTAC (D). Image differences with respect to the gold standard CTAC of our method ... Table 1 Summary of voxel- and ROI-based results between our method (atlas) and the current manufacturer’s method (Dixon) Figure 2 PET images from a validation subject reconstructed with our proposed method (A), with the SyN method (B) and with the manufacturer’s built-in Dixon method (C), compared with the gold standard CTAC (D). Relative changes (in % with respect to gold ... We presented a new MRAC technique for brain images acquired on simultaneous PET/MR scanners. The new approach relies on segmentation- and atlas-based features to provide robust and more accurate LACs than using state-of-art non-rigid image registration while avoiding sophisticated user input or interaction.

We present a novel framework for describing intensity-based multi-modal similarity measures. Our framework is based around a concept of internal, or self, similarity. Firstly the locations of multiple regions or patches which are "similar" to each other are identified within a single image. The term "similar" is used here to represent a generic intra-modal similarity measure. Then if we examine a second image in the same locations, and this image is registered to the first image, we should find that the patches in these locations are also "similar", though the actual features in the patches when compared between the images could be very different. We propose that a measure based on this principle could be used as an inter-modal similarity measure because, as the two images become increasingly misregistered then the patches within the second image should become increasingly dissimilar. Therefore, our framework results in an inter-modal similarity measure by using two intra-modal similarity measures applied separately within each image. In this paper we describe how popular multi-modal similarity measures such as mutual information can be described within this framework. In addition the framework has the potential to allow the formation of novel similarity measures which can register using regional information, rather than individual pixel/voxel intensities. An example similarity measure is produced and its ability to guide a registration algorithm is investigated. Registration experiments are carried out using three datasets. The pairs of images to be registered were specifically chosen as they were expected to challenge (i.e. result in misregistrations) standard intensity-based measures, such as mutual information. The images include synthetic data, cadaver data and clinical data and cover a range of modalities. Our experiments show that our proposed measure is able to achieve accurate registrations where standard intensity-based measures, such as mutual information, fail.

Multimodal image registration is a difficult task, due to the significant intensity variations between the images. A common approach is to use sophisticated similarity measures, such as mutual information, that are robust to those intensity variations. However, these similarity measures are computationally expensive and, moreover, often fail to capture the geometry and the associated dynamics linked with the images. Another approach is the transformation of the images into a common space where modalities can be directly compared. Within this approach, we propose to register multimodal images by using diffusion maps to describe the geometric and spectral properties of the data. Through diffusion maps, the multimodal data is transformed into a new set of canonical coordinates that reflect its geometry uniformly across modalities, so that meaningful correspondences can be established between them. Images in this new representation can then be registered using a simple Euclidean distance as a similarity measure. Registration accuracy was evaluated on both real and simulated brain images with known ground-truth for both rigid and non-rigid registration. Results showed that the proposed approach achieved higher accuracy than the conventional approach using mutual information.

Subpixel accuracy image registration is needed for applications such as digital elevation model extraction, change detection, pan-sharpening, and data fusion. In order to achieve this accuracy, the deformation between the two images to be registered is usually modeled by a displacement vector field which can be estimated by measuring rigid local shifts for each pixel in the image. In order to measure subpixel shifts, one uses image resampling. Sampling theory says that, if a continuous signal has been sampled according to the Nyquist criterion, a perfect continuous reconstruction can be obtained from the sampled version. Therefore, a shifted version of a sampled signal can be obtained by interpolation and resampling with a shifted origin. Since only a sampled version of the shifted signal is needed, the reconstruction needs only to be performed for the new positions of the samples, so the whole procedure comes to computing the value of the signal for the new sample positions. In the case of image registration, the similarity between the reference image and the shifted versions of the image to be registered is measured, assuming that the maximum of similarity determines the most likely shift. The image interpolation step is thus performed a high number of times during the similarity optimization procedure. In order to reduce the computation cost, approximate interpolations are performed. Approximate interpolators will introduce errors in the resampled image which may induce errors in the similarity measure and therefore produce errors in the estimated shifts. In this paper, it is shown that the interpolation has a smoothing effect which depends of the applied shift. This means that, in the case of noisy images, the interpolation has a denoising effect, and therefore, it increases the quality of the similarity estimation. Since this blurring is not the same for every shift, the similarity may be low for a null shift (no blurring) and higher for shifts close to half a pixel (strong blurring). This paper presents an analysis of the behavior of the different interpolators and their effects on the similarity measures. This analysis will be done for the two similarity measures: the correlation coefficient and the mutual information. Finally, a strategy to attenuate the interpolation artifacts is proposed

We present a new strategy to constrain nonrigid registrations of multi-modal images using a low-dimensional statistical deformation model and test this in registering pre-operative and post-operative images from epilepsy patients. For those patients who may undergo surgical resection for treatment, the current gold-standard to identify regions of seizure involves craniotomy and implantation of intracranial electrodes. To guide surgical resection, surgeons utilize pre-op anatomical and functional MR images in conjunction with post-electrode implantation MR and CT images. The electrode positions from the CT image need to be registered to pre-op functional and structural MR images. The post-op MRI serves as an intermediate registration step between the pre-op MR and CT images. In this work, we propose to bypass the post-op MR image registration step and directly register the pre-op MR and post-op CT images using a low-dimensional nonrigid registration that captures the gross deformation after electrode implantation. We learn the nonrigid deformation characteristics from a principal component analysis of a set of training deformations and demonstrate results using clinical data. We show that our technique significantly outperforms both standard rigid and nonrigid intensity-based registration methods in terms of mean and maximum registration error.

Motion tracking in MR-guided liver therapy by using navigator echoes and projection profile matching 1

Diffeomorphic demons algorithm is an efficient and robust method in nonrigid image registration. It uses mean squared error as the similarity measure and set the gradient and gray level difference as the interior and exterior force respectively to make the points move. However it cannot deal with multiple modality images. In this paper, mutual information is used as the similarity measure instead of mean squared error and the gradient of mutual information respect to parameters is set as the force to make the points move. Then the diffeomorphic demons algorithm is extended to match multi-modal images. The experiment with magnetic resonance T1 image and magnetic resonance T2 image shows that this method is effective and performs better and more quickly compared with B-spline free-form deformation method.