Co-Sparse Analysis Model Based Image Registration to Compensate Brain Shift by Using Intra-Operative Ultrasound Imaging.

Deep learning based data-adaptive descriptor for non-rigid multi-modal medical image registration

The Spiking Cortical Model Based Structural Representations for Non-Rigid Multi-Modal Medical Image Registration

In non-rigid image registration there are some distinct outliers introduced by the insertion or removal of some image contents, abrupt intensity changes, and local non-smooth deformations in the two images to register. Recently, non-rigid registration of images with outliers has received a great deal of attention in medical imaging research. This work presents a framework for robust non-rigid registration based on the theory of optical flow, within which we use outlier detection to implement locally-refined control point setup in multilevel free- form deformation (FFD) model. We also introduce robust M-estimator to weaken the effect of outliers in global optimization to get improved registration result. We have successfully applied our proposed method to a number of registration tasks to demonstrate its applicability and robustness to outliers.

A non-rigid medical image registration method based on improved linear elastic model

Non-rigid image registration is an essential tool required for overcoming the inherent local anatomical variations that exist between medical images acquired from different individuals or atlases, among others. This type of registration defines a deformation field that gives a translation or mapping for every pixel in the image. One popular local approach for estimating this deformation field, known as block matching, is where a grid of control points are defined on an image and are each taken as the centre of a small window. These windows are then translated in the second image to maximise a local similarity criterion. This generates two corresponding sets of control points for the two images, yielding a sparse deformation field. This sparse field can then be propagated to the entire image using well known methods such as the thin-plate spline warp or simple Gaussian convolution. Previous block matching procedures all utilise uniformly distributed grid points. This results in the generation of a sparse deformation field containing displacement estimates at uniformly spaced locations. This neglects to make use of the evidence that block matching results are dependent on the amount of local information content. That is, results are better in regions of high information when compared to regions of low information. Consequently, this paper presents a solution to this drawback by proposing the use of a Reversible Jump Markov Chain Monte Carlo (RJMCMC) statistical procedure to optimally select grid points of interest. These grid points have a greater concentration in regions of high information and a lower concentration in regions of small information. Results show that non-rigid registration can by improved by using optimally selected grid points of interest.

Background[18F]fluoro-2-deoxy-D-glucose ([18F]FDG) positron emission tomography (PET) is a valuable tool for monitoring response to therapy in oncology. In longitudinal studies, however, patients are not scanned in exactly the same position. Rigid and non-rigid image registration can be applied in order to reuse baseline volumes of interest (VOI) on consecutive studies of the same patient. The purpose of this study was to investigate the impact of various image registration strategies on standardized uptake value (SUV) and metabolic volume test-retest variability (TRT).MethodsTest-retest whole-body [18F]FDG PET/CT scans were collected retrospectively for 11 subjects with advanced gastrointestinal malignancies (colorectal carcinoma). Rigid and non-rigid image registration techniques with various degrees of locality were applied to PET, CT, and non-attenuation corrected PET (NAC) data. VOI were drawn independently on both test and retest scans. VOI drawn on test scans were projected onto retest scans and the overlap between projected VOI and manually drawn retest VOI was quantified using the Dice similarity coefficient (DSC). In addition, absolute (unsigned) differences in TRT of SUVmax, SUVmean, metabolic volume and total lesion glycolysis (TLG) were calculated in on one hand the test VOI and on the other hand the retest VOI and projected VOI. Reference values were obtained by delineating VOIs on both scans separately.ResultsNon-rigid PET registration showed the best performance (median DSC: 0.82, other methods: 0.71-0.81). Compared with the reference, none of the registration types showed significant absolute differences in TRT of SUVmax, SUVmean and TLG (p > 0.05). Only for absolute TRT of metabolic volume, significant lower values (p < 0.05) were observed for all registration strategies when compared to delineating VOIs separately, except for non-rigid PET registrations (p = 0.1). Non-rigid PET registration provided good volume TRT (7.7%) that was smaller than the reference (16%).ConclusionIn particular, non-rigid PET image registration showed good performance similar to delineating VOI on both scans separately, and with smaller TRT in metabolic volume estimates.

Background: Non-rigid registration of medical images with intensity distortions is a difficult problem due to the change in pixel intensity. It is caused by contrast agent or intensity bias field. Methods: In some cases, this problem can be solved using Residual Complexity (RC) method. However, relative modification of parameter in residual complexity would result in completely different experimental effect. Another drawback is sensitivity to noise. To handle this problem, a new intensity-based similarity measure, Weighted Residual Complexity (WRC) has been proposed for effective medical image registration in this paper. Specifically, the local entropy image of two images is computed to be aligned respectively. Then, a weighting function using a function of the local entropy difference is modeled. The weighting function is used to weight the residual image in residual complexity adaptively. The residual image is defined as the difference between reference image and warped floating image. Results: The weighting function assigns smaller weight to residual image if the corresponding pixel value is larger in local entropy difference. The proposed technique was applied to simulative and real medical images. The contrast experiments were made with mutual information, diffeomorphic demons and residual complexity. Conclusion: Also, the analysis of experimental results was made qualitatively and quantitatively, which indicates that this new approach gives a better performance than diffeomorphic demons, mutual information and residual complexity. Keywords: Weighting function, local entropy, residual complexity, intensity distortion, image registration, WRC.

Nonrigid medical image registration with locally linear reconstruction

In this paper we exploit normalized mutual information for the nonrigid registration of multimodal images. Rather than assuming that image statistics are spatially stationary, as often done in traditional information-theoretic methods, we take into account the spatial variability through a weighted combination of global normalized mutual information and local matching statistics. Spatial relationships are incorporated into the registration criterion by adoptively adjusting the weight according to the strength of local cues. With a continuous representation of images and Parzen window estimators, we have developed closed-form expressions of the first-order variation with respect to any general, nonparametric, infinite-dimensional deformation of the image domain. To characterize the performance of the proposed approach, synthetic phantoms, simulated MRIs, and clinical data are used in a validation study. The results suggest that the augmented normalized mutual information provides substantial improvements in terms of registration accuracy and robustness.

Multi-modal image registration has significant meanings in clinical diagnosis, treatment planning, and image-guided surgery. Since different modalities exhibit different characteristics, finding a fast and accurate correspondence between images of different modalities is still a challenge. In this paper, we propose an image synthesis-based multi-modal registration framework. Image synthesis is performed by a ten-layer fully convolutional network (FCN). The network is composed of 10 convolutional layers combined with batch normalization (BN) and rectified linear unit (ReLU), which can be trained to learn an end-to-end mapping from one modality to the other. After the cross-modality image synthesis, multi-modal registration can be transformed into mono-modal registration. The mono-modal registration can be solved by methods with lower computational complexity, such as sum of squared differences (SSD). We tested our method in T1-weighted vs T2-weighted, T1-weighted vs PD, and T2-weighted vs PD image registrations with BrainWeb phantom data and IXI real patients' data. The result shows that our framework can achieve higher registration accuracy than the state-of-the-art multi-modal image registration methods, such as local mutual information (LMI) and α-mutual information (α-MI). The average registration errors of our method in experiment with IXI real patients' data were 1.19, 2.23, and 1.57 compared to 1.53, 2.60, and 2.36 of LMI and 1.34, 2.39, and 1.76 of α-MI in T2-weighted vs PD, T1-weighted vs PD, and T1-weighted vs T2-weighted image registration, respectively. In this paper, we propose an image synthesis-based multi-modal image registration framework. A deep FCN model is developed to perform image synthesis for this framework, which can capture the complex nonlinear relationship between different modalities and discover complex structural representations automatically by a large number of trainable mapping and parameters and perform accurate image synthesis. The framework combined with the deep FCN model and mono-modal registration methods (SSD) can achieve fast and robust results in multi-modal medical image registration. Graphical abstract The workflow of proposed multi-modal image registration framework.

Deep learning-based affine medical image registration for multimodal minimal-invasive image-guided interventions – A comparative study on generalizability

A comparative interpretation of mammograms has become increasingly important, and it is crucial to develop subtraction processing and registration methods for mammograms. However, nonrigid image registration has seldom been applied to subjects constructed with soft tissue only, such as mammograms. We examined whether subtraction processing for the comparative interpretation of mammograms can be performed using nonrigid image registration. As a preliminary study, we evaluated the results of subtraction processing by applying nonrigid image registration to normal mammograms, assuming a comparative interpretation between the left and right breasts. Mediolateral-oblique-view mammograms were taken from noncancer patients and divided into 1000 cases for training, 100 cases for validation, and 500 cases for testing. Nonrigid image registration was applied to align the horizontally flipped left-breast mammogram with the right one. We compared the sum of absolute differences (SAD) of the difference of bilateral images (Difference Image) with and without the application of nonrigid image registration. Statistically, the average SAD was significantly lower with the application of nonrigid image registration than without it (without: 0.0692; with: 0.0549 (p < 0.001)). In four subgroups using the breast area, breast density, compressed breast thickness, and Difference Image without nonrigid image registration, the average SAD of the Difference Image was also significantly lower with nonrigid image registration than without it (p < 0.001). Nonrigid image registration was found to be sufficiently useful in aligning bilateral mammograms, and it is expected to be an important tool in the development of a support system for the comparative interpretation of mammograms.

Currently, non-rigid image registration algorithms are too computationally intensive to use in time-critical applications. Existing implementations that focus on speed typically address this by either parallelization on GPU-hardware, or by introducing methodically novel techniques into CPU-oriented algorithms. Stochastic gradient descent (SGD) optimization and variations thereof have proven to drastically reduce the computational burden for CPU-based image registration, but have not been successfully applied in GPU hardware due to its stochastic nature. This paper proposes 1) NiftyRegSGD, a SGD optimization for the GPU-based image registration tool NiftyReg, 2) random chunk sampler, a new random sampling strategy that better utilizes the memory bandwidth of GPU hardware. Experiments have been performed on 3D lung CT data of 19 patients, which compared NiftyRegSGD (with and without random chunk sampler) with CPU-based elastix Fast Adaptive SGD (FASGD) and NiftyReg. The registration runtime was 21.5s, 4.4s and 2.8s for elastix-FASGD, NiftyRegSGD without, and NiftyRegSGD with random chunk sampling, respectively, while similar accuracy was obtained. Our method is publicly available at https://github.com/SuperElastix/NiftyRegSGD.

To develop and test an automatic free-breathing, delayed enhancement imaging method with improved image signal-to-noise ratio (SNR). The proposed approach uses free-breathing, inversion-recovery single-shot fast imaging with steady precession (FISP) delayed-enhancement with respiratory motion compensation based on nonrigid image registration. Motion-corrected averaging is used to enhance SNR. Fully automatic, nonrigid registration was compared to previously validated rigid body registration that required user interaction. The performance was measured using the variance of edge positions in intensity profiles through the myocardial infarction (MI) enhanced region and through the right ventricular (RV) wall. Measured variation of the MI edge was 1.16 +/- 0.71 mm (N = 6 patients; mean +/- SD) for rigid body and 1.08 +/- 0.76 mm for nonrigid registration (no significant difference). On the other hand, significant improvement (P < 0.005) was found in the measurements at the RV edge where the SD was 2.06 +/- 0.56 mm for rigid body and 0.59 +/- 0.22 mm for nonrigid registration. The proposed approach achieves delayed enhancement images with high resolution and SNR without requiring a breathhold. Motion correction of free-breathing delayed-enhancement imaging using nonrigid image registration may be implemented in a fully automatic fashion and performs uniformly well across the full field of view (FOV).

Rigid registration of intraoperative ultrasound (US) and preoperative CT image is important for providing real-time guidance during operations. However, due to the low spatial and temporal resolutions and the dissimilarity between US and CT, accurate registration of CT and US images is still a challenging problem. The authors propose an adaptive-mask-based CT and US registration method. The registration is initialized by matching the image regions of CT and US with intensity distinctiveness. The registration is a multistage iterative process in which the US region mask is adaptively updated. Each stage is an interleaving process of optimizing a global similarity energy and updating the mask of US by selecting high saliency and local statistical dependency regions. Performances of their proposed method and mutual information (MI) based method are validated with simulated, in vitro phantom and real patient datasets. Results show that their method has larger capture range in all datasets. The estimated transformation parameters in their method are more accurate than the mutual information based method. By using an adaptively updated mask of the US image, regions with salient intensity information and high intensity correlation with CT are included in the registration. Regions which have low correlation with CT such as artifacts are excluded in the registration so that the robustness and accuracy of the intensity-based registration method are improved.