3D Hybrid VNet and spatial transformer model for deformable medical image registration

In deep brain stimulation surgery the most important step is the correct location of the neurostimulator device. Here, the medical specialist needs to robustly locate the basal ganglia area (i.e. subthalamic nucleus) to implant the neurostim-ulator. 3D brain atlas reconstruction methods have become in useful tools for building image guide surgery systems due to the benefits for 3D brain structure location in an interactive way. In this paper we proposes a 3D brain atlas reconstruction using deformable medical image registration for applications in deep brain stimulation surgeries. To quantitatively evaluate the performance of the registration process, we vary the registration parameters such as optimizers, similarity metrics and interpolators. The experimental results shows that deformable image registration in 3D medical images can efficiently reconstructs a 3D brain atlas from volumetric data previously labeled (Brain atlas labeled for a baseline patient) and can be used in image guide surgery systems.

The aim of this study is to evaluate anatomic lung tumor changes and dosimetric consequences utilizing the deformable daily kilovolt (KV) cone-beam computer tomography (CBCT) image registration. Five patients diagnosed with NSCLC were treated with three-dimensional conformal radiotherapy (3D CRT) and 10 daily KV CBCT image sets were acquired for each patient. Each CBCT image and plan CT were imported into the deformable image registration (DIR) system. The plan CT image was deformed by the DIR system and a new contour on CBCT was obtained by using the auto-contouring function of the DIR. These contours were individually marked as CBCT f1, CBCT f2,..., and CBCT f10, and imported into a treatment planning system (TPS). The daily CBCT plan was individually generated with the same planning criteria based on new contours. These plans were individually marked as CBCTp1, CBCTp2,..., and CBCTp10, followed by generating a dose accumulation plan (DA plan) in original pCT image contour sets by adding all CBCT plans using Varian Eclipse TPS. The maximum, minimum and mean doses to the plan target volume (PTV) in the 5 DA plans were the same with the CT plans. However, the volume of radiation 5, 10, 20, 30, and 50 Gy of the total lungs in DA plans were less than those of the CT plans. The maximum dose of the spinal cord in the DA plans were average 27.96% less than the CT plans. The mean dose for the left, right, and total lungs in the DA plans were reduced by 13.80%, 23.65%, and 12.96%, respectively. The adaptive 3D CRT based on the deformable registration can reduce the dose to the lung and the spinal cord with the same PTV dose coverage. Moreover, it provides a method for further adaptive radiotherapy exploration.

Landmark-guided deformable image registration for accurate cumulative dose evaluation in lung reirradiation.

Deformable medical image registration plays a vital role in medical image applications, such as placing different temporal images at the same time point or different modality images into the same coordinate system. Various strategies have been developed to satisfy the increasing needs of deformable medical image registration. One popular registration method is estimating the displacement field by computing the optical flow between two images. The motion field (flow field) is computed based on either gray-value or handcrafted descriptors such as the scale-invariant feature transform (SIFT). These methods assume that illumination is constant between images. However, medical images may not always satisfy this assumption. In this study, we propose a metric learning-based motion estimation method called Siamese Flow for deformable medical image registration. We train metric learners using a Siamese network, which produces an image patch descriptor that guarantees a smaller feature distance in two similar anatomical structures and a larger feature distance in two dissimilar anatomical structures. In the proposed registration framework, the flow field is computed based on such features and is close to the real deformation field due to the excellent feature representation ability of the Siamese network. Experimental results demonstrate that the proposed method outperforms the Demons, SIFT Flow, Elastix, and VoxelMorph networks regarding registration accuracy and robustness, particularly with large deformations.

To measure the sensitivity of deformable image registration to image noise. Deformable image registration can be used to map organ contours and other treatment planning data from one CT to another. These CT studies can be acquired with either conventional fan-beam CT systems or more novel cone-beam CT techniques. However, cone-beam CT images can have higher noise levels than fan-beam CT, which might reduce registration accuracy. We have investigated the effect of image quality differences on the deformable registration of fan-beam CTs and CTs with simulated cone-beam noise. Our study used three CT studies for each of five prostate patients. Each CT was contoured by three experienced radiation oncologists. For each patient, one CT was designated the source image and the other two were target images. A deformable image registration process was used to register each source CT to each target CT and then transfer the manually drawn treatment planning contours from the source CT to the target CTs. The accuracy of the automatically transferred contours (and thus of the deformable registration process) was assessed by comparing them to the manual contours on the target CTs, with the differences evaluated with respect to interobserver variability in the manual contours. Then each of the target CTs was modified to include increased noise characteristic of cone-beam CT and the tests were repeated. Changes in registration accuracy due to increased noise were detected by monitoring changes in the automatically transferred contours. We found that the additional noise caused no significant loss of registration accuracy at magnitudes that exceeded what would normally be found in an actual cone-beam CT. We conclude that noise levels in cone-beam CTs that might reduce manual contouring accuracy do not reduce image registration and automatic contouring accuracy.

While deep learning has achieved significant advances in accuracy for medical image segmentation, its benefits for deformable image registration have so far remained limited to reduced computation times. Previous work has either focused on replacing the iterative optimization of distance and smoothness terms with CNN-layers or using supervised approaches driven by labels. Our method is the first to combine the complementary strengths of global semantic information (represented by segmentation labels) and local distance metrics that help align surrounding structures. We demonstrate significant higher Dice scores (of 86.5 %) for deformable cardiac image registration compared to classic registration (79.0 %) as well as label-driven deep learning frameworks (83.4%).

A prerequisite for adaptive dose-tracking in radiotherapy is the assessment of the deformable image registration (DIR) quality. In this work, various metrics that quantify DIR uncertainties are investigated using realistic deformation fields of 26 head and neck and 12 lung cancer patients. Metrics related to the physiologically feasibility (the Jacobian determinant, harmonic energy (HE), and octahedral shear strain (OSS)) and numerically robustness of the deformation (the inverse consistency error (ICE), transitivity error (TE), and distance discordance metric (DDM)) were investigated. The deformable registrations were performed using a B-spline transformation model. The DIR error metrics were log-transformed and correlated (Pearson) against the log-transformed ground-truth error on a voxel level. Correlations of r ⩾ 0.5 were found for the DDM and HE. Given a DIR tolerance threshold of 2.0 mm and a negative predictive value of 0.90, the DDM and HE thresholds were 0.49 mm and 0.014, respectively. In conclusion, the log-transformed DDM and HE can be used to identify voxels at risk for large DIR errors with a large negative predictive value. The HE and/or DDM can therefore be used to perform automated quality assurance of each CT-based DIR for head and neck and lung cancer patients.

To commission and validate commercial deformable image registration (DIR) systems (SmartAdapt® and Velocity™) using task group 132 (TG-132) digital phantom datasets. Additionally, the study compares and verifies the DIR algorithms of the two systems. TG-132 digital phantoms were obtained from the American Association of Physicists in Medicine website and imported into SmartAdapt® and Velocity™ systems for commissioning and validation. The registration results were compared with known shifts using rigid registrations and deformable registrations. Virtual head and neck phantoms obtained online (DIR Evaluation Project) and some selected clinical data sets from the department were imported into the two DIR systems. For both of these datasets, DIR was carried out between the source and target images, and the contours were then propagated from the source to the target image data set. The dice similarity coefficient (DSC), mean distance to agreement (MDA), and Jacobian determinant measures were utilised to evaluate the registration results. The recommended criteria for commissioning and validation of DIR system from TG-132 was error <0.5*voxel dimension (vd). Translation only registration: Both systems met TG-132 recommendations except computed tomography (CT)-positron emission tomography registration in both systems (Velocity ~1.1*vd, SmartAdapt ~1.6*vd). Translational and rotational registration: Both systems failed the criteria for all modalities (For velocity, error ranged from 0.6*vd [CT-CT registration] to 3.4*vd [CT-cone-beam CT (CBCT) registration]. For SmartAdapt® the range was 0.6*vd [CT-CBCT] to 3.6*vd [CT-CT]). Mean ± standard deviation for DSC, MDA and Jacobian metrics were used to compare the DIR results between SmartAdapt® and Velocity™. The DIR algorithms of SmartAdapt® and Velocity™ were commissioned and their deformation results were compared. Both systems can be used for clinical purpose. While there were only minimal differences between the two systems, Velocity™ provided lower values for parotids, bladder, rectum, and prostate (soft tissue) compared to SmartAdapt. However, for mandible, spinal cord, and femoral heads (rigid structures), both systems showed nearly identical results.

Deformable medical image registration estimates corresponding deformation to align the regions of interest (ROIs) of two images to a same spatial coordinate system. However, recent unsupervised registration models only have correspondence ability without perception, making misalignment on blurred anatomies and distortion on task-unconcerned backgrounds. Label-constrained (LC) registration models embed the perception ability via labels, but the lack of texture constraints in labels and the expensive labeling costs causes distortion internal ROIs and overfitted perception. We propose the first few-shot deformable medical image registration framework, Perception-Correspondence Registration (PC-Reg), which embeds perception ability to registration models only with few labels, thus greatly improving registration accuracy and reducing distortion. 1) We propose the Perception-Correspondence Decoupling which decouples the perception and correspondence actions of registration to two CNNs. Therefore, independent optimizations and feature representations are available avoiding interference of the correspondence due to the lack of texture constraints. 2) For few-shot learning, we propose Reverse Teaching which aligns labeled and unlabeled images to each other to provide supervision information to the structure and style knowledge in unlabeled images, thus generating additional training data. Therefore, these data will reversely teach our perception CNN more style and structure knowledge, improving its generalization ability. Our experiments on three datasets with only five labels demonstrate that our PC-Reg has competitive registration accuracy and effective distortion-reducing ability. Compared with LC-VoxelMorph( λ = 1), we achieve the 12.5%, 6.3% and 1.0% Reg-DSC improvements on three datasets, revealing our framework with great potential in clinical application.

Purpose: To build a framework for deformable B‐spline image registration with user controlled accuracy by using the positions of point landmarks as guidance. Methods: Manually identified point landmarks are placed on the reference image and their corresponding locations on the test image. An intensity‐based similarity measure is optimized under the constraint of minimizing the distance between the corresponding landmarks. After the Result of registration is visually inspected, the positions of the landmarks may be manually adjusted to improve matching accuracy. Each landmark adjustment operation triggers a re‐optimization routine, which can be repeated until the desired accuracy is reached. Results: We have tested the method on medical use cases to register CT images of the upper body, common in radiotherapy of head and neck cancer. The registration of these images compensates for position error, including pitch and yaw of the head, and flexion of the spine. Deformable registration of these images is very challenging when there is a large spatial shift of the head position. Placing landmarks on a single cross‐section and running optimization one or two times leads to a satisfactory Result. Conclusion: We have developed a practical method of interactive, intuitive correction of deformable registration of medical images. The method integrates both intensity and landmark matching. Our results prove the feasibility of the method and demonstrate potential of the tool for clinical applications.

Deformable image registration by combining uncertainty estimates from supervoxel belief propagation.

Tissue-specific deformable image registration using a spatial-contextual filter.

Purpose: Accurate image registration in patients with Head and Neck (HN) cancer is critical for assessment of treatment response, and must be rigorously validated. To evaluate deformable image registration (DIR) with our previously developed DIR techniques on HN PET‐CT scans, we incorporated distance discordance metric (DDM), a novel method for quantification of local registration uncertainty, as well as traditional quantitative and qualitative measurements. Methods: For each CT image series, six landmark (LM) points relevant for patients with HN cancer were placed, including the inferior‐most extent of the styloid processes, the lateral‐and inferior‐most aspects of the hyoid bone, and the superior central face of the sternum. These points were used either to assist or assess registration.Inter‐patient HN CT image registration for 10 patients was performed using standard intensity‐based B‐Spline and LM‐assisted B‐Spline DIR techniques. Intra‐patient DIR of the same patients was performed using pre‐and post‐radiation therapy image sets of the same patient.DDM metrics were obtained for inter‐patient DIR results and compared to traditional qualitative and quantitative methods including whole image mean square error (MSE), relative landmark distances between scans, and visual judgment by physicians. Results: Registration results were improved with the addition of landmarks on bony structures relevant to HN cancer. While MSE only provides a general estimate of the overall image registration error, the novel DDM metric shows the registration uncertainties for each pixel on the fixed image, allowing for localization of error. Based on DDM, DIR uncertainty was significantly reduced (&gt;50% in region of interest) by LM‐assisted DIR techniques on inter‐patient registration. Relative LM distance was reduced as well for both intra‐and inter‐patient DIR (&lt;1mm). Conclusion: Evaluation of DIR results with novel and traditional methods demonstrates the LM‐assisted DIR approach achieves better registration results on both inter‐and intra‐patient DIR.

Deformable image registration is a very important topic in the field of image processing. It is widely used in image fusion and shape analysis. Generally speaking, image registration models can be divided into two categories: smooth registration and non-smooth registration. During the last decades, many smooth registration models (i.e., diffeomorphic registration) were proposed. However, image with strong noise may lead to discontinuous deformation, which cannot be modelled by smooth registration. To simulate this kind of deformation, some non-smooth registration models were also proposed. However, numerical algorithms for these models are easily trapped into a local minimum because of the nonconvexity of the object functional. To overcome the local minimum of the object functional, we propose a multiscale approach for a non-smooth registration model: the bounded deformation (BD) model. The convergence of the approach is shown, and numerical tests are also performed to show the good performance of the proposed multiscale approach.

To develop and evaluate a tool that can improve the accuracy of contour transfer between different image modalities under challenging conditions of low image contrast and large image deformation, comparing to a few commonly used methods, for radiation treatment planning. The software tool includes the following steps and functionalities: (1) accepting input of images of different modalities, (2) converting existing contours on reference images (e.g., MRI) into delineated volumes and adjusting the intensity within the volumes to match target images (e.g., CT) intensity distribution for enhanced similarity metric, (3) registering reference and target images using appropriate deformable registration algorithms (e.g., B-spline, demons) and generate deformed contours, (4) mapping the deformed volumes on target images, calculating mean, variance, and center of mass as the initialization parameters for consecutive fuzzy connectedness (FC) image segmentation on target images, (5) generate affinity map from FC segmentation, (6) achieving final contours by modifying the deformed contours using the affinity map with a gradient distance weighting algorithm. The tool was tested with the CT and MR images of four pancreatic cancer patients acquired at the same respiration phase to minimize motion distortion. Dice's Coefficient was calculated against direct delineation on target image. Contours generated by various methods, including rigid transfer, auto-segmentation, deformable only transfer and proposed method, were compared. Fuzzy connected image segmentation needs careful parameter initialization and user involvement. Automatic contour transfer by multi-modality deformable registration leads up to 10% of accuracy improvement over the rigid transfer. Two extra proposed steps of adjusting intensity distribution and modifying the deformed contour with affinity map improve the transfer accuracy further to 14% averagely. Deformable image registration aided by contrast adjustment and fuzzy connectedness segmentation improves the contour transfer accuracy between multi-modality images, particularly with large deformation and low image contrast.