An Accurate Affine Registration Method for Appearance Phenotype Measurement Based on Distance Reference and Camera Attitude Parameters

Image registration is a fundamental requirement for medical image analysis. Deep registration methods based on deep learning have been widely recognized for their capabilities to perform fast end-to-end registration. Many deep registration methods achieved state-of-the-art performance by performing coarse-to-fine registration, where multiple registration steps were iterated with cascaded networks. Recently, Non-Iterative Coarse-to-finE (NICE) registration methods have been proposed to perform coarse-to-fine registration in a single network and showed advantages in both registration accuracy and runtime. However, existing NICE registration methods mainly focus on deformable registration, while affine registration, a common prerequisite, is still reliant on time-consuming traditional optimization-based methods or extra affine registration networks. In addition, existing NICE registration methods are limited by the intrinsic locality of convolution operations. Transformers may address this limitation for their capabilities to capture long-range dependency, but the benefits of using transformers for NICE registration have not been explored. In this study, we propose a Non-Iterative Coarse-to-finE Transformer network (NICE-Trans) for image registration. Our NICE-Trans is the first deep registration method that (i) performs joint affine and deformable coarse-to-fine registration within a single network, and (ii) embeds transformers into a NICE registration framework to model long-range relevance between images. Extensive experiments with seven public datasets show that our NICE-Trans outperforms stateof-the-art registration methods on both registration accuracy and runtime.

A fully automated application was developed and used for the registration of T1-weighted magnetic resonance images (MRIs) for Alzheimer patients. Two methods for image registration were implemented and compared: affine and nonlinear registration. Nonlinear registration uses continuum-mechanics-based elastic deformation. The affine registration algorithm is linear and is generated by an amplitude-modulated phase-only filter. The nonlinear registration method uses an elastic transformation generated by Navier-Stokes continuum-mechanics models. The validation method to quantitatively compare the performance of the affine and nonlinear registration algorithms uses root-mean-square error and three-dimensional volume rendering.

PurposeKilovoltage cone beam computed tomography (kVCBCT)‐guided adaptive radiation therapy (ART) uses daily deformed CT (dCT), which is generated automatically through deformable registration methods. These registration methods may perform poorly in reproducing volumes of the target organ, rectum, and bladder during treatment. We analyzed the registration errors between the daily kVCBCTs and corresponding dCTs for these organs using the default optical flow algorithm and two registration procedures. We validated the effectiveness of these registration methods in replicating the geometry for dose calculation on kVCBCT for ART.MethodsWe evaluated three deformable image registration (DIR) methods to assess their registration accuracy and dose calculation effeciency in mapping target and critical organs. The DIR methods include (1) default intensity‐based deformable registration, (2) hybrid deformable registration, and (3) a two‐step deformable registration process. Each technique was applied to a computerized imaging reference system (CIRS) phantom (Model 062 M) and to five patients who received volumetric modulated arc therapy to the prostate. Registration accuracy was assessed using the 95% Hausdorff distance (HD95) and Dice similarity coefficient (DSC), and each method was compared with the intensity‐based registration method. The improvement in the dCT image quality of the CIRS phantom and five patients was assessed by comparing dCT with kVCBCT. Image quality quantitative metrics for the phantom included the signal‐to‐noise ratio (SNR), uniformity, and contrast‐to‐noise ratio (CNR), whereas those for the patients included the mean absolute error (MAE), mean error, peak signal‐to‐noise ratio (PSNR), and structural similarity index measure (SSIM). To determine dose metric differences, we used a dose‐volume histogram (DVH) and 3.0%/0.3 mm gamma analysis to compare planning computed tomography (pCT) and kVCBCT recalculations with restimulated CT images used as a reference.ResultsThe dCT images generated by the hybrid (dCTH) and two‐step (dCTC) registration methods resulted in significant improvements compared to kVCBCT in the phantom model. Specifically, the SNR improved by 107% and 107.2%, the uniformity improved by 90% and 75%, and the CNR improved by 212.2% and 225.6 for dCTH and dCTC methods, respectively. For the patient images, the MAEs improved by 98% and 94%, the PSNRs improved by 16.3% and 22.9%, and the SSIMs improved by 1% and 1% in the dCTH and dCTC methods, respectively. For the geometric evaluation, only the two‐step registration method improved registration accuracy. The dCTH method yielded an average HD95 of 12 mm and average DSC of 0.73, whereas dCTC yielded an average HD95 of 2.9 mm and average DSC of 0.902. The DVH showed that the dCTC‐based dose calculations differed by <2% from the expected results for treatment targets and volumes of organs at risk. Additionally, gamma indices for dCTC‐based treatment plans were >95% at all points, whereas they were <95% for kVCBCT‐based treatment plans.ConclusionThe two‐step registration method outperforms the intensity‐based and hybrid registration methods. While the hybrid and two‐step‐based methods improved the image quality of kVCBCT in a linear accelerator, only the two‐step method improved the registration accuracy of the corresponding structures among the pCT and kVCBCT datasets. A two‐step registration process is recommended for applying kVCBCT to ART, which achieves better registration accuracy for local and global image structures. This method appears to be beneficial for radiotherapy dose calculation in patients with pelvic cancer.

End-to-end affine registration framework for histopathological images with weak annotations

Affine registration has a long and venerable history in computer vision literature, and in particular, extensive work has been done for affine registration in R(2) and R(3). This paper studies affine registration in R(m) with m typically ranging from 4 to 12. To justify breaking of this dimension barrier, the first part of the paper describes three novel matching problems that can be formulated and solved as affine point-set registration problems in dimensions greater than three: stereo correspondence under motion, image set matching, and covariant point-set matching, problems that are not only interesting in their own right but also have potential for important vision applications. Unfortunately, most of the existing affine registration algorithms do not generalize easily to higher dimensions due to their inefficiency. Therefore, the second part of this paper develops a novel algorithm for estimating the affine transform between two point sets in R(m). Specifically, the algorithm follows the common approach of iteratively solving the correspondences and transform. The initial correspondences are determined using the novel notion of local spectral features, features constructed from local distance matrices. Unlike many correspondence-based methods, the proposed algorithm is capable of registering point sets of different size, and the use of local features provides some degree of robustness against noise and outliers. The proposed algorithm is validated on a variety of synthetic point sets in different dimensions with varying degrees of deformation and noise, and the paper also shows experimentally that several instances of the aforementioned three matching problems can indeed be solved satisfactorily using the proposed affine registration algorithm.

Laser three-dimensional (3D) image is a novel non-cooperative target 3D image acquisition technology, and the improvements in detection capability and imaging accuracy of the system are critically dependent on efficient echo-signal processing technique and 3D reconstruction method. The registration process is an essential step in array 3D imaging laser point cloud data processing. Registration of point clouds is an effective method that solves the problem caused by the target self-occlusion in the laser 3D imaging system. The accurate registration result will help provide better support for subsequent applications, such as object reconstruction and target recognition. In this study, a set of thresholds in the iterative closest point (ICP) algorithm is analysed on the basis of the characteristics of the laser array 3D imaging system and is combined with the range error and visual lateral resolution of the system, which are both important parameters in the imaging system. To improve the accuracy and speed of registration, the stop threshold of the iterative algorithm and the corresponding point-distance threshold in the algorithm are established in a novel way based on the range error and visual lateral resolution of the system. This forms the foundation, based on which an adaptive threshold ICP algorithm is proposed. The principal idea of the algorithm is to improve the threshold set that has a considerable effect on the accuracy and speed of registration. At first, the characteristics of the imaging point clouds of the laser array 3D imaging system are analysed in the algorithm. Based on this analysis, the distance between the two point clouds and corresponding points with ideal registrations are estimated theoretically, according to the range error and visual lateral resolution of the system. The simulation results show that the theoretically estimated results and actual results have the same variation tendency, thus providing a theoretical basis for subsequent improvements. Next, the estimated results are added according to the iterative closest point algorithm. This implies that the registration thresholds are capable of changing and adapting under different iterations and imaging systems, thus improving the speed and accuracy of registrations. This phenomenon is not seen in other algorithms. Experiments involving laser array imaging of a point cloud and laser scanning of depth imaging data show that the algorithm is practical and effective for both imaging types of point clouds and can improve the speed and accuracy of registration notably. The effectiveness and feasibility of the proposed algorithm are thus verified. In addition, for its full consideration of the imaging system, the basic idea of the proposed algorithm can be used for designing future applications as required.

Image registration technology has been widely used in many parts of the computer vision system such as the automatic optical inspection system which is used to detect the printed circuit board (PCB) defects. The accuracy of the image registration will deeply influence the system's performance, so this study proposed an accurate image registration algorithm and applied it to the PCB defect detection. Good features to track feature detector and speeded up robust feature descriptor are combined to extract efficient features to achieve the first accurate image registration. Afterwards, cross‐correlation functions were used to compute the shift between the reference image and the first‐registered image for further accurate registration. Experimental results show that the authors’ algorithm performs a much better registration, with a lower root‐mean‐square error value between the reference image and transformed image. What is more, they applied it to detect the defects of PCB with a high accuracy.

A method for accurate registration on point clouds is presented in the paper. Manual alignment or the use of landmarks is avoided in the process of multi-view point clouds. Firstly, the differential geometric information is extracted from the point clouds. The extended Gaussian sphere and combination features are used to define the corresponding points of crude alignment. Secondly, the optimal algorithm，the point-to-point Iterated Closest Point, is applied to the accurate registration on point clouds. Thus, the complete point cloud can be obtained in the method.

PurposeA fast and accurate intraoperative registration method is important for Computer-Aided Orthopedic Surgery (CAOS). A-mode ultrasound (US) is able to acquire bone surface data in a non-invasive manner. To utilize A-mode US in CAOS, a suitable registration algorithm is necessary with a small number of registration points and the presence of measurement errors. Therefore, we investigated the effects of (1) the number of registration points and (2) the Ultrasound Point Localization Error (UPLE) on the overall registration accuracy.MethodsWe proposed a new registration method (ICP-PS), including the Iterative Closest Points (ICP) algorithm and a Perturbation Search algorithm. This method enables to avoid getting stuck in the local minimum of ICP iterations and to find the adjacent global minimum. This registration method was subsequently validated in a numerical simulation and a cadaveric experiment using a 3D-tracked A-mode US system.ResultsThe results showed that ICP-PS outperformed the standard ICP algorithm. The registration accuracy improved with the addition of ultrasound registration points. In the numerical simulation, for 25 sample points with zero UPLE, the averaged registration error of ICP-PS reached 0.25 mm, while 1.71 mm for ICP, decreasing by 85.38%. In the cadaver experiment, using 25 registration points, ICP-PS achieved an RMSE of 2.81 mm relative to 5.84 mm for the ICP, decreasing by 51.88%.ConclusionsThe simulation approach provided a well-defined framework for estimating the necessary number of ultrasound registration points and acceptable level of UPLE for a given required level of accuracy for intraoperative registration in CAOS. ICP-PS method is suitable for A-mode US based intraoperative registration. This study would facilitate the application of A-mode US probe in registering the point cloud to a known shape model, which also has the potential for accurately estimating bone position and orientation for skeletal motion tracking and surgical navigation.

The complex folding pattern of the cerebral cortex has presented a major obstacle for functional MRI studies. The considerable variability in the folding structure of the cortex virtually prevents all low-dimensional registration methods from giving accurate normalization in this area. On the other hand, growing research on localizing the human neurological behavior on cortex, and the need for mapping the subjects into a standard coordinate space before performing statistical analysis, calls for more accurate mapping and registration methods. In this paper we present our approach of using the FreeSurfer software package together with the large deformation differomorphic metric maps (LDDMM) to first automatically segment the Cortex using the former and then compute accurate differomorphic mappings between each subject and the selected template's brain using the latter. We present a comparison of the accuracy of our approach with the mapping algorithm implemented in the SPM software package using a synthetic FMRI data-set.

To evaluate the accuracy of four dominant methods of three-dimensional multisource data registration. Laser-scanned dental model and maxillofacial cone-bean CT rebuilt model were collected for one orthodontic patient before treatment. Registration process was done based on locating spheres' center, anatomic landmarks, partial characteristic region and global data separately. The registration errors were detected by the function of Geomagic Studio 12.0 software. A comparison of the registration accuracy among these four methods was done by analyzing mean error and standard deviation. The mean errors and standard deviations of methods of locating spheres' center, anatomic landmarks, partial characteristic region and global data were -(0.082 ± 0.221), -(0.104 ± 0.218), -(0.047 ± 0.138) and -(0.025 ± 0.129) mm respectively. ICP registration methods had better reliability than landmark methods. The global registration was more accurate than partial characteristic region registration and the locating spheres' center method was better than anatomic landmarks method.

Purpose. To develop an optimization scheme and semi-physical regularization method for accurate and efficient deformable image registration (DIR) between magnetic resonance imaging (MRI) and computed tomography (CT) for breast cancer radiation therapy planning. Methods and Materials. A sequentially-applied semi-physical model regularization method with bending energy limited diffeomorphism (BELD) was developed to regularize transformation in free-form (B-spline) DIR. Bony structures were first segmented to account for their rigidity during deformation. Feature points were identified and filtered to be used as a penalty term. Rigidity, BELD, and feature point distance penalty terms are applied sequentially during the optimization process based on mutual information metric. Sample MRI and CT images of breast cancer patients were used to demonstrate the registration. Agreements of feature point pairs and contour transfer between the two image modalities were measured to illustrate the accuracy and efficiency for the developed DIR method. Results. The newly developed tool can accurately register MRI to CT with minimum user intervention for situations with moderate to large deformations. For the sample cases tested, the average feature point pair distance was reduced from 6.7 to 3.2 mm by simple B-spline registration and to 1.8 mm by the newly-developed registration method. Compared to simultaneous optimization of multiple penalty terms, the computation time was shortened by 30% and the accuracy was improved by 15%. The bone area warping artifact in deformed MRI disappeared in the proposed method. Considering the resolution of MR images is 1.3 mm, the resulting feature point pair distance is satisfactory. Conclusion. The proposed DIR method of sequentially applying physical model regularization terms during the optimization process can reduce the pitfalls of local extremity, weighting factor adjustment and multi-resolution optimization stages, therefore improving the accuracy and efficiency of MRI to CT registration even with moderate to large deformations commonly seen in radiation treatment planning with multi-modality images.

Multimodality image registration quality assurance for conformal three-dimensional treatment planning

In this study, we evaluated SPECT/MRI registration of ictal data, using similarity based registration methods. An absolute gold standard for registration evaluation was obtained by considering realistic normal and ictal SPECT simulations deduced from a high resolution T1-weighted MRI data set.Those simulations were also used to study the impact of photon attenuation and Compton scatter corrections on registration accuracy. Evaluation of registration was also performed using inconsistency measurements for six patients with temporo-mesial epilepsy. For these data, as no Gold Standard was available, registration accuracy was assessed using inconsistency measurements involving a registration loop between inter-ictal SPECT, ictal SPECT and MRI data. Five registration methods based on statistical similarity measurements were compared, namely: mutual information (MI), normalized mutual information (NMI), L1 and L2 norm-based correlation ratios (CR) and correlation coefficient (CC).It was found that the simulation context had more influence on registration accuracy than the choice of the similarity criterion. Ictal SPECT as well as correction for uniform attenuation clearly decreased SPECT/MRI registration accuracy.KeywordsMutual InformationAttenuation CorrectionScatter CorrectionRegistration MethodNormalize Mutual InformationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

To balance dose reduction and image registration accuracy in breast setup imaging. In particular, the authors demonstrate the relationship between scan angle and dose delivery for cone beam tomosynthesis (CBTS) when employed for setup verification of breast cancer patients with surgical clips. The dose measurements were performed in a female torso phantom for varying scan angles of CBTS. Setup accuracy was measured using three registration methods: Clip centroid localization accuracy and the accuracy of two semiautomatic registration algorithms. The dose to the organs outside of the ipsilateral breast and registration accuracy information were compared to determine the optimal scan angle for CBTS for breast patient setup verification. Isocenter positions at the center of the patient and at the breast-chest wall interface were considered. Image registration accuracy was within 1 mm for the CBTS scan angles theta above 20 degrees for some scenarios and as large as 80 degrees for the worst case, depending on the imaged breast and registration algorithm. Registration accuracy was highest based on clip centroid localization. For left and right breast imaging with the isocenter at the chest wall, the dose to the contralateral side of the patient was very low (<0.5 cGy) for all scan angles considered. For central isocenter location, the optimal scan angles were 30 degrees - 50 degrees for the left breast imaging and 40 degrees - 50 degrees for the right breast imaging, with the difference due to the geometric asymmetry of the current clinical imaging system. The optimal scan angles for CBTS imaging were found to be between 10 degrees and 50 degrees, depending on the isocenter location and ipsilateral breast. Use of the isocenter at the breast-chest wall locations always resulted in greater accuracy of image registration (<1 mm) at smaller angles (10 degrees - 20 degrees) and at lower doses (<0.1 cGy) to the contralateral organs. For chest wall isocenters, doses delivered to organs outside of the target breast were much smaller than the scattered and leakage doses of the treatment beams. The complete volumetric information of all clips in the region of interest, combined with the small dose to the contralateral organs and the small scan angle, could result in an advantage for small angle CBTS with off center isocenters over simple orthogonal pairs.