SU-E-J-37: An Open Platform for 2D-3D Image Registration Experiments

Deep learning-based lung image registration: A review

Attenuation correction of PET activation studies in the presenceof task-related motion

Comparison of an image registration technique based on normalized mutual information with a standard method utilizing implanted markers in the staged treatment of large arteriovenous malformations

Image registration is a valuable technique for medical diagnosis and treatment. Due to the inferiority of image registration using maximum mutual information, a new hybrid method of multimodality medical image registration based on mutual information of spatial information is proposed. The new measure that combines mutual information, spatial information and feature characteristics, is proposed. Edge points are used as features, obtained from a morphology gradient detector. Feature characteristics like location, edge strength and orientation are taken into account to compute a joint probability distribution of corresponding edge points in two images. Mutual information based on this function is minimized to find the best alignment parameters. Finally, the translation parameters are calculated by using a modified Particle Swarm Optimization (MPSO) algorithm. The experimental results demonstrate the effectiveness of the proposed registration scheme.

In various applications, such as computer vision, medical imaging and robotics, three-dimensional (3D) image registration is a significant step. It enables the alignment of various datasets into a single coordinate system, consequently providing a consistent perspective that allows further analysis. By precisely aligning images we can compare, analyze, and combine data collected in different situations. This paper presents a novel approach for 3D or z-stack microscopy and medical image registration, utilizing a combination of conventional and deep learning techniques for feature extraction and adaptive likelihood-based methods for outlier detection. The proposed method uses the Scale-invariant Feature Transform (SIFT) and the Residual Network (ResNet50) deep neural learning network to extract effective features and obtain precise and exhaustive representations of image contents. The registration approach also employs the adaptive Maximum Likelihood Estimation SAmple Consensus (MLESAC) method that optimizes outlier detection and increases noise and distortion resistance to improve the efficacy of these combined extracted features. This integrated approach demonstrates robustness, flexibility, and adaptability across a variety of imaging modalities, enabling the registration of complex images with higher precision. Experimental results show that the proposed algorithm outperforms state-of-the-art image registration methods, including conventional SIFT, SIFT with Random Sample Consensus (RANSAC), and Oriented FAST and Rotated BRIEF (ORB) methods, as well as registration software packages such as bUnwrapJ and TurboReg, in terms of Mutual Information (MI), Phase Congruency-Based (PCB) metrics, and Gradiant-based metrics (GBM), using 3D MRI and 3D serial sections of multiplex microscopy images.

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

Mutual information is a popular intensity-based image similarity measure mainly used in image registration. This measure has been also very successful as the similarity metric in our knowledge-based computer-assisted detection (CADe) system for the detection of masses in screening mammograms. Our CADe system is designed to assess a new, query case based on its similarity with known cases stored in the knowledge database. However, intensity-based mutual information captures only relationships between the gray level values of corresponding pixels. This study presents a novel advancement of our CADe system by incorporating neighborhood textural information when estimating the mutual information of two images. Specifically, an entropy filter is applied to the images, effectively replacing each image pixel value with its neighborhood entropy. This pixel-based entropy is a localized measure of image texture. Then, the information-theoretic CAD system is asked to make a decision regarding the query case using the texture-based mutual information similarity metric. The entropy-based image enhancement and MI-based decision making processes are repeated at different neighborhood scales. Finally, an artificial network merges intensity-based and texture-based decisions to investigate possible improvements in mass detection performance. Given a database of 1,820 regions of interest (ROIs) extracted from screening mammograms (901 depicting a biopsy-proven mass and 919 depicting normal parenchyma) and a leave-one out sampling scheme, the study showed that our CADe system achieves an ROC area of 0.87±0.01 using the intensity-based ROC. The ROC performance for the texture-based CADe system ranges from 0.69±0.01 to 0.83±0.01 depending on the scale of analysis. The synergistic approach of the ANN using both intensity-based and texture-based information resulted in statistically significantly better performance with an ROC area index of 0.93±0.01.

Purpose: To enhance the speed and accuracy of certain treatment plan in radiation therapy, a real‐time dose calculation strategy is performed based on Fast Fourier Transformation (FFT) and CT image registration. For this purpose, different registration methods are achieved by using Insight Segmentation and Registration Toolkit (ITK) in MATLAB. Method and materials: Our group developed an efficient method for kernel calculation for pencil beam convolution‐superposition based on FFT. In this paper, different image registration methods are developed in MATLAB by using ITK, such as rigid 3D, similarity 3D, Demons deformation and BSpline based Free Form. As ITK is written and used in a C++ template, smart pointer and generic structure, an open source wrapper MATITK is modified to facilitate the use of its image processing capabilities while working in the high‐level environment of MATLAB. The image registration methods are demonstrated on head and neck (H&N) CT image sets of 6 patients. The results of evaluation and visualization for the registration are presented. Results and discussion: Transformation similarity and computation complexity as evaluation criteria are compared between rigid and deformable registration methods. Judging from the mutual information histogram and mean square metric, the deformable registration will get more improvement on similarity than the rigid registration. Checkerboard composites of the reference and registered images are displayed. The computational time of deformable registration are 10∼20 times longer with the whole set of images because of the pre‐processing, complicated transformation matrix and more parameters. A similarity3D registration method is adopted for the image registration of H&N radiation therapy to trade off between registration effect and the time consumption. Conclusion: The image registration can greatly contribute to the automatic contour adjustment and dose deformation for adaptive radiation therapy. A GUI program will be designed for the customized deformation region to speed up deformable registration.

A medical image registration simulation toolkit built using MATLAB graphical user interface (GUI) is presented. The developed Matlab GUI's provide an effective tool for exploring different image registration techniques including a new feature‐based image registration technique proposed by the authors. The toolkit is useful for biomedical engineering students and can be used as a web‐based learning tool. © 2009 Wiley Periodicals, Inc. Comput Appl Eng Educ 18: 225–237, 2010; Published online in Wiley InterScience (www.interscience.wiley.com); DOI 10.1002/cae.20179

Brain magnetic resonance (MR) image registration is an important part of brain MR image registration. Practically, there are many images needing to be registered at same time and the requirement for time cost is high. The current image registration methods register images one by one independently, so the methods are called static image registration methods. The methods are time consuming since they need to randomly initialize the relevant parameters before they start to register some image, so they can not meet the requirement of the practical applications for time cost. Through statistical research, we find the differences of the brain MR images are limited due to some reasons such as similar poses, similar brain structures and so on. So, we proposed one dynamic brain MR image registration algorithm combining inheritance idea and PSO. The algorithm can inherit the information of the former image's registration and use it to guide the current image registration to improve the stability of high precision of image registration and reduce time cost for image registration. The experimental results show that this algorithm can obtain higher stability of image registration, higher registration precision and lower time cost.

ObjectivesTo investigate the use of a fast dynamic hyperpolarised 129Xe ventilation magnetic resonance imaging (DXeV-MRI) method for detecting and quantifying delayed ventilation in patients with chronic obstructive pulmonary disease (COPD).MethodsThree male participants (age range 31–43) with healthy lungs and 15 patients (M/F = 12:3, age range = 48–73) with COPD (stages II–IV) underwent spirometry tests, quantitative chest computed tomography (QCT), and DXeV-MRI at 1.5-Tesla. Regional delayed ventilation was captured by measuring the temporal signal change in each lung region of interest (ROI) in comparison to that in the trachea. In addition to its qualitative assessment through visual inspection by a clinical radiologist, delayed ventilation was quantitatively captured by calculating a covariance measurement of the lung ROI and trachea signals, and quantified using both the time delay, and the difference between the integrated areas covered by the signal-time curves of the two signals.ResultsRegional temporal ventilation, consistent with the expected physiological changes across a free breathing cycle, was demonstrated with DXeV-MRI in all patients. Delayed ventilation was observed in 13 of the 15 COPD patients and involved variable lung ROIs. This was in contrast to the control group, where no delayed ventilation was demonstrated (p = 0.0173).ConclusionsDXeV-MRI offers a non-invasive way of detecting and quantifying delayed ventilation in patients with COPD, and provides physiological information on regional pulmonary function during a full breathing cycle.Key Points• Dynamic xenon MRI allows for the non-invasive detection and measurement of delayed ventilation in COPD patients.• Dynamic xenon MRI during a free breathing cycle can provide unique information about pulmonary physiology and pulmonary disease pathophysiology.• With further validation, dynamic xenon MRI could offer a non-invasive way of measuring collateral ventilation which can then be used to guide lung volume reduction therapy (LVRT) for certain COPD patients.

Abstract. Automatic registration of multi-sensor data is a basic step in data fusion for photogrammetric and remote sensing applications. The effectiveness of intensity-based methods such as Mutual Information (MI) for automated registration of multi-sensor image has been previously reported for medical and remote sensing applications. In this paper, a new multivariable MI approach that exploits complementary information of inherently registered LiDAR DSM and intensity data to improve the robustness of registering optical imagery and LiDAR point cloud, is presented. LiDAR DSM and intensity information has been utilised in measuring the similarity of LiDAR and optical imagery via the Combined MI. An effective histogramming technique is adopted to facilitate estimation of a 3D probability density function (pdf). In addition, a local similarity measure is introduced to decrease the complexity of optimisation at higher dimensions and computation cost. Therefore, the reliability of registration is improved due to the use of redundant observations of similarity. The performance of the proposed method for registration of satellite and aerial images with LiDAR data in urban and rural areas is experimentally evaluated and the results obtained are discussed.

Abstract. This paper looks into the latest achievement in the low-cost Unmanned Aerial Vehicle (UAV) technology in their capacity to map the semi-development areas. The objectives of this study are to establish a new methodology or a new algorithm in image registration during interior orientation process and to determine the accuracy of the photogrammetric products by using UAV images. Recently, UAV technology has been used in several applications such as mapping, agriculture and surveillance. The aim of this study is to scrutinize the usage of UAV to map the semi-development areas. The performance of the low cost UAV mapping study was established on a study area with two image processing methods so that the results could be comparable. A non-metric camera was attached at the bottom of UAV and it was used to capture images at both sites after it went through several calibration steps. Calibration processes were carried out to determine focal length, principal distance, radial lens distortion, tangential lens distortion and affinity. A new method in image registration for a non-metric camera is discussed in this paper as a part of new methodology of this study. This method used the UAV Global Positioning System (GPS) onboard to register the UAV image for interior orientation process. Check points were established randomly at both sites using rapid static Global Positioning System. Ground control points are used for exterior orientation process, and check point is used for accuracy assessment of photogrammetric product. All acquired images were processed in a photogrammetric software. Two methods of image registration were applied in this study, namely, GPS onboard registration and ground control point registration. Both registrations were processed by using photogrammetric software and the result is discussed. Two results were produced in this study, which are the digital orthophoto and the digital terrain model. These results were analyzed by using the root mean square errors and mean absolute error to determine the level of accuracy and the precision of photogrammetric products. It can be concluded that the new method of image registration by using the GPS onboard of the UAV produces medium accuracy result compared to the method that uses the ground control point. This new method can be used for the medium accuracy requirements. Unmanned Aerial Vehicle can be used for several applications, which requires a medium accuracy.

Radiation therapy of prostate cancer requires a high geometric accuracy of dose delivery. Unfortunately, this accuracy can be hardly obtained because of imperfect patients repositioning throughout the preparation and execution of the treatment. In order to correct for such errors we developed methods for online registration of portal images (PI) and digitally reconstructed radiographs (DRR). To improve the preconditions for registration of the Pis and DRRs image preprocessing, such as contrast-limited adaptive histogram equalization (CLAHE) of the Pis, noise and edge filtering of the Pis and the DRRs, has been performed. Then, similarity measures as mean square error, cross correlation and mutual information were investigated. A fully automatic bony-based registration method has been developed. As a reference method a semi-automatic stereometric fiducial-based registration has also been developed. This method uses gold seeds implanted into the prostate as internal fiducial markers for assessment of the similarity measures. Small lead spheres as well were adhered as external fiducial markers on the Alderson phantom. This method was also used for the verification of the automatic bony-based registration. The registration of repositioning was then performed fully automatically in less than 30 s with the Alderson phantom and with the patient data. At the present stage, there remains a registration error of less than 2 mm.

The Magnetic Flux Leakage (MFL) visualization technique is widely used in the surface defect inspection of ferromagnetic materials. However, the information of the images detected through the MFL method is incomplete when the defect (especially for the cracks) is complex, and some information would be lost when magnetized unidirectionally. Then, the multidirectional magnetization method is proposed to fuse the images detected under different magnetization orientations. It causes a critical problem: the existing image registration methods cannot be applied to align the images because the images are different when detected under different magnetization orientations. This study presents a novel image registration method for MFL visualization to solve this problem. In order to evaluate the registration, and to fuse the information detected in different directions, the mutual information between the reference image and the MFL image calculated by the forward model is designed as a measure. Furthermore, Particle Swarm Optimization (PSO) is used to optimize the registration process. The comparative experimental results demonstrate that this method has a higher registration accuracy for the MFL images of complex cracks than the existing methods.