Analysis of Artifacts in Subpixel Remote Sensing Image Registration

Purpose: To build a framework for deformable B‐spline image registration with user controlled accuracy by using the positions of point landmarks as guidance. Methods: Manually identified point landmarks are placed on the reference image and their corresponding locations on the test image. An intensity‐based similarity measure is optimized under the constraint of minimizing the distance between the corresponding landmarks. After the Result of registration is visually inspected, the positions of the landmarks may be manually adjusted to improve matching accuracy. Each landmark adjustment operation triggers a re‐optimization routine, which can be repeated until the desired accuracy is reached. Results: We have tested the method on medical use cases to register CT images of the upper body, common in radiotherapy of head and neck cancer. The registration of these images compensates for position error, including pitch and yaw of the head, and flexion of the spine. Deformable registration of these images is very challenging when there is a large spatial shift of the head position. Placing landmarks on a single cross‐section and running optimization one or two times leads to a satisfactory Result. Conclusion: We have developed a practical method of interactive, intuitive correction of deformable registration of medical images. The method integrates both intensity and landmark matching. Our results prove the feasibility of the method and demonstrate potential of the tool for clinical applications.

Similarity measure is an important key in image registration. Most traditional intensity-based similarity measures (e.g. sum-of-squared-differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non-stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this assumption, a novel similarity measure is proposed here based on sparse representation in a mono-modal setting. The zero norm (l 0 ) in the transform domain is introduced as a new similarity measure in the presence of non-stationary intensity distortion. The present study defines a sparsity similarity measure that indicates the complexity of the residual image between two registered images in the transform domain such as discrete cosine transform or wavelet. It is attempted here to analytically and statistically illustrate that the proposed similarity measure has important properties such as metric properties in vector space, from correntropy perspective. This measure produces accurate registration results on both artificial and real-world problems which are examined in the present study.

To present a new approach to the problem of estimating errors in deformable image registration (DIR) applied to sequential phases of a 4DCT data set. A set of displacement vector fields (DVFs) are made by registering a sequence of 4DCT phases. The DVFs are assumed to display anatomical movement, with the addition of errors due to the imaging and registration processes. The positions of physical landmarks in each CT phase are measured as ground truth for the physical movement in the DVF. Principal component analysis of the DVFs and the landmarks is used to identify and separate the eigenmodes of physical movement from the error eigenmodes. By subtracting the physical modes from the principal components of the DVFs, the registration errors are exposed and reconstructed as DIR error maps. The method is demonstrated via a simple numerical model of 4DCT DVFs that combines breathing movement with simulated maps of spatially correlated DIR errors. The principal components of the simulated DVFs were observed to share the basic properties of principal components for actual 4DCT data. The simulated error maps were accurately recovered by the estimation method. Deformable image registration errors can have complex spatial distributions. Consequently, point-by-point landmark validation can give unrepresentative results that do not accurately reflect the registration uncertainties away from the landmarks. The authors are developing a method for mapping the complete spatial distribution of DIR errors using only a small number of ground truth validation landmarks.

Dental radiographs are used to diagnose and treat dental diseases. For accurate diagnosis, geometrical registration of radiographs is required. In order to compare two dental radiographs prepared from the same part of teeth in different time intervals. It is necessary to register the locality of images, so that by removing deviations between the two images and finding a mapping function, local transitions, rotational and scale differences and even visual differences are corrected. In this paper, an X-ray dental image registration algorithm based on template matching and similarity measure is proposed. The proposed registration process is carried out in three stages. The first stage includes the automatic selection of a set of control points of the sensed image based on its features. In the second stage, displacement vectors of these control points are determined by comparing a template and normalized cross correlation (NCC) as a similarity measure. Full displacement vector field is then performed using these displacement vectors and the affine transformation function, and the final correction is carried out by changing the geometric shape of the reference image. The obtained results of the proposed algorithm show effectiveness of this algorithm in dental radiograph image registration.

Mutual Information (MI) has been extensively used as a similarity measure in image registration and motion estimation, and it is particularly robust for 3D multimodal medical image registration. However, MI estimators are known i) to have a high variance and ii) to be computationally costly. In order to overcome these drawbacks, we propose a new similarity measure based on an Edgeworth-based third order expansion of MI and named 3-EMI in the following. This kind of approximation is well known in signal processing, and especially in Independent Components Analysis (ICA), but its computation is easier since data can be prewhitened contrary to images in registration. The performance of affine and non-rigid registrations based on the 3-EMI metric is studied through computer results in the context of cardiac multislice computed tomography. In fact, an estimate of the 3-EMI metric using sample statistics is compared to a histogram-based estimate of the standard normalized MI, showing a better robustness of the 3-EMI measure with respect to the range of the searched deformation. In addition, in practice, one part of the floating image may be missing regarding the reference image. Computer results show that our approach is less sensitive to such a practical problem.

We present an efficient solver for diffeomorphic image registration problems in the framework of Large Deformations Diffeomorphic Metric Mappings (LDDMM). We use an optimal control formulation, in which the velocity field of a hyperbolic PDE needs to be found such that the distance between the final state of the system (the transformed/transported template image) and the observation (the reference image) is minimized. Our solver supports both stationary and non-stationary (i.e., transient or time-dependent) velocity fields. As transformation models, we consider both the transport equation (assuming intensities are preserved during the deformation) and the continuity equation (assuming mass-preservation). We consider the reduced form of the optimal control problem and solve the resulting unconstrained optimization problem using a discretize-then-optimize approach. A key contribution is the elimination of the PDE constraint using a Lagrangian hyperbolic PDE solver. Lagrangian methods rely on the concept of characteristic curves. We approximate these curves using a fourth-order Runge-Kutta method. We also present an efficient algorithm for computing the derivatives of the final state of the system with respect to the velocity field. This allows us to use fast Gauss-Newton based methods. We present quickly converging iterative linear solvers using spectral preconditioners that render the overall optimization efficient and scalable. Our method is embedded into the image registration framework FAIR and, thus, supports the most commonly used similarity measures and regularization functionals. We demonstrate the potential of our new approach using several synthetic and real world test problems with up to 14.7 million degrees of freedom.

Image registration is a fundamental problem for applications in remote sensing. In this paper, a new coarse-to-fine registration framework is proposed. In coarse registration step, Quantum Particle Swarm Optimization (QPSO) is used as optimizer to find best rigid parameters. The similarity measure is the Mutual Information (MI) of whole images. This method is valid under various displacements. In fine registration step, Harris detector is implemented to extract feature points in reference image, and template window is used to obtain corresponding points in sensed image. The parameters of the best affine transformation are estimated using the corresponding feature points. Analysis and experiments show our method leads to highly automatic registration, and is able to handle large displacements between remote sensing images fast and robustly.

An iterative point correspondence algorithm for automatic image registration: An application to dental subtraction radiography

Similarity measure is a main core of image registration algorithms. Spatially varying intensity distortion is an important challenge, which affects the performance of similarity measures. Correlation among the pixels is the main characteristic of this distortion. Similarity measures such as sum-of-squared-differences (SSD) and mutual information ignore this correlation; hence, perfect registration cannot be achieved in the presence of this distortion. In this paper, we model this correlation with the aid of the low rank matrix theory. Based on this model, we compensate this distortion analytically and introduce rank-regularized SSD (RRSSD). This new similarity measure is a modified SSD based on singular values of difference image in mono-modal imaging. In fact, image registration and distortion correction are performed simultaneously in the proposed model. Based on our experiments, the RRSSD similarity measure achieves clinically acceptable registration results, and outperforms other state-of-the-art similarity measures, such as the well-known method of residual complexity.

To simulate the process of generating fluoroscopic 3D treatment images from 4DCT and measured 2D x-ray projections using a realistic modified XCAT phantom based on measured patient 3D tumor trajectories. First, the existing XCAT phantom is adapted to incorporate measured patient lung tumor trajectories. Realistic diaphragm and chest wall motion are automatically generated based on input tumor motion and position, producing synchronized, realistic motion in the phantom. Based on 4DCT generated with the XCAT phantom, we derive patient-specific motion models that are used to generate 3D fluoroscopic images. Patient-specific models are created in two steps: first, the displacement vector fields (DVFs) are obtained through deformable image registration of each phase of 4DCT with respect to a reference image (typically peak-exhale). Each phase is registered to the reference image to obtain (n-1) DVFs. Second, the most salient characteristics in the DVFs are captured in a compact representation through principal component analysis (PCA). Since PCA is a linear decomposition method, all the DVFs can be represented as linear combinations of eigenvectors. Fluoroscopic 3D images are obtained using the projection image to determine optimal weights for the eigenvectors. These weights are determined through iterative optimization of a cost function relating the projection image to the 3D image via the PCA lung motion model and a projection operator. Constructing fluoroscopic 3D images is thus reduced to finding optimal weights for the eigenvectors. Fluoroscopic 3D treatment images were generated using the modified XCAT phantom. The average relative error of the reconstructed image over 30 sec is 0.0457 HU and the standard deviation is 0.0063. The XCAT phantom was modified to produce realistic images by incorporating patient tumor trajectories. The modified XCAT phantom can be used to simulate the process of generating fluoroscopic 3D treatment images from 4DCT and 2D x-ray projections.

We have previously developed a retrospective 4D‐MRI technique using body area as the respiratory surrogate, but generally, the reconstructed 4D MR images suffer from severe or mild artifacts mainly caused by irregular motion during image acquisition. Those image artifacts may potentially affect the accuracy of tumor target delineation or the shape representation of surrounding nontarget tissues and organs. So the purpose of this study is to propose an approach employing principal component analysis (PCA), combined with a linear polynomial fitting model, to remodel the displacement vector fields (DVFs) obtained from deformable image registration (DIR), with the main goal of reducing the motion artifacts in 4D MR images. Seven patients with hepatocellular carcinoma (2/7) or liver metastases (5/7) in the liver, as well as a patient with non‐small cell lung cancer (NSCLC), were enrolled in an IRB‐approved prospective study. Both CT and MR simulations were performed for each patient for treatment planning. Multiple‐slice, multiple‐phase, cine‐MRI images were acquired in the axial plane for 4D‐MRI reconstruction. Single‐slice 2D cine‐MR images were acquired across the center of the tumor in axial, coronal, and sagittal planes. For a 4D MR image dataset, the DVFs in three orthogonal direction (inferior–superior (SI), anterior–posterior (AP), and medial–lateral (ML)) relative to a specific reference phase were calculated using an in‐house DIR algorithm. The DVFs were preprocessed in three temporal and spatial dimensions using a polynomial fitting model, with the goal of correcting the potential registration errors introduced by three‐dimensional DIR. Then PCA was used to decompose each fitted DVF into a linear combination of three principal motion bases whose spanned subspaces combined with their projections had been validated to be sufficient to represent the regular respiratory motion. By wrapping the reference MR image using the remodeled DVFs, ‘synthetic’ MR images with reduced motion artifacts were generated at selected phase. Tumor motion trajectories derived from cine‐MRI, 4D CT, original 4D MRI, and ‘synthetic’ 4D MRI were analyzed in the SI, AP, and ML directions, respectively. Their correlation coefficient (CC) and difference (D) in motion amplitude were calculated for comparison. Of all the patients, the means and standard deviations (SDs) of CC comparing ‘synthetic’ 4D MRI and cine‐MRI were 0.98±0.01,0.98±0,01, and 0.99±0.01 in SI, AP, and ML directions, respectively. The mean±SD Ds were 0.59±0.09 mm,0.29±0.10 mm, and 0.15±0.05 mm in SI, AP and ML directions, respectively. The means and SDs of CC comparing ‘synthetic’ 4D MRI and 4D CT were 0.96±0.01,0.95±0.01, and 0.95±0.01 in SI, AP, and ML directions, respectively. The mean±SD Ds were 0.76±0.20 mm,0.33±0.14 mm, and 0.19±0.07 mm in SI, AP, and ML directions, respectively. The means and SDs of CC comparing ‘synthetic’ 4D MRI and original 4D MRI were 0.98±0.01,0.98±0.01, and 0.97±0.01 in SI, AP, and ML directions, respectively. The mean±SD Ds were 0.58±0.10 mm,0.30±0.09 mm, and 0.17±0.04 mm in SI, AP, and ML directions, respectively. In this study we have proposed an approach employing PCA combined with a linear polynomial fitting model to capture the regular respiratory motion from a 4D MR image dataset. And its potential usefulness in reducing motion artifacts and improving image quality has been demonstrated by the preliminary results in oncological patients.PACS numbers: 87.57.cp, 87.57.nj, 87.61.‐c

Many similarity measures (SMs) were proposed to measure the similarity between multimodal remote sensing (RS) images. Each SM is efficient to a different degree in different registration cases (we consider visible-to-infrared, visible-to-radar, visible-to-digital elevation model (DEM), and radar-to-DEM ones), but no SM was shown to outperform all other SMs in all cases. In this article, we investigate the possibility of deriving a more powerful SM by combining two or more existing SMs. This combined SM relies on a binary linear support vector machine (SVM) classifier trained using real RS images. In the general registration case, we order SMs according to their impact on the combined SM performance. The three most important SMs include two structural SMs based on modality independent neighborhood descriptor (MIND) and scale-invariant feature transform-octave (SIFT-OCT) descriptors and one area-based logarithmic likelihood ratio (logLR) SM: the former ones are more robust to structural changes of image intensity between registered modes, the latter one is to image noise. Importantly, we demonstrate that a single combined SM can be applied in the general case as well as in each particular considered registration case. As compared to existing multimodal SMs, the proposed combined SM [based on five existing SMs, namely, MIND, logLR, SIFT-OCT, phase correlation (PC), histogram of orientated phase congruency (HOPC)] increases the area under the curve (AUC) by from 1% to 21%. From a practical point of view, we demonstrate that complex multimodal image pairs can be successfully registered with the proposed combined SM, while existing single SMs fail to detect enough correspondences for registration. Our results demonstrate that MIND, SIFT, and logLR SMs capture essential aspects of the similarity between RS modes, and their properties are complementary for designing a new more efficient multimodal SM.

Registration of multi-modal remote sensing images is an essential and challenging task in different remote sensing applications such as image fusion and multi-temporal change detection. Mutual Information (MI) has shown to be successful similarity measure for multi-modal image registration applications, however it has some drawbacks. 1. MI surface is highly non-convex with many local maxima. 2. Spatial information is completely lost in the calculation of the joint intensity probability distribution. In this paper, we present an improved MI similarity measure based on a new concept in integrating other image features as well as spatial information in the estimation of the joint intensity histogram which is used as an estimate of the joint probability distribution. The proposed method is based on the idea that each pixel in the reference image is assigned a weight, then each bin in the joint histogram is calculated as the summations of the weights of the pixels corresponding to that bin. The weight given to each pixel in the reference image is an exponential function of the corresponding pixel values in a distance image and a normalized gradient image such that higher weights are given to points close to one or more selected key points as well as points with high normalized gradient values. The proposed method is in essence a kind of calculating similarity measure using irregular sampling where sampling frequency is higher in areas close to key-points or areas with higher gradients. We have compared the proposed method with the conventional MI and Normalized MI methods for registration of pairs of multi-temporal multi-modal remote sensing images. We observed that the proposed method produces considerably better registration function containing fewer erroneous maxima and leading to higher success rate.

The image navigation and registration (INR) performance assessment for the Geostationary Operational Environmental Satellite-R (GOES-R) Advanced Baseline Imager (ABI) is an image registration process at the subpixel level, which involves the registration of images from GOES-R ABI with corresponding images from Landsat at the same scene. We present an optimized image registration algorithm for GOES-R ABI INR performance assessment (GAIPA): a gradient descent algorithm with a Sobel edge-enhancement procedure and a normalized lookup table (NLUT) image transformation (GDSN), which is an area-based image registration approach with Pearson correlation as the similarity metric. Sobel edge enhancements increase the sensitivity of Pearson correlations that lead to much improved registration accuracy. NLUT transformations minimize radiance differences between GOES-R ABI images acquired at different times and Landsat reference images on image registration outcomes, which lead to improvements in correlation values and quality of image registration outputs. Evaluations of registration accuracy and quality metrics in GDSN show considerable improvements in computational efficiency, registration accuracy, and quality over the approach implemented on the image INR performance assessment tool set. Pearson correlations can be effective and robust similarity metrics in image registration at the subpixel level with implementing Sobel edge enhancement and NLUT transformation. The statistical analysis of the image registration outputs for GAIPA shows the new algorithm provides considerable improvements in reducing the uncertainties in the registration outputs.

Image registration is a vital problem in medical imaging. It has many potential applications in clinical diagnosis (e.g. diagnosis of cardiac, retinal, pelvic, renal, abdomen, liver and tissue disorders). In this paper a new technique has been proposed based on feature based image registration using SURF (Speeded up robust features) and Hessian matrix feature extraction techniques for detecting, extracting and matching corresponding points in the reference (fixed) and source image (deformed). The source image is aligned using SURF and Hessian feature extraction techniques to align with the reference image. The proposed technique is based on multimodality of images, feature based classification and non-rigid images as the images taken are from different sources like MRI, CT scan and X-ray. The proposed technique uses feature-based methods to find correspondence between image features such as points, lines and contours. The non-rigid images use fixed (source) image and also deformable (target) image of the subject and aligns target image according to source/reference image.