Gradient descent approaches to image registration

This research was designed to compare three types of image shading: One generated with a Lambertian BRDF and homogeneous illumination such that image intensity was determined entirely by local surface orientation irrespective of position; one that was textured with a linear intensity gradient, such that image intensity was determined entirely by local surface position irrespective of orientation; and another that was generated with a Lambertian BRDF and inhomogeneous illumination such that image intensity was influenced by both position and orientation. A gauge figure adjustment task was used to measure observers' perceptions of local surface orientation on the depicted surfaces, and the probe points included 60 pairs of regions that both had the same orientation. The results show clearly that observers' perceptions of these three types of stimuli were remarkably similar, and that probe regions with similar apparent orientations could have large differences in image intensity. This latter finding is incompatible with any process for computing shape from shading that assumes any plausible reflectance function combined with any possible homogeneous illumination. Meeting abstract presented at VSS 2014

The research described in the present article was designed to compare three types of image shading: one generated with a Lambertian BRDF and homogeneous illumination such that image intensity was determined entirely by local surface orientation irrespective of position; one that was textured with a linear intensity gradient, such that image intensity was determined entirely by local surface position irrespective of orientation; and another that was generated with a Lambertian BRDF and inhomogeneous illumination such that image intensity was influenced by both position and orientation. A gauge figure adjustment task was used to measure observers' perceptions of local surface orientation on the depicted surfaces, and the probe points included 60 pairs of regions that both had the same orientation. The results show clearly that observers' perceptions of these three types of stimuli were remarkably similar, and that probe regions with similar apparent orientations could have large differences in image intensity. This latter finding is incompatible with any process for computing shape from shading that assumes any plausible reflectance function combined with any possible homogeneous illumination.

Multimodal registration of remotely sensed images based on Jeffrey’s divergence

Similarity measure is an important key in image registration. Most traditional intensity-based similarity measures (e.g. sum-of-squared-differences, correlation coefficient, mutual information and correlation ratio) assume a stationary image and pixel-by-pixel independence. These similarity measures ignore the correlation among pixel intensities; hence, a perfect image registration cannot be achieved especially in the presence of spatially varying intensity distortions and outlier objects that appear in one image but not in the other. It is supposed here that non-stationary intensity distortion (such as bias field) has a sparse representation in the transformation domain. Based on this assumption, a novel similarity measure is proposed here based on sparse representation in a mono-modal setting. The zero norm (l 0 ) in the transform domain is introduced as a new similarity measure in the presence of non-stationary intensity distortion. The present study defines a sparsity similarity measure that indicates the complexity of the residual image between two registered images in the transform domain such as discrete cosine transform or wavelet. It is attempted here to analytically and statistically illustrate that the proposed similarity measure has important properties such as metric properties in vector space, from correntropy perspective. This measure produces accurate registration results on both artificial and real-world problems which are examined in the present study.

We sought to describe the value of electron beam computed tomography (EBCT) to detect stent restenoses at follow-up and to identify characteristic EBCT features of coronary stents. Six coronary stents (GRII, Jostent, NIR, PS, Micro, Wiktor) were scanned in vitro (10 1. 5-mm-thick slices, 15-cm FoV, 100-ms AT) before and after inflation with 3.0-, 3.5-, and 4.0-mm balloons to study intensity values and different stent dimensions in comparison to caliper measurements. In 44 patients (60 +/- 10 years, 4 females) we prospectively compared EBCT findings [eight 8-mm-thick slices in the flow mode, 26-cm field of view (FoV), 50 ms acquisition time (AT)] with 6-month angiographic outcome after placement of 86 stents in 49 vessels. For the detection of significant angiographic luminal narrowing (>/=75%) that occurred in 17/49 vessels (35%), we found a sensitivity of 65%, a specificity of 84%, and a positive and negative predictive value of 69% and 82%, respectively. The in vitro data show significant differences in image intensity between the stents and a significant gradual decrease in image intensity with increasing stent diameters. The mean differences between EBCT and caliper measurements for the length and diameter were 0.17 +/- 0.18 mm and -0.32 +/- 0.25mm, respectively. EBCT is currently the only noninvasive technique that allows the assessment of stent dimensions and stent geometry, which may prove useful in patients when intravascular ultrasound cannot be performed. Contrast-enhanced EBCT is a promising tool for minimally invasive stent patency evaluation at follow-up, especially in patients that are unable to exercise. Cathet. Cardiovasc. Intervent. 48:39-47, 1999.

Image registration is a fundamental procedure in image processing that aligns two or more images of the same scene taken from different times, different viewpoints, or even different sensors. It is reasonable to orientate two images by matching corresponding pixels or regions that are considered identical. Based on this concept, this paper proposes a novel image registration method that applies the information theorem on intensity difference data. An entropy-based objective function is then developed according to the histogram of the intensity difference. The intensity difference represents the absolute gray-level difference of the corresponding pixels between the reference and sensed images over the overlapped region. The proposed registration method is to align the sensed image onto the reference image by minimizing the entropy of the intensity difference through iteratively updating the parameters of the similarity transformation. For performance evaluation, the proposed method is compared with the two exiting registration methods in terms of eight test image sets. The experiment is divided into two scenarios. One is to investigate the sensitivity (i.e., robustness) of the objective functions in these three different methods; the other is to verify the effectiveness of the proposed method. Through the experimental results, the proposed method is shown to be very effective in image registration and outperforms the other two methods over the test image sets.

A protocol for evaluation of similarity measures for non-rigid registration

An iterative point correspondence algorithm for automatic image registration: An application to dental subtraction radiography

In this article, the authors evaluate a merit function for 2D/3D registration called stochastic rank correlation (SRC). SRC is characterized by the fact that differences in image intensity do not influence the registration result; it therefore combines the numerical advantages of cross correlation (CC)-type merit functions with the flexibility of mutual-information-type merit functions. The basic idea is that registration is achieved on a random subset of the image, which allows for an efficient computation of Spearman's rank correlation coefficient. This measure is, by nature, invariant to monotonic intensity transforms in the images under comparison, which renders it an ideal solution for intramodal images acquired at different energy levels as encountered in intrafractional kV imaging in image-guided radiotherapy. Initial evaluation was undertaken using a 2D/3D registration reference image dataset of a cadaver spine. Even with no radiometric calibration, SRC shows a significant improvement in robustness and stability compared to CC. Pattern intensity, another merit function that was evaluated for comparison, gave rather poor results due to its limited convergence range. The time required for SRC with 5% image content compares well to the other merit functions; increasing the image content does not significantly influence the algorithm accuracy. The authors conclude that SRC is a promising measure for 2D/3D registration in IGRT and image-guided therapy in general.

We present a new method for detecting interest points using histogram information. Unlike existing interest point detectors, which measure pixel-wise differences in image intensity, our detectors incorporate histogram-based representations, and thus can find image regions that present a distinct distribution in the neighborhood. The proposed detectors are able to capture large-scale structures and distinctive textured patterns, and exhibit strong invariance to rotation, illumination variation, and blur. The experimental results show that the proposed histogram-based interest point detectors perform particularly well for the tasks of matching textured scenes under blur and illumination changes, in terms of repeatability and distinctiveness. An extension of our method to space-time interest point detection for action classification is also presented.

We present a novel framework for describing intensity-based multi-modal similarity measures. Our framework is based around a concept of internal, or self, similarity. Firstly the locations of multiple regions or patches which are "similar" to each other are identified within a single image. The term "similar" is used here to represent a generic intra-modal similarity measure. Then if we examine a second image in the same locations, and this image is registered to the first image, we should find that the patches in these locations are also "similar", though the actual features in the patches when compared between the images could be very different. We propose that a measure based on this principle could be used as an inter-modal similarity measure because, as the two images become increasingly misregistered then the patches within the second image should become increasingly dissimilar. Therefore, our framework results in an inter-modal similarity measure by using two intra-modal similarity measures applied separately within each image. In this paper we describe how popular multi-modal similarity measures such as mutual information can be described within this framework. In addition the framework has the potential to allow the formation of novel similarity measures which can register using regional information, rather than individual pixel/voxel intensities. An example similarity measure is produced and its ability to guide a registration algorithm is investigated. Registration experiments are carried out using three datasets. The pairs of images to be registered were specifically chosen as they were expected to challenge (i.e. result in misregistrations) standard intensity-based measures, such as mutual information. The images include synthetic data, cadaver data and clinical data and cover a range of modalities. Our experiments show that our proposed measure is able to achieve accurate registrations where standard intensity-based measures, such as mutual information, fail.

Deep metrics have been shown effective as similarity measures in multi-modal image registration; however, the metrics are currently constructed from aligned image pairs in the training data. In this paper, we propose a strategy for learning such metrics from roughly aligned training data. Symmetrizing the data corrects bias in the metric that results from misalignment in the data (at the expense of increased variance), while random perturbations to the data, i.e. dithering, ensures that the metric has a single mode, and is amenable to registration by optimization. Evaluation is performed on the task of registration on separate unseen test image pairs. The results demonstrate the feasibility of learning a useful deep metric from substantially misaligned training data, in some cases, the results are significantly better than from Mutual Information. Data augmentation via dithering is, therefore, an effective strategy for discharging the need for well-aligned training data; this brings deep learning based registration from the realm of supervised to semi-supervised machine learning.

Subpixel accuracy image registration is needed for applications such as digital elevation model extraction, change detection, pan-sharpening, and data fusion. In order to achieve this accuracy, the deformation between the two images to be registered is usually modeled by a displacement vector field which can be estimated by measuring rigid local shifts for each pixel in the image. In order to measure subpixel shifts, one uses image resampling. Sampling theory says that, if a continuous signal has been sampled according to the Nyquist criterion, a perfect continuous reconstruction can be obtained from the sampled version. Therefore, a shifted version of a sampled signal can be obtained by interpolation and resampling with a shifted origin. Since only a sampled version of the shifted signal is needed, the reconstruction needs only to be performed for the new positions of the samples, so the whole procedure comes to computing the value of the signal for the new sample positions. In the case of image registration, the similarity between the reference image and the shifted versions of the image to be registered is measured, assuming that the maximum of similarity determines the most likely shift. The image interpolation step is thus performed a high number of times during the similarity optimization procedure. In order to reduce the computation cost, approximate interpolations are performed. Approximate interpolators will introduce errors in the resampled image which may induce errors in the similarity measure and therefore produce errors in the estimated shifts. In this paper, it is shown that the interpolation has a smoothing effect which depends of the applied shift. This means that, in the case of noisy images, the interpolation has a denoising effect, and therefore, it increases the quality of the similarity estimation. Since this blurring is not the same for every shift, the similarity may be low for a null shift (no blurring) and higher for shifts close to half a pixel (strong blurring). This paper presents an analysis of the behavior of the different interpolators and their effects on the similarity measures. This analysis will be done for the two similarity measures: the correlation coefficient and the mutual information. Finally, a strategy to attenuate the interpolation artifacts is proposed

Image registration is a digital image processing technique that takes two or more of images of a scene in different coordinate systems and transforms them into a single coordinate system. Image registration is a necessary step in many advanced image processing techniques, such as multi-frame super-resolution. For that reason, registration accuracy is very crucial. While image registration is usually performed on images, one can perform the registration using metric images as well. This paper will present registration methods and their accuracies for various noise levels for the case of pure translational image motion. Registration techniques will be applied to the images themselves as well as to phase congruency images, gradient images, and edge-detected images. This study will also investigate registration of under-sampled images. Noise-free images are degraded using three types of noise: additive Gaussian noise, fixed-pattern noise along the column direction, and a combination of these two. The registration error is quantified for two registration algorithms with three different images as a function of the signal-to-noise ratio. A test on the usefulness of the image registration and registration accuracy performed on the intensity images of the Stokes imaging polarimeter. The Stokes images calculated before and after registration of the intensity images are compared to each other to show the improvement.

Many similarity measures (SMs) were proposed to measure the similarity between multimodal remote sensing (RS) images. Each SM is efficient to a different degree in different registration cases (we consider visible-to-infrared, visible-to-radar, visible-to-digital elevation model (DEM), and radar-to-DEM ones), but no SM was shown to outperform all other SMs in all cases. In this article, we investigate the possibility of deriving a more powerful SM by combining two or more existing SMs. This combined SM relies on a binary linear support vector machine (SVM) classifier trained using real RS images. In the general registration case, we order SMs according to their impact on the combined SM performance. The three most important SMs include two structural SMs based on modality independent neighborhood descriptor (MIND) and scale-invariant feature transform-octave (SIFT-OCT) descriptors and one area-based logarithmic likelihood ratio (logLR) SM: the former ones are more robust to structural changes of image intensity between registered modes, the latter one is to image noise. Importantly, we demonstrate that a single combined SM can be applied in the general case as well as in each particular considered registration case. As compared to existing multimodal SMs, the proposed combined SM [based on five existing SMs, namely, MIND, logLR, SIFT-OCT, phase correlation (PC), histogram of orientated phase congruency (HOPC)] increases the area under the curve (AUC) by from 1% to 21%. From a practical point of view, we demonstrate that complex multimodal image pairs can be successfully registered with the proposed combined SM, while existing single SMs fail to detect enough correspondences for registration. Our results demonstrate that MIND, SIFT, and logLR SMs capture essential aspects of the similarity between RS modes, and their properties are complementary for designing a new more efficient multimodal SM.