A robust registration method for high resolution remote sensing images

Automatic image registration is important for many multiframe-based image analysis applications. With an increasing number of images collected every day from different sensors, automated registration of multi-sensor/multi-spectral images has become an important issue. A wide range of registration techniques exists for different types of applications and data sources, however no algorithm is known that can accurately register multi-source images consistently. This research addresses this problem by investigating the development of a fully automatic registration system for remote sensing images. The development of this new automatic image registration method is based on the extraction and matching of common features that are visible in both images. The algorithm involves the following five steps: noise removal, edge extraction, edge linking pattern extraction and pattern matching.

Fine registration is a fundamental step for further application of remote sensing images. Focused on deficiencies in traditional manual registration, this paper presents a new method for automatic fine registration of multi-spectral images. To make the most of image information, the algorithm detects and matches feature points in the selected bands. Then pick up the common control points which contain more reliability relative to others after eliminating wrong matching points. The last registration model can be built based on common control points and the points selected by common ones. Experimental results with Landsat TM5 images demonstrate that the method is more accurate and suitable for automatic batch processing.

Multisensor systems with optical 3D sensors are frequently employed to capture complete surface information by measuring workpieces from different views. During coarse and fine registration the resulting datasets are afterward transformed into one common coordinate system. Automatic fine registration methods are well established in dimensional metrology, whereas there is a deficit in automatic coarse registration methods. The advantage of a fully automatic registration procedure is twofold: it enables a fast and contact-free alignment and further a flexible application to datasets of any kind of optical 3D sensor. In this paper, an algorithm adapted for a robust automatic coarse registration is presented. The method was originally developed for the field of object reconstruction or localization. It is based on a segmentation of planes in the datasets to calculate the transformation parameters. The rotation is defined by the normals of three corresponding segmented planes of two overlapping datasets, while the translation is calculated via the intersection point of the segmented planes. First results have shown that the translation is strongly shape dependent: 3D data of objects with non-orthogonal planar flanks cannot be registered with the current method. In the novel supplement for the algorithm, the translation is additionally calculated via the distance between centroids of corresponding segmented planes, which results in more than one option for the transformation. A newly introduced measure considering the distance between the datasets after coarse registration evaluates the best possible transformation. Results of the robust automatic registration method are presented on the example of datasets taken from a cutting tool with a fringe-projection system and a focus-variation system. The successful application in dimensional metrology is proven with evaluations of shape parameters based on the registered datasets of a calibrated workpiece.

The geometric correction of images under the scope of remote sensing applications is still mostly a manual work. This is a time and effort consuming task associated with an intra- and inter-operator subjectivity. One of the main reasons may be the lack of a proper evaluation of the different available automatic image registration (AIR) methods, since some of them are only adequate for certain types of applications/data. In order to fulfill a gap in this context, a first reference dataset of pairs of images comprising some types of geometric distortions was created, different spatial and spectral resolutions, and divided according to the Level 1 of CORINE Land Cover nomenclature (European Environment Agency). This dataset will allow for gaining perception of the abilities and limitations of some AIR methods. Some AIR methods were evaluated in this work, including the traditional correlation-based method and the SIFT approach, for which a set of measures for an objective evaluation of the geometric correction process quality was computed for every combination of pair of images/AIR method. The reference dataset is available from an internet address, being expected that it becomes a channel of interaction among the remote sensing community interested in this field.

A method for automatic image registration which is characterized by its insensitivity to scaling, rotation, and intensity changes is described. The method is based on similarity assessment of the structures in the images and on a check of their spatial arrangement. Pairs of structures that correspond to each other provide sets of control points to geometric mapping functions. An application of the method to remote-sensing image alignment with a reference map is presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A visually inspired variational method for automatic image registration is proposed to solve local deformation which traditional global registration model cannot well satisfy. The variational model considers local transformation, global smoothness and visual constraints. To account for intensity variations, we incorporate change of local contrast and brightness into our model. Firstly, the data entry of registration model is built according to the root-mean-square error of intensity; secondly, adaptive constraint using &lt;i&gt;H&lt;/i&gt;&lt;sup&gt;1&lt;/sup&gt; half norm is used to ensure the global smooth in the model; finally, in order to make sure that the spatial attributes of the image satisfy the visual requirements and without distortion, the linear features are used as priori constraints. During the solution of model parameters, the whole image is used to globally estimate the transformation parameters, and then local estimation of the parameters is taken in a small neighbor. The entire procedure is built upon a multi-level differential framework, and the transformation parameters are calculated iteratively, which can consider both global smoothness and local distortion. To assess the quality of the proposed method, ZY-3 satellite images were used. Visual and quantitative analysis proved that the proposed method can significantly improve the registration precision.

Electro-anatomical maps (EAM) and CT surface registration are widely used for catheter navigation in atrial fibrillation ablations. However, few studies have investigated the registration algorithm. Moreover, some of them are semiautomatic, so that physicians must be proficient; some are inaccurate for catheter navigation. A both automatic and accurate registration method is needed. A Hausdorff distance based approach (HD) was proposed for EAM/CT registration. First, using principal axes based registration, EAM/CT pairs were coarsely aligned. Then, using Hausdorff distance as the fine objective function, EAM/CT pairs were finely aligned. Six real EAM/CT pairs were collected from five patients and 38 simulated pairs were generated. Each pair was aligned using Carto-Merge, a stochastic approach (SA) and HD. Considering the balance of operability, accuracy and robustness, HD obtained the best EAM/CT registration results among the three approaches. Experiments validate that the proposed method registers EAM and CT surface both automatically and accurately.

As the need for automating registration techniques is recognized, the authors feel that there is a need to survey all the registration methods which may be applicable to Earth and space science problems and to evaluate their performances on a large variety of existing remote sensing data as well as on simulated data of soon-to-be-flown instruments. The authors present the first steps towards this quantitative evaluation: a few automatic image registration algorithms are described and first results of their evaluation are presented for three different datasets.

Radiotherapy is a major treatment for head and neck cancer. Currently, computed tomography (CT) is utilized to delineate the target area and make radiotherapy plans. Compared with CT images, magnetic resonance (MR) images have excellent soft tissue contrast, which can distinguish normal surrounding tissues. It is necessary to register MR and CT images since it helps physicists to improve the accuracy of radiotherapy plans. Most of the current registrations require manual intervention to select regions of interest, which increases the workload of doctors and the time of registration to a certain extent. In this paper, an automatic registration method is proposed to delineate the regions of interest. Herein, surface meshes are extracted from the CT and MR images and utilized to perform the surface matching, then the regions of interest are extracted automatically by calculating the overlapped regions between the surface mesh of the surface-aligned MR images and CT images. Furthermore, a multi-level resolution registration mechanism is utilized to improve the registration speed. Surface matching is performed using low-resolution images to obtain transformation parameters as initial parameters for rigid registration followed by deformable registration. The experiments demonstrate that our proposed method performs better in the registration speed and accuracy over the conventional methods of delineating regions of interest manually.

Automatic Registration between Real-Time Ultrasonography and Pre-Procedural Magnetic Resonance Images: A Prospective Comparison between Two Registration Methods by Liver Surface and Vessel and by Liver Surface Only

This paper presents a near real-time automatic sub-pixel registration method of high-resolution panchromatic (PAN) and multispectral (MS) images using a graphics processing unit (GPU). In the first step, the method uses differential geo-registration to enable accurate geographic registration of PAN and MS images. Differential geo-registration normalizes PAN and MS images to the same direction and scale. There are also some residual misalignments due to the geometrical configuration of the acquisition instruments. These residual misalignments mean the PAN and MS images still have deviations after differential geo-registration. The second step is to use differential rectification with tiny facet primitive to eliminate possible residual misalignments. Differential rectification corrects the relative internal geometric distortion between PAN and MS images. The computational burden of these two steps is large, and traditional central processing unit (CPU) processing takes a long time. Due to the natural parallelism of the differential methods, these two steps are very suitable for mapping to a GPU for processing, to achieve near real-time processing while ensuring processing accuracy. This paper used GaoFen-6, GaoFen-7, ZiYuan3-02 and SuperView-1 satellite data to conduct an experiment. The experiment showed that our method’s processing accuracy is within 0.5 pixels. The automatic processing time of this method is about 2.5 s for 1 GB output data in the NVIDIA GeForce RTX 2080Ti, which can meet the near real-time processing requirements for most satellites. The method in this paper can quickly achieve high-precision registration of PAN and MS images. It is suitable for different scenes and different sensors. It is extremely robust to registration errors between PAN and MS.

ABSTRACTReliable feature point matching is a pivotal step in the integrated 3D reconstruction of aerial and ground images. Nonetheless, the substantial viewpoint and scale differences between aerial and ground images introduce significant challenges in feature point matching. To address this issue, this study presents a robust matching method for feature points from these two types of images, employing partitioned attention aggregation along with a coarse‐to‐fine hierarchical matching approach, enabling efficient matching of large‐scale aerial‐ground images even with hardware limitations. In the coarse matching phase, region segmentation is carried out on the ground image based on its depth map, which is computed using a monocular depth estimation network. Subsequently, a graph neural network serves as an intermediary for information exchange between features, and partitioned attention aggregation is performed while adhering to the constraints imposed by the segmented image regions. During this process, feature points are grouped to form distinct subgraphs, enhancing the structural similarity between corresponding graphs when there are significant depth and viewpoint changes. This also reduces redundant connections within graph neural networks, leading to efficient feature matching. Corresponding local patches are generated from the full‐sized aerial‐ground images using the initial matches, and further feature point extraction and matching from these patches yield accurate point matches. The experimental results indicate the proposed method performs better than state‐of‐the‐art matching methods. Specifically, compared with the best‐performing method among the comparative ones, our method enhances the average number of correct matches and the average matching precision by approximately 6‐fold and 36%, respectively.

UAV remote sensing is low cost and convenient, but traditional methods for image registration and mosaic are low efficient, since it is hard to find ground control points for large quantity of images with small scale. In this study, adjacency transfer algorithm is used between every two adjacent images to automate the work flow: First SIFT(Scale-invariant feature transform) algorithm is employed to detect correspondence points, then polynomial algorithm is used for geometric correction, and finally all the digital orthophoto images are mosaiced into one image. Developed in VC program environment, this study builds its prototype system, which can produce digital orthophoto images with relative high efficiency. Through image overlay and visual interpretation, no image displacement was found. Adjacency transfer algorithm used in this study provides a new way for efficienct and automatic UAV image registration and mosaic.

A new method of automatic multisensor image registration based on contour description and contour matching is proposed. It first employs wavelet descriptor moments to represent the contour. Second, a contour matching criteria is presented based on a principle of minimum distance classifier; algorithms of contour matching are given according to closed and unclosed contours. Finally, control points are located according. to matched contours, and consistency checking is employed to remove mismatched control points. The image transformation model and parameter estimation using the rule of least root mean-square error (RMSE) are discussed. The method is accurate, robust and fast, and has been tested on many images. Two experimental results are presented.

Background: Structural and functional characterization of skeletal muscles is often assessed by histochemical techniques, which enable the classification into different fiber types by combining the reactions in serial transverse muscle cross-sections. A drawback is that a knowledgeable operator is required to combine and evaluate the reactions, which is a time-consuming, tedious, and subjective task. To enhance the speed and reproducibility of muscle fiber typing, the registration of serial transverse sections of muscle fibers images has been proposed as a preprocessing step. Methods: Three different registration methods were considered: first, a semi-automatic elastic point-based registration; second, an automatic (rigid, affine, and projective) whole image content-based registration; and, third, an automatic hierarchical elastic registration obtained by integration of the first two methods. The performances of the methods were tested on a database of 50 image stacks each containing three images of histochemically differently stained serial human muscle cross-sections. Results: The amounts of successful globally and locally registered stacks were approximately 20% for automatic rigid registration, 60% for automatic affine and projective registrations, and 80% for semi-automatic and automatic elastic registration. Conclusions: By using robust elastic registration methods, the automatic registration of serial transverse muscle fiber images seems feasible and might allow automatic muscle fiber typing, and consequently, improve the characterization of skeletal muscles. Cytometry 37:93–106, 1999. © 1999 Wiley-Liss, Inc.