Adaptive stochastic gradient descent optimization in multi temporal satellite image registration

Accurate assessment of land-cover/land-use change is essential for understanding the impacts of global change and necessitates the use of satellite data. Satellite change detection requires large volumes of multitemporal images to be precisely registered. Image registration is particularly difficult in dynamic (i.e., rapidly time varying) landscapes since the changes themselves interfere with the process of tie-point identification. Despite the existence of sophisticated registration algorithms, it is still problematic to register images acquired over such areas due to a dearth of stable features. Hence, we propose an automated image registration method using tie points derived from pseudoinvariant features (PIFs) and apply the method to register satellite images for hydrologic change detection in the Arctic, where abundant shallow lakes dominate the landscape but change significantly over time. A key to the method is the identification of ldquoshape-stablerdquo lakes as PIFs, which preserve their geometric shape even though the shorelines may migrate significantly. The proposed method automatically identifies PIFs based on scale-invariant shape descriptors and employs their center points for establishing the registration model. Our method thus consists of water-body feature extraction, PIF detection based on feature shape criteria, and image registration using tie points derived from the PIFs. The approach is used to register 1978 and 2000 Landsat images in Alaska, where lakes dominate the landscape and change significantly over time. The performance of the proposed approach is evaluated quantitatively, and a high subpixel registration accuracy of 0.66 pixel at Enhanced Thematic Mapper Plus resolution (i.e., 19 m) is achieved. A comparative evaluation indicates that the proposed approach outcompetes the conventional manual tie-point selection method and automated image registration techniques based on fast Fourier transform.

Satellite imagery is being used to monitor the Earth, as it allows for the continuous provision of multi-temporal observations with consistent quality. To analyze time series remote sensing data with high accuracy, the process of image registration must be conducted beforehand. Image registration techniques are mainly divided into region-based registration and feature-based registration, and both techniques extract the same points based on the similarity of spectral characteristics and object shapes between master and slave images. In addition, recently, deep learning-based siamese neural network and convolutional neural network models have been utilized to match images. This has high performance compared to previous non-deep learning algorithms, but a very large amount of data is required to train a deep learning-based image registration model. In this study, we aim to generate a dataset for training a deep learning-based optical image registration model. To build the data, we acquired Satellite Side-Looking (S2Looking) data, an open dataset, and performed preprocessing and data augmentation on the data to create input data. After that, we added offsets to the X and Y directions between the master and slave images to create label data. The preprocessed input data and labeled data were used to build a dataset suitable for image registration. The data is expected to be useful for training deep learning-based satellite image registration models.

Biomedical Engineering: Applications, Basis and CommunicationsVol. 19, No. 06, pp. 359-374 (2007) No AccessHYBRID REGISTRATION OF CORRESPONDING MAMMOGRAM IMAGES FOR AUTOMATIC DETECTION OF BREAST CANCERYih-Chih Chiou, Chern-Sheng Lin, and Cheng-Yu LinYih-Chih ChiouDepartment of Mechanical Engineering, Chung Hua University, Hsinchu, Taiwan, R. O. C. Search for more papers by this author , Chern-Sheng LinDepartment of Automatic Control Engineering, Feng Chia University, Taichung, Taiwan, R. O. C.Corresponding author: Chern Sheng Lin, Professor, Department of Automatic Control Engineering, Feng Chia University, Taichung, Taiwan, R. O. C. Search for more papers by this author , and Cheng-Yu LinDepartment of Mechanical Engineering, Chung Hua University, Hsinchu, Taiwan, R. O. C. Search for more papers by this author https://doi.org/10.4015/S101623720700046XCited by:2 PreviousNext AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack CitationsRecommend to Library ShareShare onFacebookTwitterLinked InRedditEmail AbstractMammogram registration is a critical step in automatic detection of breast cancer. Much research has been devoted to registering mammograms using either feature-matching or similarity measure. However, a few studies have been done on combining these two methods. In this research, a hybrid mammogram registration method for the early detection of breast cancer is developed by combining feature-based and intensity-based image registration techniques. Besides, internal and external features were used simultaneously during the registration to obtain a global spatial transformation. The experimental results indicates that the similarity between the two mammograms increases significantly after a proper registration using the proposed TPS-registration procedures.Keywords:Mutual informationMammogram registrationThin-plate splinesFeature matching References S. V. Engelandet al., IEEE Trans. Med. Imag. 22(11), 1436 (2003), DOI: 10.1109/TMI.2003.819273. Crossref, ISI, Google ScholarG. Ertas, H. O. Gulcur and K. Gursoy, Alignment of mammogram pairs, Proceedings of the Second Joint EMBS/BMES Conference2 (2002) pp. 1013–1014. Google ScholarF. L. Bookstein, IEEE Trans. pattern Anal. Machine Intell. 11(6), 567 (1989), DOI: 10.1109/34.24792. Crossref, ISI, Google ScholarL. G. Brown, ACM Computing Surveys 24(4), 325 (1992), DOI: 10.1145/146370.146374. Crossref, ISI, Google ScholarH. M. Chen, P. K. Varshney and M. K. Arora, IEEE Trans. Geosci. Remote Sensing 41(11), 2445 (2003), DOI: 10.1109/TGRS.2003.817664. Crossref, ISI, Google ScholarM. L. Gigeret al., Computerized detection and characterization of mass lesions in digital mammography, Proceedings of IEEE International Conference on Systems, Man and Cybernetics2 (1992) pp. 1370–1372. Google ScholarR. L. Harder and R. N. Desmarais, Journal of Aircraft 9(2), 189 (1972), DOI: 10.2514/3.44330. Crossref, Google ScholarP. J. Kostelec, J. B. Weaver and D. M. Healy, Medical Physics 25(9), 1593 (1998), DOI: 10.1118/1.598403. Crossref, ISI, Google ScholarF. Maeset al., IEEE Trans. Med. Imag. 16(2), 187 (1997), DOI: 10.1109/42.563664. Crossref, ISI, Google Scholar K. Marias et al. , Registration and matching of temporal mammograms for detecting abnormalities , Third Conference Medical Image Analysis and Understanding ( 1999 ) . Google ScholarR. Marti, R. Zwiggelaar and C. Rubin, A novel similarity measure to evaluate image correspondence, International Conference on Pattern Recognition (IEEE Computer Society, 2000) pp. 167–170. Google ScholarR. Marti, R. Zwiggelaar and C. Rubin, Tracking mammographic structures over time, Proceedings of the 12th British Machine Vision Conference (2001) pp. 143–152. Google ScholarJ. P. W. Pluim, J. B. A. Maintz and M. A. Viergerver, IEEE Trans. Med. Imag. 22(8), 986 (2003), DOI: 10.1109/TMI.2003.815867. Crossref, ISI, Google ScholarD. Rueckertet al., Non-rigid Registration of Breast MR Images using Mutual Information, Proceedings of the First International Conference on Medical Image Computing and Computer-Assisted Intervention (1998) pp. 1144–1152. Google ScholarM. Y. Sallam and K. W. Bowyer, Medical Image Analysis 3(2), 103 (1999), DOI: 10.1016/S1361-8415(99)80001-2. Crossref, Google ScholarR. Sivaramakrishna, Computer Systems Engineering, Medical Physics 25(11), 2249 (1998). ISI, Google ScholarE. A. Stamatakiset al., Detecting abnormalities on mammograms by bilateral comparison, IEEE Colloquium on Digital Mammography12 (1996) pp. 1–4. Google ScholarN. Vujovic and D. Brzakovic, IEEE Trans. Image Processing 6(10), 1388 (1997), DOI: 10.1109/83.624955. Crossref, ISI, Google Scholar Wirth M. A., A Nonrigid Approach to Medical Image Registration: Matching Images of the Breast, PhD Thesis, RMIT University, Melbourne, Australia, 2000 . Google ScholarM. A. Wirth, J. Narhan and D. Gray, Nonrigid Mammogram registration using mutual information, Medical Imaging: Image Processing, Proc SPIE 4684 (2002) pp. 562–573. Google ScholarB. Zitová and J. Flusser, Image and Vision Computing 21(11), 977 (2003). Crossref, ISI, Google Scholar Remember to check out the Most Cited Articles! Notable Biomedical TitlesAuthors from Harvard, Rutgers University, University College London and more! FiguresReferencesRelatedDetailsCited By 2LYMPHATIC INFILTRATION DETECTION IN BREAST CANCER H&E IMAGE PRIOR TO LYMPHADENECTOMYYung-Lung Kuo, Chien-Chuan Ko, and Ming-Ji Lee6 August 2014 | Biomedical Engineering: Applications, Basis and Communications, Vol. 26, No. 04Visual Perception Driven Registration of MammogramsA. Boucher, F. Cloppet, N. Vincent and P. Jouve1 Aug 2010 Recommended Vol. 19, No. 06 Metrics History Accepted 13 November 2007 KeywordsMutual informationMammogram registrationThin-plate splinesFeature matchingPDF download

Positron Emission Tomography (PET) is an essential tool for in vivo molecular imaging, due to its high sensitivity. Combined with the unparalleled soft tissue contrast of magnetic resonance imaging (MRI), PET-MRI allows for accurate quantification required for cell tracking studies. The purpose of this study is to develop a landmark based image registration program capable of fiducial registration, for application in a stem cell tracking study. PET and MR Images acquired with fiducial markers located in the field of view around the specimen were registered using our landmark based program, written in Matlab. The fiducial markers were also used to test the accuracy of our Matlab program, along with other automated image registration software including: automatic image registration (AIR), FMRIB's Linear Image Registration Tool (FLIRT), and Medical Image Processing, Analysis, and Visualization (MIPAV), all available online free for download. All of the programs were able to successfully register the images; however, our Matlab program provided the best registration accuracy, with a Fiducial Registration Error (FRE) of 1.36 mm. Our Matlab developed program is available free for download at www.sourceforge.net/p/matlabaffinereg.

Satellite image registration has been investigated for several years. Nevertheless, little attention has been paid to the linear geometry of the satellite's imaging sensor, often consisting of several pushbroom cameras. Each pushbroom camera captures 1-D image and uses straight motion of the satellite to build a 2-D image. Yet, attitude variations of the satellite during the aquisition process can lead to significant distortions in the 2-D image. In this paper, we expose the problem and present a constrained image registration method to estimate the satellite's attitude variations, and thus correct the distorted images. We use a Lucas Kanade framework and a piecewise polynomial model under constraints to deduce the registration equation. The performances of our algorithm are shown on two satellite datasets.

High-resolution satellite imagery raises new prospects for detailed study of the Greenland ice sheet (GrIS) surface processes and ice discharge. However, dramatic spatiotemporal variability of ice surface reflectance and features poses significant challenges for registration of satellite imagery. This study proposes a new feature-based registration method to register high-resolution panchromatic images of the ice sheet ablation zone. Its idea is to use relatively stable supraglacial rivers as tie points for automated image registration. A first demonstration is made using WorldView-1/2/3 panchromatic images (spatial resolution 0.5 m) as follows: first, supraglacial rivers are delineated using spectral analysis, nonlocal means denoising, Gabor filtering, and path opening. Next, buffer and overlay tools are combined to generate an area of interest and eliminate tie point outliers, yielding subset of high-confidence tie points for registration. Finally, a coherent point drift algorithm is applied to match these tie points and implement registration. Results show that the proposed method demonstrates good performance, despite a heterogeneous ice surface background that complicates river delineation. Accuracy of image registration negatively correlates with seasonal spatiotemporal variability of supraglacial river patterns, suggesting that for the best results, repeat images and time-adaptive techniques should be used. For time-stable meltwater channels, however, the method offers a novel, automated way to register high-resolution satellite imagery of the GrIS ablation zone. Well-registered ice surface high-resolution images reveal that short-term (1-2 week) variations in surface melting rate affect channel morphology (drainage densities and channel widths) significantly, whereas a signal from background advection by flowing ice is not apparent.

Registration of an image, the query or reference, to a database of rotated and translated exemplars constitutes an important image retrieval and indexing application which arises in biomedical imaging, digital libraries, georegistration, and other areas. Two important issues are the specification of a class of discriminatory and generalizable image features and determination of an appropriate image-dissimilarity measure to rank the closeness of the query image with respect to images in the database. The theoretically best set of features and dissimilarity measure are those which can be implemented with the lowest misregistration error rate. We study a method based on feature discrimination using feature coincidence trees and mutual /spl alpha/-information measures of feature correlation. Feature coincidence trees represent the commonality between pairs of images using joint histograms of many simple features, or tags, which are organized in a data structure similar to that of Y. Amit and D. Geman's randomized trees for shape recognition (see Neural Computation, vol.9, p.1545-88, 1997). The mutual alpha-information measure is a ranking discriminant applied to the joint histograms which is motivated by a large deviations framework for detection error rates. We illustrate the methodology in the context of registering ultrasound scans of human breast images.

Multi-temporal remote sensing image registration is the key step of change detection, and because of the remarkable difference and the probably unknown of sensor parameters, the automatic registration of different temporal remote sensing images is very difficult. Image registration based on Fourier-Mellin transform(FMT) is a global and phase correlation method, which is based on Fourier and Log-polar transform. This method finds the transformation parameters for registration of the images while working in the frequency-domain and it is resilient to noise, occlusions and so on. In this paper, an improved approach based on Fourier-Mellin algorithm is proposed for the registration. Spectrum aliasing and resampling interpolation will bring errors during Fourier-Mellin transform. To get a better registration result, we have improved it by adding window function and filtering to reduce spectrum aliasing and increase the robustness.

With the recent explosion in the amount of remotely sensed imagery and the corresponding interest in temporal change detection and modeling, image registration has become increasingly important as a necessary first step in the integration of multi-temporal and multi-sensor data for applications such as the analysis of seasonal and annual global climate changes, as well as land use/cover changes. The task of image registration can be divided into two major components: (1) the extraction of control points or features from images; and (2) the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual control feature extraction can be subjective and extremely time consuming, and often results in few usable points. Automated feature extraction is a solution to this problem, where desired target features are invariant, and represent evenly distributed landmarks such as edges, corners and line intersections. In this paper, we develop a novel automated registration approach based on the following steps. First, a mathematical morphology (MM)-based method is used to obtain a scale-orientation morphological profile at each image pixel. Next, a spectral dissimilarity metric such as the spectral information divergence is applied for automated extraction of landmark chips, followed by an initial approximate matching. This initial condition is then refined using a hierarchical robust feature matching (RFM) procedure. Experimental results reveal that the proposed registration technique offers a robust solution in the presence of seasonal changes and other interfering factors. Keywords-Automated image registration, multi-temporal imagery, mathematical morphology, robust feature matching.

We present a diffeomorphic image registration algorithm to learn spatial transformations between pairs of images to be registered using fully convolutional networks (FCNs) under a self‐supervised learning setting. Particularly, a deep neural network is trained to estimate diffeomorphic spatial transformations between pairs of images by maximizing an image‐wise similarity metric between fixed and warped moving images, similar to those adopted in conventional image registration algorithms. The network is implemented in a multi‐resolution image registration framework to optimize and learn spatial transformations at different image resolutions jointly and incrementally with deep self‐supervision in order to better handle large deformation between images. A spatial Gaussian smoothing kernel is integrated with the FCNs to yield sufficiently smooth deformation fields for diffeomorphic image registration. The spatial transformations learned at coarser resolutions are utilized to warp the moving image, which is subsequently used as input to the network for learning incremental transformations at finer resolutions. This procedure proceeds recursively to the full image resolution and the accumulated transformations serve as the final transformation to warp the moving image at the finest resolution. Experimental results for registering high‐resolution 3D structural brain magnetic resonance (MR) images have demonstrated that image registration networks trained by our method obtain robust, diffeomorphic image registration results within seconds with improved accuracy compared with state‐of‐the‐art image registration algorithms.

Image registration is often used in the clinic, for example during radiotherapy and image-guide surgery, but also for general image analysis. Currently, this process is often very slow, yet for intra-operative procedures the speed is crucial. For intensity-based image registration, a nonlinear optimization problem should be solved, usually by (stochastic) gradient descent. This procedure relies on a proper setting of a parameter which controls the optimization step size. This parameter is difficult to choose manually however, since it depends on the input data, optimization metric and transformation model. Previously, the Adaptive Stochastic Gradient Descent (ASGD) method has been proposed that automatically chooses the step size, but it comes at high computational cost. In this paper, we propose a new computationally efficient method to automatically determine the step size, by considering the observed distribution of the voxel displacements between iterations. A relation between the step size and the expectation and variance of the observed distribution is then derived. Experiments have been performed on 3D lung CT data (19 patients) using a nonrigid B-spline transformation model. For all tested dissimilarity metrics (mean squared distance, normalized correlation, mutual information, normalized mutual information), we obtained similar accuracy as ASGD. Compared to ASGD whose estimation time is progressively increasing with the number of parameters, the estimation time of the proposed method is substantially reduced to an almost constant time, from 40 seconds to no more than 1 second when the number of parameters is 10⁵.

Resource-based enterprises face increasing pressure to address resource challenges amid global issues. This paper explores how digital transformation can enhance their sustainability, economic efficiency, and innovation performance. The authors propose a model combining a capsule network (CapsNet) with adaptive stochastic gradient descent optimization (ASGDO) to analyze the impact of digital transformation. The study involves data collection, preprocessing, and the development of an optimized CapsNet enhanced by ASGDO for performance prediction. The results, compared with existing methods, show that the proposed model achieves 95.5% accuracy, demonstrating its potential to significantly improve the performance of resource-based enterprises.

Post-processing of migrated common image gathers can be a time-consuming task, as it requires careful parameterization of post-processing workflows. Additionally, this step must be taken before residual moveout (RMO) picking or velocity-based pore pressure analysis. We train a deep Residual network architecture with an adaptive stochastic gradient descent optimizer using back-propagation to attenuate noise. To achieve compelling performance, we incorporate bottleneck residual blocks, batch normalization, and ReLU pre-activations into the network architecture. We use field seismic from multiple Gulf of Mexico datasets to train and test the 2D neural network. Further, to preserve the timing of events, RMO curvatures, frequencies, and amplitudes, we propose a composite loss function based on the short-time Fourier transform. Experimental results show that the designed loss helps to mitigate the introduction of any edge artifacts, low or high-frequency artifacts, amplitudes shifts, and spurious kinks in the data.

An algorithm for automatic registration of pairs of portal images based on image correlation is presented. It uses a fast-Fourier-transform-based cross-correlation operator to find the optimal registration, accounting for both in-plane translations and rotations. Different cross-correlation operators have been tested: the Pearson linear correlation coefficient has been implemented by fast Fourier transform and its performance has been compared to that of the more conventional normalized cross-correlation. A sequential approach has been applied to speed up the registration considerably without degrading the performance of the algorithm. The algorithm has also been applied to the automatic registration of portal images to digitally reconstructed radiographs (DRRs), which have been modified to resemble megavoltage images. The results are indicative of the feasibility of this approach to the inspection of patient setup in radiation therapy.

Image registration is concerned with the problem of how to combine data and/or information from multiple sensors in order to achieve improved accuracies and better inference about the environment than could be attained through the use of a single sensor. Registration of imagery and information from multiple sources is essential for a variety of applications in remote sensing, medical diagnosis, computer vision, and pattern recognition. In general, an image registration methodology must deal with four issues. First, a decision has to be made regarding the choice of primitives for the registration procedure. The second issue is concerned with establishing the registration transformation function that mathematically relates geometric attributes of corresponding primitives. Then, a similarity measure should be devised to ensure the correspondence of conjugate primitives. Finally, a matching strategy has to be designed and implemented as a controlling framework that utilizes the primitives, the similarity measure, and the transformation function to solve the registration problem. This paper outlines a comprehensive investigation and implementation of the involved issues in a semi-automatic registration procedure capable of handling multi-source satellite imagery with varying geometric resolutions.