Scale-invariant feature transform based on the frequency spectrum and the grid for remote sensing image registration

Road visual navigation relies on accurate road models. This study was aimed at proposing an improved scale-invariant feature transform (SIFT) algorithm for recovering depth information from farmland road images, which would provide a reliable path for visual navigation. The mean image of pixel value in five channels (R, G, B, S and V) were treated as the inspected image and the feature points of the inspected image were extracted by the Canny algorithm, for achieving precise location of the feature points and ensuring the uniformity and density of the feature points. The mean value of the pixels in 5×5 neighborhood around the feature point at an interval of 45o in eight directions was then treated as the feature vector, and the differences of the feature vectors were calculated for preliminary matching of the left and right image feature points. In order to achieve the depth information of farmland road images, the energy method of feature points was used for eliminating the mismatched points. Experiments with a binocular stereo vision system were conducted and the results showed that the matching accuracy and time consuming for depth recovery when using the improved SIFT algorithm were 96.48% and 5.6 s, respectively, with the accuracy for depth recovery of –7.17%-2.97% in a certain sight distance. The mean uniformity, time consuming and matching accuracy for all the 60 images under various climates and road conditions were 50%-70%, 5.0-6.5 s, and higher than 88%, respectively, indicating that performance for achieving the feature points (e.g., uniformity, matching accuracy, and algorithm real-time) of the improved SIFT algorithm were superior to that of conventional SIFT algorithm. This study provides an important reference for navigation technology of agricultural equipment based on machine vision. Keywords: scale-invariant feature transform (sift), feature matching, canny operator, energy method of feature point, farmland road, depth recovery, visual navigation DOI: 10.25165/j.ijabe.20191204.4821 Citation: Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric & Biol Eng, 2019; 12(4): 141–147.

Registration of pathological section images is an important part of three-dimensional reconstruction of sections. In this paper, a registration method was proposed to solve the problem of mismatching of pathological section images of pulmonary nodules. Firstly, rough matching is performed, and the feature points are extracted according to the scale-invariant feature transform (SIFT) algorithm. Then fast sparse coding (FSC) is used for fine matching to eliminate mismatched pairs. The algorithm presented in this paper is applied to the registration between sequence sections of pulmonary nodules. The experimental results show that the algorithm can effectively find more matching point pairs, accurately remove the false matching point pairs, and significantly improve the registration accuracy.

Distributed array radar provides new prospects for three-dimensional (3D) inverse synthetic aperture radar (ISAR) imaging. The accuracy of image registration, as an essential part of 3D ISAR imaging, affects the performance of 3D reconstruction. In this paper, the imaging process of distributed array ISAR is proposed according to the imaging model. The ISAR images of distributed array radar at different APCs have different distribution of scatters. When the local distribution of scatters for the same target are quite different, the performance of the existing ISAR image registration methods may not be optimal. Therefore, an image registration method is proposed by integrating the feature-based method and the area-based method. The proposed method consists of two stages: coarse registration and fine registration. In the first stage, a dominant scatters model is established based on scale-invariant feature transform (SIFT). In the second stage, sub-pixel precision registration is achieved using the local correlation matching method. The effectiveness of the proposed method is verified by comparison with other image registration methods. The 3D reconstruction of the registered experimental data is carried out to assess the practicability of the proposed method.

A novel digital image stabilization approach using Harris and Scale Invariant Feature Transform (SIFT) was presented in this article. Using SIFT in digital image stabilization, too many feature points and matches were extracted, but some of them were not so stable. Using these feature points and matches can not only increase the computational effort, but also enhance the wrong matching probability. We proposed to use SIFT to detect feature points and incorporate the Harris criterion to select the most stable feature points in the video sequence where image motion was happened due to vehicle or platform vibration. With these feature points, we use general feature descriptor and matching algorithm to achieve the image stabilization. Experimental results show that the proposed algorithm can not only bring down the probability of wrong matching and get more accurate matches, but also reduce the computation burden than SIFT effectively.

Image registration is an essential step in many remote-sensing (RS) applications. This article presents a study of a multisource image automatic registration system (MIARS) based on the scale-invariant feature transform (SIFT), which has been demonstrated to be the most robust local invariant feature descriptor for automatically registering various RS images. The SIFT descriptor has two shortcomings: it is unsuitable for extremely large images and has an irregular distribution of feature points. Therefore, three steps are proposed for the MIARS: image division, histogram equalization and the elimination of false point matches by a subregion least squares iteration. Image division makes it possible to use the SIFT descriptor to extract control points from an extremely large RS image. Histogram equalization in prematching improves the contrast sensitivity of RS images. The subregion least squares iteration refines the registration accuracy. Images from multisensor systems, including Quickbird, IRS-P6, Landsat/TM, HJ-CCD, HJ-IRS, light detection and ranging (LiDAR) intensity images and aerial data, were selected to test the reliability of the MIARS. The results indicated that better registration accuracy was achieved, which will be very helpful in the future development of a registration model.

In various applications, such as computer vision, medical imaging and robotics, three-dimensional (3D) image registration is a significant step. It enables the alignment of various datasets into a single coordinate system, consequently providing a consistent perspective that allows further analysis. By precisely aligning images we can compare, analyze, and combine data collected in different situations. This paper presents a novel approach for 3D or z-stack microscopy and medical image registration, utilizing a combination of conventional and deep learning techniques for feature extraction and adaptive likelihood-based methods for outlier detection. The proposed method uses the Scale-invariant Feature Transform (SIFT) and the Residual Network (ResNet50) deep neural learning network to extract effective features and obtain precise and exhaustive representations of image contents. The registration approach also employs the adaptive Maximum Likelihood Estimation SAmple Consensus (MLESAC) method that optimizes outlier detection and increases noise and distortion resistance to improve the efficacy of these combined extracted features. This integrated approach demonstrates robustness, flexibility, and adaptability across a variety of imaging modalities, enabling the registration of complex images with higher precision. Experimental results show that the proposed algorithm outperforms state-of-the-art image registration methods, including conventional SIFT, SIFT with Random Sample Consensus (RANSAC), and Oriented FAST and Rotated BRIEF (ORB) methods, as well as registration software packages such as bUnwrapJ and TurboReg, in terms of Mutual Information (MI), Phase Congruency-Based (PCB) metrics, and Gradiant-based metrics (GBM), using 3D MRI and 3D serial sections of multiplex microscopy images.

Geometric correction is used to correct the registration errors in remotely sensed images. These images are often compared to ground control points (GCPs) either by using an accurate map (image to map) or using another geo-referenced image (image to image) and then resampled. Accordingly, the exact locations and the appropriate pixel values can be calculated in more accurate, time-wise and effortless manner. In the traditional methods, the GCPs are manually selected and then the transformation models are applied which yield time consuming and less accurate processes. The objective of this work is to develop an automatic approach for image registration based on another geo-referenced image using five feature extraction models. They are Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Discrete Wavelet Transforms (DWT), (SIFT & DWT), and (SURF & DWT). The GCPs were selected based on the least-squares adjustments as the basis for improving the spatial accuracy of all the linking points in both images. The obtained results showed that models have higher accuracy in image registration with Root Mean Square Error (RMSE) less than 0.5. The developed automated image registration method provides more accurate results and saves time, money and effort.

얼굴추적은 3차원 공간상에서 머리(head)와 안면(face)의 움직임을 추정하는 기술로, 얼굴 표정 감정인식과 같은 상위 분석단계의 중요한 기반기술이다. 본 논문에서는 AAM 기반의 얼굴추적 알고리즘을 제안한다. AAM은 변형되는 대상을 분할하고 추적하는데 광범위하게 적용되고 있다. 그러나 여전히 여러 가지 해결해야할 제약사항들이 있다. 특히 자체중첩(self-occlusion)과 부분적인 중첩, 그리고 일시적으로 완전히 가려지는 완전중첩 상황에서 보통 국부해에 수렴(local convergence)하거나 발산하기 쉽다. 본 논문에서는 이러한 중첩상황에 대한 AAM의 강인성을 향상시키기 위해서 SIFT 특징을 이용하고 있다. SIFT는 일부 영상의 특징점으로 안정적인 추적이 가능하기 때문에 자체와 부분중첩에 효과적이며, 완전중첩의 상황에도 SIFT의 전역적인 매칭성능으로 별도의 재초기화 없이 연속적인 추적이 가능하다. 또한 추적과정에서 큰 자세변화에 따른 움직임을 효과적으로 추정하기 위해서 다시점(multi-view) 얼굴영상의 SIFT 특징을 온라인으로 등록하여 활용하고 있다. 제안한 알고리즘의 이러한 강인성은 위 세 가지 중첩상황에 대해서 기존 알고리즘들과의 비교실험을 통해서 보여준다. Face tracking is to estimate the motion of a non-rigid face together with a rigid head in 3D, and plays important roles in higher levels such as face/facial expression/emotion recognition. In this paper, we propose an AAM-based face tracking algorithm. AAM has been widely used to segment and track deformable objects, but there are still many difficulties. Particularly, it often tends to diverge or converge into local minima when a target object is self-occluded, partially or completely occluded. To address this problem, we utilize the scale invariant feature transform (SIFT). SIFT is an effective method for self and partial occlusion because it is able to find correspondence between feature points under partial loss. And it enables an AAM to continue to track without re-initialization in complete occlusions thanks to the good performance of global matching. We also register and use the SIFT features extracted from multi-view face images during tracking to effectively track a face across large pose changes. Our proposed algorithm is validated by comparing other algorithms under the above 3 kinds of occlusions.

For feature point detection with variable scale, rotation, variable illumination and variable 3D view port, a feature point detection and tracking method combining scale invariant feature transform (SIFT) and KLT (Kanade-Lucas-Tomasi) is proposed in this paper. SIFT feature point detection method is improved and it is used to detect feature points of image, and then KLT method is used to track the feature points continuously. In order to verify the feasibility of the proposed method, simulation experiments are carried out in real scene image sequences with different complexity using this method, better results of detection and tracking are obtained and the obtained feature point is more stable than conventional method.

Feature-based retinal fundus image registration (RIR) technique aligns fundus images according to geometrical transformations estimated between feature point correspondences. To ensure accurate registration, the feature points extracted must be from the retinal vessels and throughout the image. However, noises in the fundus image may resemble retinal vessels in local patches. Therefore, this paper introduces a feature extraction method based on a local feature of retinal vessels (CURVE) that incorporates retinal vessels and noises characteristics to accurately extract feature points on retinal vessels and throughout the fundus image. The CURVE performance is tested on CHASE, DRIVE, HRF and STARE datasets and compared with six feature extraction methods used in the existing feature-based RIR techniques. From the experiment, the feature extraction accuracy of CURVE (86.021%) significantly outperformed the existing feature extraction methods (p ≤ 0.001*). Then, CURVE is paired with a scale-invariant feature transform (SIFT) descriptor to test its registration capability on the fundus image registration (FIRE) dataset. Overall, CURVE-SIFT successfully registered 44.030% of the image pairs while the existing feature-based RIR techniques (GDB-ICP, Harris-PIIFD, Ghassabi’s-SIFT, H-M 16, H-M 17 and D-Saddle-HOG) only registered less than 27.612% of the image pairs. The one-way ANOVA analysis showed that CURVE-SIFT significantly outperformed GDB-ICP (p = 0.007*), Harris-PIIFD, Ghassabi’s-SIFT, H-M 16, H-M 17 and D-Saddle-HOG (p ≤ 0.001*).

There remain many difficult problems in computer vision research such as object recognition, three dimensional reconstruction, object tracking, etc. And the basis of solving these problems relies on image matching. The scale invariant feature transform (SIFT) algorithm has been widely used for image matching application. The SIFT algorithm can successfully extract the most descriptive feature points in given input images taken under different viewpoints. However, the performance of the original SIFT algorithm degrades under the influence of noise. We propose to modify the SIFT algorithm to produce better invariant feature points for image matching under noise. We also propose to employ the Earth mover's distance (EMD) as the measurement of similarity between two descriptors. We present extensive experiment results to demonstrate the performance of the proposed methods in image matching under noise.

A novel scale-invariant feature transform (SIFT) algorithm is proposed for soccer target recognition application in a robot soccer game. First, the method of generating scale space is given, extreme points are detected. This gives the precise positioning of the extraction step and the SIFT feature points. Based on the gradient and direction of the feature point neighboring pixels, a description of the key points of the vector is generated. Finally, the matching method based on feature vectors is extracted from SIFT feature points and implemented on the image of the football in a soccer game. By employing the proposed SIFT algorithm for football and stadium key feature points extraction and matching, significant increase can be achieved in the robot soccer ability to identify and locate the football.

In allusion to non-rigid registration of medical images, the paper gives a novel algorithm based on improved Scale Invariant Features Transform (SIFT) feature matching algorithm. First, Harris corner detection algorithm is used in the process of scale invariant feature extraction, so the number of right matching points is increased; with regard to the feature points detected in the scale space, an improved SIFT feature extraction algorithm with global context vector is presented to solve the problem that SIFT descriptors result in a lot of mismatches when an image has many similar regions. On this basis, affine transformation is chosen to implement the non-rigid registration, and weighted mutual information (WMI) measure and Particle Swarm Optimization (PSO) algorithm are also chosen to optimize the registration process. The experimental results show that the method can achieve better registration results than the method based on mutual information.

In this paper we propose a novel feature based on SIFT (Scale Invariant Feature Transform) algorithm1 for the robust representation of local visual contents. SIFT features have raised much interest for their power of description of visual content characterizing punctual information against variation of luminance and change of viewpoint and they are very useful to capture local information. For a single image hundreds of keypoints are found and they are particularly suitable for tasks dealing with image registration or image matching. In this work we stretched the spatial coverage of descriptors creating a novel feature as composition of keypoints present in an image region while maintaining the invariance properties of SIFT descriptors. The number of descriptors is reduced, limiting the computational weight, and at the same time a more abstract descriptor is achieved. The new feature is therefore suitable at describing objects and characteristic image regions. We tested the retrieval performance with a dataset used to test PCA SIFT2 and image matching capability among images processed with affine transformations. Experimental results are reported.

Quantitative evaluation of an image registration method for a NIPAM gel dosimeter