Characterizing Feature Matching Performance over Long Time Periods

In the field of image processing, a piece of information can be defined as a feature, which can be utilised to determine the computational task with references to some applications. Features may refer to specific structures in the images such as points, edges or objects. Feature detection and feature description are the initial steps of image registration process. There are many algorithms of feature detection like BRISK, FAST, SURF, etc. In our earlier paper [15], a hybrid algorithm for feature detection and feature description has been proposed. The next step of image registration is feature matching. This paper deals with feature matching algorithm using BRISK [4], FAST [3] and hybrid [15]. The proposed hybrid algorithm performs better results than the above two existing algorithms in terms of elapsed time, CPU time and performance measuring time. This can be clearly concluded that the feature matching result of hybrid in terms of tables and figures performs well. For testing the algorithm, river dam images of Odisha are chosen.

At present, there are many methods to realize the matching of specified images with features, and the basic components include image feature point detection, feature description, and image matching. Based on this background, this article has done different research and exploration around these three aspects. The image feature point detection method is firstly studied, which commonly include image edge information-based feature detection method, corner information-based detection method, and various interest operators. However, all of the traditional detection methods are involved in problems of large computation burden and time consumption. In order to solve this problem, a feature detection method based on image grayscale information-FAST operator is used in this paper, which is combined with decision tree theory to effectively improve the speed of extracting image feature points. Then, the feature point description method BRIEF operator is studied, which is a local expression of detected image feature points based on descriptors. Since the descriptor does not have rotation invariance, the detection operator is endowed by a direction that is proposed in this paper, and then the local feature description is conducted on the feature descriptor to generate a binary string array containing direction information. Finally, the feature matching machine learning method is analyzed, and the nearest search method is used to find the nearest feature point pair in Euclidean distance, of which the calculation burden is small. The simulation results show that the proposed nearest neighbor search and matching machine learning algorithm has higher matching accuracy and faster calculation speed compared with the classical feature matching algorithm, which has great advantages in processing a large number of array images captured by the light field camera.

During the continual transitions from lab research to real-world applications of vision-based algorithms, there are significant challenges, e.g. the robustness to adapt to complex environments and the high demands of multi-task, multi-modal learning models. More specifically on visual simultaneous localization and mapping (visual SLAM) for mobile robots, two notable limitations are: (i) the drift issue during pose estimation, especially in dynamic environments, makes the positioning system unstable; (ii) the additional loop detection takes many computational resources for image retrieval and global feature geometry checking. This thesis explores the approaches of visual localization on unmanned ground vehicles by solving those limitations. The conventional feature extraction and matching pipeline for vision-based tasks involve three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. The first contribution of the thesis is adding a feature selection and anticipation stage to reduce tracking drift. We explore the noise model of image features to select a subset of all the observed image features with the best ”contribution” during data association and pose estimation across multiple frames. Conventional SLAM algorithms take a strong assumption of scene rigidity, which limits the application under challenging environments. The second part of the thesis addresses the tough issue in dynamic environments with moving objects. We presented GMC, namely the motion clustering approach, a lightweight dynamic object filtering method. It can distinguish moving objects from static landmarks. Based on the theory of motion coherence within a particular image area, GMC could segment dynamic objects in 3D space. We can provide an efficient and robust correspondence algorithm that can extract dynamic objects from a static background with the method. In this way, we propose a dynamic SLAM system that is real-time and free from expensive GPU processors. In contrast to GMC, the thesis’s third part turns to an object-aware learning-based model for more general dynamic scenarios. We use object detection and tracking as points, lines, planes, etc. We utilize semantic information and extract sparse image features simultaneously to keep track of dynamic objects. The static background and different dynamic objects are jointly optimized in a newly developed bundle adjustment sliding window. The estimated 3D bounding boxes can provide more robust camera tracking and better scene understanding, and better map merging. The fourth part of the thesis leverages the emerging feature learning framework. It proposes a unified self-supervised model called LGDNet to generate both global and local image feature descriptors end-to-end. Global feature descriptors embed the whole image into a compact representation, leading to easier scene comparison. On the other hand, local features focus more on the local region similarities of some salient parts for structure from motion. Our proposed method can directly extract features together with descriptors that encode both local maximum responses and global context information, avoiding duplicate calculations based on different feature extraction criteria.

In 3D object recognition, local feature-based recognition is known to be robust against occlusion and clutter. Local feature estimation requires feature correspondences, including feature extraction and matching. Feature extraction is normally a two-stage process that estimates keypoints and keypoint descriptors, and existing studies show repeatability to be a good indicator of keypoint feature detector robustness. However, the impact of keypoint repeatability on feature correspondence estimation and overall feature matching accuracy has not yet been studied. In this paper, local features are extracted at both regular and repeatable 3D keypoints using leading keypoint detectors combined with the SHOT descriptor to estimate a set of correspondences. When using a keypoint detector of high repeatability, experimental results show improved feature matching accuracy and reduced computational requirements for the feature description and matching, and overall correspondence estimation process.

Image-based localization plays an important role in today's autonomous driving technologies. However, in large scale outdoor environments, challenging conditions, e.g., lighting changes or different weather, heavily affect image appearance and quality. As a key component of feature-based visual localization, image feature detection and matching deteriorate severely and cause worse localization performance. In this paper, we propose a novel method for robust image feature matching under drastically changing outdoor environments. In contrast to existing approaches which try to learn robust feature descriptors, we train a deep network that outputs the low-rank representations of the images where the undesired variations on the images are removed, and perform feature extraction and matching on the learned low-rank space. We demonstrate that our learned low-rank images largely improve the performance of image feature matching under varying conditions over a long period of time.

Global and local geometric constrained feature matching for high resolution remote sensing images

Infrared to Visible (IR2VIS) image registration suffers from the challenge of cross-modal feature extraction and matching. Conventional methods usually design the same keypoint detector for both Infrared (IR) and Visible (VIS) images. The VIS images are even converted to gray-scale images before the keypoint detection. IR and VIS gray-scale images have different properties which might not be applicable for the same feature detector. Therefore, this paper proposes an IR2VIS image registration method, namely, Image Translation for Image Enhanced Registration (ITIER). The IR images are first translated to realistic VIS images by Wavelet-Guided Generative Adversarial Network (WGGAN) for the convenience of cross-modal feature detection. Then the keypoint detection and matching and the homography transformation, which have been integrated into our ITIER, are conducted on the translated and original VIS images. Experimental results demonstrate that the IR2VIS image registration accuracy is greatly enhanced by the image-to-image translation procedure, which transfers IR images to realistic VIS images.

In this paper, we propose a novel Visual Odometry (VO) system using a feature detector and feature matcher based on neural networks. The networks for feature detectors and descriptors learning consists of a conventional CNN for feature detection and description, and a graph neural network (GNN) final feature matching. The learned feature has several advantages over traditional handcrafted features such as being robust to light variation and scale. By applying state-of-the-art deep learning-based feature Matcher-SuperGlue, we developed a new monocular VO framework which can exploit the advantages of deep learning-based feature detector and matcher, which performs better than many other learning-based VO methods.

Many computer vision applications rely on feature matching, hence the need for computationally efficient and robust 4D light field (LF) feature detectors and descriptors for applications using this imaging modality. In this paper, we propose a novel LF feature extraction method in the scale-disparity space, based on a Fourier disparity layer representation. The proposed feature extraction takes advantage of both the Harris feature detector and SIFT descriptor, and is shown to yield more accurate feature matching, compared with the LiFF light field feature with low computational complexity. In order to evaluate the feature matching performance with the proposed descriptor, we generated synthetic LF datasets with ground truth matching points. Experimental results with synthetic and real datasets show that, our solution outperforms existing methods in terms of both feature detection robustness and feature matching accuracy.

The SIFT algorithm (Scale Invariant Feature Transform) proposed by Lowe [1] is an approach for extracting distinctive invariant features from images. It has been successfully applied to a variety of computer vision problems based on feature matching including object recognition, pose estimation, image retrieval and many others. However, in real-world applications there is still a need for improvement of the algorithm’s robustness with respect to the correct matching of SIFT features. In this paper, an improvement of the original SIFT algorithm providing more reliable feature matching for the purpose of object recognition is proposed. The main idea is to divide the features extracted from both the test and the model object image into several sub-collections before they are matched. The features are divided into several sub-collections considering the features arising from different octaves, that is from different frequency domains. The SIFT algorithm (Scale Invariant Feature Transform) proposed by Lowe [1] is an approach for extracting distinctive invariant features from images. It has been successfully applied to a variety of computer vision problems based on feature matching including object recognition, pose estimation, image retrieval and many others. However, in real-world applications there is still a need for improvement of the algorithm’s robustness with respect to the correct matching of SIFT features. In this paper, an improvement of the original SIFT algorithm providing more reliable feature matching for the purpose of object recognition is proposed. The main idea is to divide the features extracted from both the test and the model object image into several sub-collections before they are matched. The features are divided into several sub-collections considering the features arising from different octaves, that is from different frequency domains.

Image mosaicing is a method that combines several images or pictures of the superposing field of view to create a panoramic high-resolution picture. In the field of medical imagery, satellite data, computer vision, military automatic target recognition can be seen the importance of image mosaicing. The present domains of studies in computer vision, computer graphics and photo graphics are image stitching and video stitching. The registration of images includes five primary phases: feature detection and description; matching feature; rejection of outliers; transformation function derivation; image replication. Stitching images from specific scenes is a difficult job when images can be picked up under different noise. In this paper, we examine an algorithm for seamless stitching of images in order to resolve all such problems by employing dehazing methods to the collected images, and before defining image features and bound energy characteristics that match image-based features of the SIFT-Scale Invariant Feature Transform. The proposed method experimentation is compared with the conventional methods of stitching of image using squared distance to match the feature. The proposed seamless stitching technique is assessed on the basis of the metrics, HSGV and VSGV. The analysis of this stitching algorithm aims to minimize the amount of computation time and discrepancies in the final stitched results obtained.

Purpose:To assess the performance of the deformable image registration algorithm used for MRI‐guided adaptive radiation therapy using image feature analysis.Methods:MR images were collected from five patients treated on the MRIdian (ViewRay, Inc., Oakwood Village, OH), a three head Cobalt‐60 therapy machine with an 0.35 T MR system. The images were acquired immediately prior to treatment with a uniform 1.5 mm resolution. Treatment sites were as follows: head/neck, lung, breast, stomach, and bladder. Deformable image registration was performed using the ViewRay software between the first fraction MRI and the final fraction MRI, and the DICE similarity coefficient (DSC) for the skin contours was reported. The SIFT and Harris feature detection and matching algorithms identified point features in each image separately, then found matching features in the other image. The target registration error (TRE) was defined as the vector distance between matched features on the two image sets. Each deformation was evaluated based on comparison of average TRE and DSC.Results:Image feature analysis produced between 2000–9500 points for evaluation on the patient images. The average (± standard deviation) TRE for all patients was 3.3 mm (±3.1 mm), and the passing rate of TRE<3 mm was 60% on the images. The head/neck patient had the best average TRE (1.9 mm±2.3 mm) and the best passing rate (80%). The lung patient had the worst average TRE (4.8 mm±3.3 mm) and the worst passing rate (37.2%). DSC was not significantly correlated with either TRE (p=0.63) or passing rate (p=0.55).Conclusions:Feature matching provides a quantitative assessment of deformable image registration, with a large number of data points for analysis. The TRE of matched features can be used to evaluate the registration of many objects throughout the volume, whereas DSC mainly provides a measure of gross overlap.We have a research agreement with ViewRay Inc.

In this work, we construct a Kd–Octree hybrid index structure to organize the point cloud and generate patch-based feature descriptors at its leaf nodes. We propose a simple yet effective convolutional neural network, termed KdO-Net, with Kd–Octree based descriptors as input for 3D pairwise point cloud matching. The classic pipeline of 3D point cloud registration involves two steps, viz., the point feature matching and the globally consistent refinement. We focus on the first step that can be further divided into three parts, viz., the key point detection, feature descriptor extraction, and pairwise-point correspondence estimation. In practical applications, the point feature matching is ambiguous and challenging owing to the low overlap of multiple scans, inconsistency of point density, and unstructured properties. To solve these issues, we propose the KdO-Net for 3D pairwise point feature matching and present a novel nearest neighbor searching strategy to address the computation problem. Thereafter, our method is evaluated with respect to an indoor BundleFusion benchmark, and generalized to a challenging outdoor ETH dataset. Further, we have extended our method over our complicated and low-overlapped TUM-lab dataset. The empirical results graphically demonstrate that our method achieves a superior precision and a comparable feature matching recall to the prior state-of-the-art deep learning-based methods, despite the overlap being less than 30 percent. Finally, we implement quantitative and qualitative ablated experiments and visualization interpretations for illustrating the insights and behavior of our network.

The paper treats a problem of building an image mosaic or a 2D map from the sequence of images taken by an Unmanned Aerial Vehicle (UAV). The proposed algorithm is composed of the following parts: feature detection and extraction using Speeded Up Robust Features (SURF) descriptor, feature matching using Fast Approximate Nearest Neighbor Search (FANN) with automatic algorithm configuration, estimation of the optimal transform between images using RANdom SAmple Consensus (RANSAC) and finally creating a mosaic by stitching images together using the obtained transformations. Summarizing results obtained in two different scenarios, it can be concluded that the proposed algorithm exhibits near real time performance for automated UAV 2D aerial mapping.

Abstract. 3D reconstruction relies on accurate detection, extraction, description and matching of image features. This is even truer for complex architectural scenes that pose needs for 3D models of high quality, without any loss of detail in geometry or color. Illumination conditions influence the radiometric quality of images, as standard sensors cannot depict properly a wide range of intensities in the same scene. Indeed, overexposed or underexposed pixels cause irreplaceable information loss and degrade digital representation. Images taken under extreme lighting environments may be thus prohibitive for feature detection/extraction and consequently for matching and 3D reconstruction. High Dynamic Range (HDR) images could be helpful for these operators because they broaden the limits of illumination range that Standard or Low Dynamic Range (SDR/LDR) images can capture and increase in this way the amount of details contained in the image. Experimental results of this study prove this assumption as they examine state of the art feature detectors applied both on standard dynamic range and HDR images.