A single-frame visual gyroscope

ABSTRACTReliable feature point matching is a pivotal step in the integrated 3D reconstruction of aerial and ground images. Nonetheless, the substantial viewpoint and scale differences between aerial and ground images introduce significant challenges in feature point matching. To address this issue, this study presents a robust matching method for feature points from these two types of images, employing partitioned attention aggregation along with a coarse‐to‐fine hierarchical matching approach, enabling efficient matching of large‐scale aerial‐ground images even with hardware limitations. In the coarse matching phase, region segmentation is carried out on the ground image based on its depth map, which is computed using a monocular depth estimation network. Subsequently, a graph neural network serves as an intermediary for information exchange between features, and partitioned attention aggregation is performed while adhering to the constraints imposed by the segmented image regions. During this process, feature points are grouped to form distinct subgraphs, enhancing the structural similarity between corresponding graphs when there are significant depth and viewpoint changes. This also reduces redundant connections within graph neural networks, leading to efficient feature matching. Corresponding local patches are generated from the full‐sized aerial‐ground images using the initial matches, and further feature point extraction and matching from these patches yield accurate point matches. The experimental results indicate the proposed method performs better than state‐of‐the‐art matching methods. Specifically, compared with the best‐performing method among the comparative ones, our method enhances the average number of correct matches and the average matching precision by approximately 6‐fold and 36%, respectively.

Image stitching is an important technology in machine vision inspection whose algorithms focus on the detection and match of feature points. For the solar panel images achieved by machine vision, it's found that the available image stitching algorithms failed to detect enough valid feature points, which led to a large number of incorrect matching points. Here, we improve an algorithm to locate feature points and find dense match points. The proposed algorithm is based on SIFT. The improved algorithm (I-SIFT) can locate the position of feature points in every solar panel images, reducing the effect of invalid feature points on the experiment. Euclidean distance is used to preliminarily ensure the matching points, and then the relative horizontal position of every matching points is used to eliminate the mismatching points caused by the space similarity of the feature points, so that the matching accuracy is improved. Then the improved algorithm is used in the traditional Harris, SIFT and SURF stitching algorithm. The experimental results show that the success rate of I-SIFT algorithm can exceed 95% and the computation time has decreased by nearly 95% than that of the traditional algorithm. In conclusion, the improved algorithm implements accurately and rapidly stitching for solar panel images.

This paper describes a model-assisted system for reconstruction of 3D faces from a single consumer quality camera using a structure from motion approach. Typical multi-view stereo approaches use the motion of a sparse set of features to compute camera pose followed by a dense matching step to compute the final object structure. Accurate pose estimation depends upon precise identification and matching of feature points between images, but due to lack of texture on large areas of the face, matching is prone to errors. To deal with outliers in both the sparse and dense matching stages, previous work either relies on a strong prior model for face geometry or imposes restrictions on the camera motion. Strong prior models result in a serious compromise in final reconstruction quality and typically bear a signature resemblance to a generic or mean face. Model-based techniques, while giving the appearance of face detail, in fact carry this detail over from the model prior. Face features such as beards, moles, and other characteristic geometry are lost. Motion restrictions such as allowing only pure rotation are nearly impossible to satisfy by the end user, especially with a handheld camera. We significantly improve the robustness and flexibility of existing monocular face reconstruction techniques by introducing a deformable generic face model only at the pose estimation, face segmentation, and preprocessing stages. To preserve data fidelity in the final reconstruction, this generic model is discarded completely and dense matching outliers are removed using tensor voting: a purely data-driven technique. Results are shown from a complete end to end system.

The algorithms of image feature matching have good stability with scale, illumination change and rotation. They can be improved continuously in practical application. In the acquisition process of the image, the different conditions make the appearance of the same area seem different in different images. These conditions include the position, posture and performance of the camera and so on. However, the correspondence area of the image may have the projective distortion. The accuracy of image feature point's extraction and matching will be affected severely. In order to improve image quality, the original image will be processed to facilitate the application of the following steps. In this paper, the histogram equalization method is adopted to preprocess the original image to enhance the useful information. Then the preprocessed image is used respectively to the SIFT algorithm and the ASIFT algorithm to achieve the extraction and matching of the image feature points. The purpose of image preprocessing is to increase the matching number of image feature point, and improve the matching rate of image feature. The experimental results show that the image processed by the histogram equalization can improve obviously the matching number of the image feature point.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Blurred image restoration: A fast method of finding the motion length and angle

At present, there are many methods to realize the matching of specified images with features, and the basic components include image feature point detection, feature description, and image matching. Based on this background, this article has done different research and exploration around these three aspects. The image feature point detection method is firstly studied, which commonly include image edge information-based feature detection method, corner information-based detection method, and various interest operators. However, all of the traditional detection methods are involved in problems of large computation burden and time consumption. In order to solve this problem, a feature detection method based on image grayscale information-FAST operator is used in this paper, which is combined with decision tree theory to effectively improve the speed of extracting image feature points. Then, the feature point description method BRIEF operator is studied, which is a local expression of detected image feature points based on descriptors. Since the descriptor does not have rotation invariance, the detection operator is endowed by a direction that is proposed in this paper, and then the local feature description is conducted on the feature descriptor to generate a binary string array containing direction information. Finally, the feature matching machine learning method is analyzed, and the nearest search method is used to find the nearest feature point pair in Euclidean distance, of which the calculation burden is small. The simulation results show that the proposed nearest neighbor search and matching machine learning algorithm has higher matching accuracy and faster calculation speed compared with the classical feature matching algorithm, which has great advantages in processing a large number of array images captured by the light field camera.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

Converting tower-mounted videos from perspective to orthographic view is beneficial for their integration with maps and remote sensing images and can provide a clearer and more real-time data source for earth observation. This paper addresses the issue of low geometric accuracy in orthographic video generation by proposing a method that incorporates 3D GIS view matching. Firstly, a geometric alignment model between video frames and 3D GIS views is established through camera parameter mapping. Then, feature point detection and matching algorithms are employed to associate image coordinates with corresponding 3D spatial coordinates. Finally, an orthographic video map is generated based on the color point cloud. The results show that (1) for tower-based video, a 3D GIS constructed from publicly available DEMs and high-resolution remote sensing imagery can meet the spatialization needs of large-scale tower-mounted video data. (2) The feature point matching algorithm based on deep learning effectively achieves accurate matching between video frames and 3D GIS views. (3) Compared with the traditional method, such as the camera parameters method, the orthographic video map generated by this method has advantages in terms of geometric mapping accuracy and visualization effect. In the mountainous area, the RMSE of the control points is reduced from 137.70 m to 7.72 m. In the flat area, it is reduced from 13.52 m to 8.10 m. The proposed method can provide a near-real-time orthographic video map for smart cities, natural resource monitoring, emergency rescue, and other fields.

Feature point extraction and matching are the key modules in the front-end of visual simultaneous localization and mapping (SLAM), which determines the accuracy of locating and map Nbuilding robots. To obtain a good performance, the image extraction algorithm being applied should have high efficiency and robustness to the changes of the surrounding environment. In this paper, three widely used feature extraction algorithms, namely Scale-invariant feature transform (SIFT), speed up robust features (SURF) and oriented FAST, and rotated BRIEF (ORB), are studied. The basic concepts and details of the three algorithms are introduced. Then the execution time of the three algorithms is compared to evaluate their efficiency. Meanwhile, the key points matching rate of the three algorithms are compared under various transformations and distortions. The experiment results show that ORB has higher efficiency than SIFT and SURF. However, SIFT has better robustness to image rotation, illumination change, scaling transformation, Gaussian noise and salt-and-pepper noise. In addition, the performance of the ORB-SLAM system is tested on different datasets. It has great ability in camera trajectory estimation and map building and therefore is suitable for deploying in industry.

Abstract. Satellite image resolution has evolved to daily revisit and sub-meter GSD. Main targets of previous remote sensing were forest, vegetation, damage area by disasters, land use and land cover. Developments in satellite images have brought expectations on more sophisticated and various change detection of objects. Accordingly, we focused on unsupervised change detection of small objects, such as vehicles and ships. In this paper, existing change detection methods were applied to analyze their performances for pixel-based and feature-based change of small objects. We used KOMPSAT-3A images for tests. Firstly, we applied two change detection algorithms, MAD and IR-MAD, which are most well-known pixel-based change detection algorithms, to the images. We created a change magnitude map using the change detection methods. Thresholding was applied to determine change and non-change pixels. Next, the satellite images were transformed as 8-bit images for extracting feature points. We extracted feature points using SIFT and SURF methods to analyze feature-based change detection. We assumed to remove false alarms by eliminating feature points of non-changed objects. Therefore, we applied a feature-based matcher and matched feature points on identical image locations were eliminated. We used non-matched feature points for change/non-change analysis. We observed changes by creating a 5x5 size ROI around extracted feature points in the change/non-change map. We determined that change has occurred on feature points if the rate of change pixels with ROI was more than 50%. We analyzed the performance of pixel-based and feature-based change detection using ground truths. The F1-score, AUC value, and ROC were used to compare the performance of change detection. Performance showed that feature-based approaches performed better than pixel-based approaches.

To resolve multi-sensor remote sensing images registration, a new approach which combines feature and regional similarity measure is proposed. This approach firstly adopts SIFT algorithm to match feature points and constructs initial affine transformation function; next uses normalized cross-correlation coefficient (NCC) to define regional similarity measure, and improves NCC with initial affine transformation parameters in order to get further matching points from those unmatched in SIFT algorithm. Then, after gross filtering the effective matching feature points, builds the model for image registration by all the matching feature points. This method combines the advantages of feature matching and regional matching, and solves the problem that the multi-sensor remote sensing images are difficult to match correctly because of geometric and radiometric differences. Experiments show that this method has strong robustness, and are of higher registration accuracy than single registration method.

We address a core problem of computer vision: Detection and description of 2D feature points for image matching. For a long time, hand-crafted designs, like the seminal SIFT algorithm, were unsurpassed in accuracy and efficiency. Recently, learned feature detectors emerged that implement detection and description using neural networks. Training these networks usually resorts to optimizing low-level matching scores, often pre-defining sets of image patches which should or should not match, or which should or should not contain key points. Unfortunately, increased accuracy for these low-level matching scores does not necessarily translate to better performance in high-level vision tasks. We propose a new training methodology which embeds the feature detector in a complete vision pipeline, and where the learnable parameters are trained in an end-to-end fashion. We overcome the discrete nature of key point selection and descriptor matching using principles from reinforcement learning. As an example, we address the task of relative pose estimation between a pair of images. We demonstrate that the accuracy of a state-of-the-art learning-based feature detector can be increased when trained for the task it is supposed to solve at test time. Our training methodology poses little restrictions on the task to learn, and works for any architecture which predicts key point heat maps, and descriptors for key point locations.

With the continuous development and improvement of sensors, sensor technology is also constantly making breakthroughs. It is also widely used in various fields of medicine today. This article summarizes the practice of two types of SLAM (Simultaneous Localization and Mapping) in medicine. The detection and matching of human luminal feature points under SLAM were studied so as to reconstruct the human internal environment so that doctors can obtain three-dimensional reconstruction feedback within the surgical range and improve the safety of surgery. And for different surgical environments, the same SLAM is improved on the basis of innovation, which is more suitable for different scenarios. Three algorithms are described in this article for taking images of the inside of the abdominal cavity and collecting data based on the images using different algorithms. Based on the collected data, the internal abdominal cavity is reconstructed in 3D. Finally, a three-dimensional visualization system for the abdominal cavity was constructed. It has important research significance in the field of medical auxiliary research.

Image stitching refers to the combination of multiple images with a certain over lappingarea rate acquired by the image acquisition device into one image, which has a full picture of each image. Image stitching technology is an important part in the field of digital image processing. With the increasing requirements for image quality and the limited resolution of ordinary cameras, it is impossible to obtain large angle of view and clear image at the same time. In this paper, the key technologies involved in pavement crack image stitching are intensively studied, including image feature point detection and matching, internal and external camera parameter estimation and optimization, and image fusion. The work done in this paper and the main innovations include: (1)Two main feature detection and description algorithms, SIFT algorithm and ORBalgorithm, are studied. In view of the fact that the SIFT algorithm has redundancy when the Gaussian pyramid is established, this paper appropriately reduces the number of groups and maintains the number and stability of feature point detection. Combined with the advantages and disadvantages of the improved SIFT algorithm and ORB algorithm, and the performance of the two algorithms in the detection of pavement cracks, the improvedSIF'T algorithm is used as the feature point detection algorithm, and rBRlEF algorithm for describing the feature points in the ORB algorithm is used as the feature point description algorithm. The description algorithm is called the BSIF'T algorithm. Experiments show that the robustness of BSIFT algorithm is similar to that of SIFT algorithm, which is better than ORB algorithm and faster than SIF'T algorithm. (2)Image fiusion algorithms are studied, including best seam-line search and image fusion. Aiming at the ghost phenomenon that may occur in overlapping regions, this paper proposes a best seam-line search algorithm based on distance transform.