A Comparative Analysis of Feature Extraction Algorithms for Augmented Reality Applications

The purpose of this article is to explore effective image feature extraction algorithms in the context of big data, and to mine their potential information from complex image data. Based on the BRISK and SIFT algorithms, this paper proposes an image feature extraction and matching algorithm based on BRISK corner points. By combining the SIFT scale space and the BRISK algorithm, a new scale space construction method is proposed. The BRISK algorithm extracts the corner invariant features. Then, by using the improved feature matching method and eliminating the mismatching algorithm, the exact matching of the images is realized. A large number of experimental verifications were performed in the standard test Mikolajczyk image database and aerial image database. The experimental results show that the improved algorithm in this paper is an effective image matching algorithm. The highest accuracy of actual aerial image matching can reach 85.19%, and it can realize the actual aerial image matching that BRISK and SIFT algorithms cannot complete. The improved algorithm in this paper has the advantages of higher matching accuracy and strong robustness.

Image-based localization plays an important role in today's autonomous driving technologies. However, in large scale outdoor environments, challenging conditions, e.g., lighting changes or different weather, heavily affect image appearance and quality. As a key component of feature-based visual localization, image feature detection and matching deteriorate severely and cause worse localization performance. In this paper, we propose a novel method for robust image feature matching under drastically changing outdoor environments. In contrast to existing approaches which try to learn robust feature descriptors, we train a deep network that outputs the low-rank representations of the images where the undesired variations on the images are removed, and perform feature extraction and matching on the learned low-rank space. We demonstrate that our learned low-rank images largely improve the performance of image feature matching under varying conditions over a long period of time.

With the emerging techniques of wireless communication and cloud computing, large volumes of multimedia data are outsourced from resource constrained users to the cloud with abundant resource for both delegated storage and computation. Unfortunately, there is a risk of users’ image privacy leakage in the process of outsourcing to untrusted cloud. Most of the existing work achieved privacy-preserving image feature extraction and matching by using public key (fully) homomorphic encryption (FHE), but the heavy computational overhead and communication overhead cannot adapt to resource-constrained mobile devices. Other works disabled to realize image denoising in the encrypted domain or only focused on the scale-invariant feature transform (SIFT) descriptor that is inappropriate for position-sensitive feature extraction. To address these issues, in this paper, a privacy-preserving shape context based image denoising and matching protocol PPOIM with efficient outsourcing is proposed. Firstly, to improve the accuracy of image matching, a privacy-preserving image denoising scheme PPID is proposed without exploiting public key FHE. Then, based on PPID, a privacy-preserving image matching protocol PPOIM adopting shape context descriptor is devised, where two secure and efficient comparison and counting protocols in the encrypted domain are presented. All the original image privacy, query image privacy and image matching result privacy are well protected. Finally, formal security proof and extensive simulations on real-world data sets demonstrate the efficiency and practicability of our proposed PPOIM.

Image matching is a process to establish correspondence between primitives of two or more images that capturing a scene from different viewing position. Various image matching techniques using image features have been known in literature. Feature-based matching algorithm cannot tackle the problem of matching ambiguities easily. This study presents an image matching technique using the structural descriptions of image. Structural descriptions are consist of lines and interline relationship in the line-extracted image. Three conditions of inter-line relationship, namely ordering, intersection and co-linearity, were defined and derived in this study. The method involves the representation of the structural descriptions of image in relational graph and the matching between relational graphs to perform image matching. The methodology is consists of six steps: (1) input image, (2) line segment extraction from the image, (3) the interpretation and derivation of structural descriptions from the line-extracted image, (4) the construction of relational graph to represent the structural descriptions, (5) the derivation of association graph from relational graphs to perform relational graph matching, and (6) the searching of the largest maximal clique in the association graph to determine the best matching. Hence, image matching is transformed as a relational graph matching problem in this study. Experiments are carried out to evaluate the applicability of incorporating structural information into the image matching algorithm. The data is consisting of 14 pairs of stereo images. From the result obtained, it was found that the usage of structural information of image is only plausible for the matching of images of simple scene. The matching accuracy of images of complicated scene remains low even after the incorporation of inter-line descriptions into the image matching algorithm.

GAT (Global Affine Transformation) and GPT (Global Projection Transformation) matchings proposed by Wakahara and Yamashita calculate the optimal AT (affine transformation) and PT (2D projection transformation), respectively. These image matching criteria realize deformation-tolerant matchings by maximizing the normalized cross-correlation between a template and a GAT/GPT-superimposed image. In order to shorten the calculation time, Wakahara and Yamashita also proposed the acceleration algorithms for GAT/GPT matchings. Later on, Wakahara et al. proposed the enhanced GPT matching to calculate the optimal PT parameters simultaneously which overcomes the incompatibility during the matching process. Zhang et al. figured out that these matching techniques do not take account of the conservation of the L2 norm, and introduced norm normalization factors that realize accurate and stable matchings. All these correlation-based matching techniques are well suited for “whole-to-whole” image matching, but are weak in “whole-to-part” image matching being cursed by complex backgrounds and noise. This research firstly proposes simplified HOG patterns for the enhanced GPT matching with norm normalization to obtain the robustness against noise and background. Secondly, this research also proposes the acceleration algorithm for the proposed matching criterion by creating several reference tables. Experiments using the Graffiti dataset show that the proposed method exhibits an outstanding matching ability compared with the original GPT correlation matching and the well-known combination of SURF feature descriptor and RANSAC algorithm. Furthermore, the computational complexity of the proposed method is significantly reduced below double figures via the acceleration algorithm.

Aiming at the problems of large calculating scale and high complexity in Scale Invariant Feature Transform (SIFT) feature matching algorithm, this paper presents an improved SIFT feature matching algorithm based on quasi Euclidean distance.The traditional Euclidean distance can only calculate out the variance of the two images to the corresponding pixel, so when a slight shift or distortion occurs in the image, it may produce a large deviation.The quasi Euclidean distance instead of Euclidean distance is as the similarity measure of feature descriptors to improve the SIFT feature matching.It reduces the dimensions of SIFT feature vector to improve the efficiency of feature matching.Experimental results show that under the condition of keeping the image matching rate and algorithm robust, the method can not only improve the matching accuracy but also shorten the matching time.The new algorithm has better performance than traditional algorithm, it is possible and valid, which are useful for the fields of image recognition, image reconstruction, etc.

Reliable banknote recognition is critical for detecting counterfeit banknotes in ATMs and help visual impaired people. To solve this problem, it was implemented a computer vision system that can recognize multiple banknotes in different perspective views and scales, even when they are within cluttered environments in which the lighting conditions may vary considerably. The system is also able to recognize banknotes that are partially visible, folded, wrinkled or even worn by usage. To accomplish this task, the system relies on computer vision algorithms, such as image preprocessing, feature detection, description and matching. To improve the confidence of the banknote recognition the feature matching results are used to compute the contour of the banknotes using an homography that later on is validated using shape analysis algorithms. The system successfully recognized all Euro banknotes in 80 test images even when there were several overlapping banknotes in the same test image.

Image to image matching has been well studied in the computer vision community. Previous studies mainly focus on training a deep metric learning model matching visual patterns between the query image and gallery images. In this study, we show that pure image-to- image matching suffers from false positives caused by matching to local visual patterns. To alleviate this issue, we propose to leverage recent advances in vision-language pretraining research. Specifically, we introduce additional image-text alignment losses into deep metric learning, which serve as constraints to the image-to-image matching loss. With additional alignments between the text (e.g., product title) and image pairs, the model can learn concepts from both modalities explicitly, which avoids matching low-level visual features. We progressively develop two variants, a 3-tower and a 4-tower model, where the latter takes one more short text query input. Through extensive experiments, we show that this change leads to a substantial improvement to the image to image matching problem. We further leveraged this model for multimodal search, which takes both image and reformulation text queries to improve search quality. Both offline and online experiments show strong improvements on the main metrics. Specifically, we see 4.95% relative improvement on image matching click through rate with the 3-tower model and 1.13% further improvement from the 4-tower model.

Image matching as a data source for forest inventory – Comparison of Semi-Global Matching and Next-Generation Automatic Terrain Extraction algorithms in a typical managed boreal forest environment

The generic combinatorial algorithm for image matching with classes of projective transformations

The main aim of this research is presenting an automated image matching methodology being used in the field of medicine for inter- and intraobjectional image matching. This paper shows a different approach avoiding the standard procedures associated with performing four main steps of the registration process: feature detection, feature matching, mapping function design and image transformation with resampling, and replacing them with the fuzzy image concept combined with the use of similarity measures. This methodology has been implemented in MATLAB and tested on clinical T1- and T2-weighted magnetic resonance imaging (MRI) slices of the knee joint in coronal and sagittal plane.

Endoscopic image feature matching via motion consensus and global bilateral regression

High-dimensional image feature matching is an important part of many image matching based problems in computer vision which are solved by local invariant features. In this paper, we propose a new indexing/searching method based on Randomized Sub-Vectors Hashing (called RSVH) for high-dimensional image feature matching. The essential of the proposed idea is that the feature vectors are considered similar (measured by Euclidean distance) when the L2 norms of their corresponding randomized sub-vectors are approximately same respectively. Experimental results have demonstrated that our algorithm can perform much better than the famous BBF (Best-Bin-First) and LSH (Locality Sensitive Hashing) algorithms in extensive image matching and image retrieval applications.

This chapter presents a method for image matching which is based on the detection and characterisation of curvilinear regions. In a curvilinear region, the ratio between its average width and its total length should be less than a predefined threshold. Besides, left and right borders should be locally parallel, it should exist a geometric similarity around the region axis and the colour along this axis should be homogeneous. That is, they constitute particular image structures and, therefore, the method is restricted to scenes where these particular items are presented. On the contrary, curvilinear regions automatically deform with changing viewpoint as to keep on covering identical physical parts of a scene. For this reason, they can be used as distinguished regions. The shape contour of these curvilinear regions is characterised using the adaptive curvature function. Experimental results show that this shape descriptor is invariant to rotation and translation, and partially invariant to noise and skewing. Scaling invariance is achieved by employing an extra normalisation stage. Thus, this descriptor, plus the region colour and position, can be used to match curvilinear regions detected on the input image with those previously stored in a database. This is the basis of the correspondence algorithm described in this paper: the similarity index between two images is determined by the presence of the same set of curvilinear regions localised in similar positions. There are many directions for further research. One of this is the integration of several types of distinguished regions. As we commented above, the obligatory presence of these regions in the images is the main disadvantage of the proposed system. Besides, further work must be accomplished in the correspondence algorithm to employ the most similar region correspondences as ground control points. These points could be used to generate a fundamental matrix describing the geometric constraints between the two images. Thus, matches that did not agree with the majority solution could be removed.

At present, there are many methods to realize the matching of specified images with features, and the basic components include image feature point detection, feature description, and image matching. Based on this background, this article has done different research and exploration around these three aspects. The image feature point detection method is firstly studied, which commonly include image edge information-based feature detection method, corner information-based detection method, and various interest operators. However, all of the traditional detection methods are involved in problems of large computation burden and time consumption. In order to solve this problem, a feature detection method based on image grayscale information-FAST operator is used in this paper, which is combined with decision tree theory to effectively improve the speed of extracting image feature points. Then, the feature point description method BRIEF operator is studied, which is a local expression of detected image feature points based on descriptors. Since the descriptor does not have rotation invariance, the detection operator is endowed by a direction that is proposed in this paper, and then the local feature description is conducted on the feature descriptor to generate a binary string array containing direction information. Finally, the feature matching machine learning method is analyzed, and the nearest search method is used to find the nearest feature point pair in Euclidean distance, of which the calculation burden is small. The simulation results show that the proposed nearest neighbor search and matching machine learning algorithm has higher matching accuracy and faster calculation speed compared with the classical feature matching algorithm, which has great advantages in processing a large number of array images captured by the light field camera.